Compare search engine for JAVA
This repository was created to test implementation of popular search engines with directly MySQL loading for JAVA Spring application.
Sources for data loading:
- Hibernate Search
- Elasticsearch
- MySQL
Hello folks :)
I was thinking lately: if I have logic for searching data in frontend and JAVA for backend, which technology should be used to find data? In this case some search engine would be a good idea. But is it easy to implement? I wanted to try a couple things and compare, which option is the best.
Learning by doing I understood how it works and I´d like to share this experience.
First of all we need to create our JAVA application. I used the "Initializr" from Spring to get basic configuration for my project. Spring Initializr
Spoiler!:-D
My end configuration looks like this (Java version 18):
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.3</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.tutorial</groupId>
<artifactId>search</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>search</name>
<description>Tutorial project for search engine</description>
<properties>
<java.version>18</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate.search</groupId>
<artifactId>hibernate-search-mapper-orm</artifactId>
<version>6.1.1.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate.search</groupId>
<artifactId>hibernate-search-backend-lucene</artifactId>
<version>6.1.1.Final</version>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-elasticsearch</artifactId>
<version>4.0.0.RELEASE</version>
</dependency>
<dependency>
<groupId>co.elastic.clients</groupId>
<artifactId>elasticsearch-java</artifactId>
<version>8.3.1</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.12.3</version>
</dependency>
<dependency>
<groupId>jakarta.json</groupId>
<artifactId>jakarta.json-api</artifactId>
<version>2.0.1</version>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
Then just install all dependencies and we are good to move forward.
mvn install
or mvn package
Ok, now we need our database and some data. To fetch data we need to have/install mysql to local machine. To make it easier, we could install Workbench
to get also GUI. We need to put information about database(username, password, database url) in application.properties as well.
spring.datasource.url={url_path}
spring.datasource.username={database_username}
spring.datasource.password={database_password}
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
Now just type Book model. I also added the annotations we need for our search engines.
A table named book should exist in your database and should include some data for test.
@Entity
@Indexed
@Table(name = "book")
@Document(indexName = "books", type = "book")
public class Book {
@Id
private int id;
@FullTextField
@Field(type = FieldType.Text, name = "name")
private String name;
@FullTextField
@Field(type = FieldType.Text, name = "isbn")
private String isbn;
public int getId() {
return id;
}
public String getIsbn() {
return isbn;
}
public String getName() {
return name;
}
public void setIsbn(String isbn) {
this.isbn = isbn;
}
public void setName(String name) {
this.name = name;
}
}
Now to get response from request we need Repository class.
public interface BookRepository extends PagingAndSortingRepository<Book, Integer>
{
List<Book> findByName(String name);
List<Book> findByIsbn(String isbn);
}
And now to get data from url we need a controller. I keep it very simple just to get response and check if data was loaded.
@RestController
@RequestMapping(value = "book")
public class BookController {
@Autowired
BookRepository bookRepository;
@GetMapping("/database")
ResponseEntity<Iterable<Book>> getBooksFromDatabase(@RequestParam("query") String query)
{
Iterable<Book> booksFromDatabase = bookRepository.findByIsbn(query);
return ResponseEntity.ok(booksFromDatabase);
}
}
Ok, it seems to be done with database.
Was not so hard, but this is not most flexible option to interact with a searching flow.
Let´s take a look at the searching engines. We will start with very powerful technology in JAVA.
First we need to create configuration file.
@Component
public class BookIndexConfiguration implements CommandLineRunner
{
@PersistenceContext
EntityManager entityManager;
@Override
@Transactional(readOnly = true)
public void run(String ...args) throws Exception
{
SearchSession searchSession = Search.session(entityManager);
MassIndexer indexer = searchSession.massIndexer(Book.class);
indexer.startAndWait();
}
}
Over here the indexer from hibernate will create index with data from our model once the application was started. To explore books in index we also need service class. Ok, let's create one.
@Service
public class BookSearchService {
@PersistenceContext
EntityManager entityManager;
@Transactional(readOnly = true)
public Iterable<Book> search(Pageable pageable, String query)
{
SearchSession session = Search.session(entityManager);
SearchResult<Book> result = session.search(Book.class)
.where(
f -> f.match().
fields("name", "isbn").
matching(query)
)
.fetch((int) pageable.getOffset(), pageable.getPageSize())
;
return result.hits();
}
}
We have search function that loads a current session and checks in index, if the query matches to the name or isbn. If some objects were found, it will be returned as Iterable of Book objects
And thats it! Was it even easier that typical database call?
Now we need to configure a different search engine.
Ok, now we need a little bit more configuration or maybe not. Just like in the case with MySQL, we should install Elasticsearch. I will use Docker image to get the client. Alternatively you can use one of the methods under the following link Elasticsearch install
version: '3.7'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.3.2
container_name: elasticsearch
environment:
- xpack.security.enabled=false
- discovery.type=single-node
ports:
- 9200:9200
- 9300:9300
Important
For testing purposes I disabled the security mode. You should not do this if you deploy your application into the live system.
After Elasticsearch has been installed and started, we can create one more configuration file with Client.
public class BookElasticsearchClient {
private ElasticsearchClient client;
public ElasticsearchClient getClient()
{
if (client == null) {
initClient();
}
return client;
}
private void initClient()
{
RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
client = new ElasticsearchClient(transport);
}
}
If you use authentication with username and password, you need to configure these as well. The information on how to do this, is under the following link: Authentication
Aaaand now service class for searching.
@Configuration
public class BookElasticsearchService {
BookElasticsearchClient client = new BookElasticsearchClient();
private static final String BOOK_INDEX = "books";
public void createBooksIndexBulk(final List<Book> books)
{
BulkRequest.Builder builder = new BulkRequest.Builder();
for (Book book : books)
{
builder.operations(op -> op
.index(index -> index
.index(BOOK_INDEX)
.id(String.valueOf(book.getId()))
.document(book)
)
);
try {
client.getClient().bulk(builder.build());
} catch (ElasticsearchException exception) {
exception.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
public List<Book> findBookByIsbn(String query)
{
List<Book> books = new ArrayList<>();
try {
SearchResponse<Book> search = client.getClient().search(s -> s
.index(BOOK_INDEX)
.query(q -> q
.match(t -> t
.field("isbn")
.query(query))),
Book.class);
List<Hit<Book>> hits = search.hits().hits();
for (Hit<Book> hit: hits) {
books.add(hit.source());
}
return books;
} catch (ElasticsearchException exception) {
exception.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
}
These are two very important functions. One of them is to create an index and put data into a document, another one have been used to get information about book from isbn.
And last but not least, we need a repository class for Elasticsearch.
public interface BookElasticsearchRepository extends ElasticsearchRepository<Book, String> {}
In the end we need to complete our controller to send requests.
@RestController
@RequestMapping(value = "book")
public class BookController {
@Autowired
BookSearchService searchService;
@Autowired
BookRepository bookRepository;
@Autowired
BookElasticsearchService bookElasticsearchService;
@GetMapping("/database")
ResponseEntity<Iterable<Book>> getBooksFromDatabase(@RequestParam("query") String query)
{
Iterable<Book> booksFromDatabase = bookRepository.findByIsbn(query);
return ResponseEntity.ok(booksFromDatabase);
}
@GetMapping("/hibernate")
ResponseEntity<Iterable<Book>> getBooksFromHibernate(Pageable pageable, @RequestParam("query") String query)
{
Iterable<Book> booksFromHibernate = searchService.search(pageable, query);
return ResponseEntity.ok(booksFromHibernate);
}
@GetMapping("/elasticsearch")
ResponseEntity<Iterable<Book>> getBooksFromElasticsearch(@RequestParam("query") String query)
{
Iterable<Book> booksFromElasticSearch = bookElasticsearchService.findBookByIsbn(query);
return ResponseEntity.ok(booksFromElasticSearch);
}
}
One more important thing before starting the application:
we need to add the annotations(see below) in the main ApplicationClass, to make it work with multiple search engine technologies in one application.
@EnableElasticsearchRepositories(basePackageClasses = BookElasticsearchRepository.class)
@EnableJpaRepositories(excludeFilters = @ComponentScan.Filter(
type = FilterType.ASSIGNABLE_TYPE, value = BookElasticsearchRepository.class
))
@SpringBootApplication
public class SearchApplication {
public static void main(String[] args) {
SpringApplication.run(SearchApplication.class, args);
}
}
Let´s test all the things together.
I also had to create an index in my very first time. I called the function createBooksIndexBulk from BookElasticsearchService. There is also an option to call index directly in Elasticsearch console. How to do this is described here: Bulk Api
I send couple requests (in my database i have an isbn=123test):
http://localhost:8080/book/hibernate?query=123test
http://localhost:8080/book/database?query=123test
http://localhost:8080/book/elasticsearch?query=123test
For each request I got a right response. Mission completed.
Thanks a lot for reading.
Additional resources that I used:
You can also check my Github to get the whole code
Compare search engine for JAVA
This repository was created to test implementation of popular search engines with directly MySQL loading for JAVA Spring application.
Sources for data loading: