Implement Search specification for pagination

Question

I implemented this Page request:

@GetMapping
public PageImpl<ProductFullDTO> list(@RequestParam(name = "page", defaultValue = "0") int page,
                                     @RequestParam(name = "size", defaultValue = "10") int size) {
    PageRequest pageRequest = PageRequest.of(page, size);
    PageImpl<ProductFullDTO> result = productRestService.page(pageRequest);
    return result;
}

public PageImpl<ProductFullDTO> page(PageRequest pageRequest){

        Page<Product> pageResult = productService.findAll(pageRequest);
        List<ProductFullDTO> result = pageResult
                .stream()
                .map(productMapper::toFullDTO)
                .collect(toList());

        return new PageImpl<ProductFullDTO>(result, pageRequest, pageResult.getTotalElements());
    }

    public Page<Product> findAll(PageRequest pageRequest) {
        return this.dao.findAll(pageRequest);
    }

@Repository
public interface ProductRepository extends JpaRepository<Product, Integer>, JpaSpecificationExecutor<Product> {

    Page<Product> findAllByTypeIn(Pageable page, String... types);

    Page<Product> findAll(Pageable page);
}

The question is how to implement search functionality for this Page request? I would like to send params like type and dateAdded into GET params and return filtered result?

Answer 1

You can implement the desired behavior in several ways.

Let's forget about the paging parameters for a moment.

First, define a POJO that agglutinates the different fields required as search criteria. For example:

public class ProductFilter {
  private String type;
  private LocalDate dateAdded;

  // Setters and getters omitted for brevity
}

This information should be the entry point to your Controller search method.

Please, although a @GetMapping is perfectly suitable, consider use a @PostMapping instead, mainly in order to avoid possible URL length problems ¹:

@PostMapping
public PageImpl<ProductFullDTO> list(ProductFilter filter) {
    //...
}

Or to consume your search criteria as JSON payload and @RequestBody in your controller:

@PostMapping
public PageImpl<ProductFullDTO> list(@RequestBody ProductFilter filter) {
    //...
}

Now, how to deal with pagination related information at Controller level? You have several options as well.

• You can include the necessary fields, page and size, as new fields in ProductFilter.

public class ProductFilter {
  private String type;
  private LocalDate dateAdded;
  private int page;
  private int size;

  // Setters and getters omitted for brevity
}

• You can create a common POJO to deal with the pagination fields and extend it in your filters (maybe you can work directly with PageRequest itself although I consider a simpler approach to create your own POJO for this functionality in order to maintain independence from Spring - an any other framework - as much as possible):

public class PagingForm {
  private int page;
  private int size;
  //...
}

public class ProductFilter extend PagingForm {
  private String type;
  private LocalDate dateAdded;

  // Setters and getters omitted for brevity
}

• You can (this is my preferred one) maintain your filter as is, and modify the url to include the paging information. This is especially interesting if you are using @RequestBody.

Let's consider this approach to continue with the service layer necessary changes. Please, see the relevant code, pay attention to the inline comments:

@PostMapping
public PageImpl<ProductFullDTO> list(
  @RequestParam(name = "page", defaultValue = "0") int page,
  @RequestParam(name = "size", defaultValue = "10") int size,
  @RequestBody ProductFilter filter
) {
    PageRequest pageRequest = PageRequest.of(page, size);
    // Include your filter information
    PageImpl<ProductFullDTO> result = productRestService.page(filter, pageRequest);
    return result;
}

Your page method can look like this ²:

public PageImpl<ProductFullDTO> page(final ProductFilter filter, final PageRequest pageRequest){
  // As far as your repository extends JpaSpecificationExecutor, my advice
  // will be to create a new Specification with the appropriate filter criteria
  // In addition to precisely provide the applicable predicates, 
  // it will allow you to control a lot of more things, like fetch join
  // entities if required, ...
  Specification<Product> specification = buildProductFilterSpecification(filter);
          
  // Use now the constructed specification to filter the actual results
  Page<Product> pageResult = productService.findAll(specification, pageRequest);
  List<ProductFullDTO> result = pageResult
                .stream()
                .map(productMapper::toFullDTO)
                .collect(toList());

  return new PageImpl<ProductFullDTO>(result, pageRequest, pageResult.getTotalElements());
}

You can implement the suggested Specification over Product as you need. Some general tips:

• Always define the Specification in a separate class in a method defined for the task, it will allow you to reuse in several places of your code and favors testability.
• If you prefer, in order to improve the legibility of the code, you can use a lambda when defining it.
• To identify the different fields used in your predicate construction, prefer always the use of metamodel classes instead of Strings for the field name. You can use the Hibernate Metamodel generator to generate the necessary artifacts.
• In your specific use case, do not forget to include the necessary sort definition to provide consistent results.

In summary, the buildProductFilterSpecification can look like this:

public static Specification<Product> buildProductFilterSpecification(final ProjectFilter filter) {
  return (root, query, cb) -> {

    final List<Predicate> predicates = new ArrayList<>();

    final String type = filter.getType();
    if (StringUtils.isNotEmpty(type)) {
      // Consider the use of like on in instead
      predicates.add(cb.equal(root.get(Product_.type), cb.literal(type)));
    }

    // Instead of dateAdded, please, consider a date range, it is more useful
    // Let's suppose that it is the case
    final LocalDate dateAddedFrom = filter.getDateAddedFrom();
    if (dateAddedFrom != null){
      // Always, specially with dates, use cb.literal to avoid underlying problems    
      predicates.add(
        cb.greaterThanOrEqualTo(root.get(Product_.dateAdded), cb.literal(dateAddedFrom))
      );
    }

    final LocalDate dateAddedTo = filter.getDateAddedTo();
    if (dateAddedTo != null){
      predicates.add(
        cb.lessThanOrEqualTo(root.get(Product_.dateAdded), cb.literal(dateAddedTo))
      );
    }

    // Indicate your sort criteria
    query.orderBy(cb.desc(root.get(Product_.dateAdded)));

    final Predicate predicate = cb.and(predicates.toArray(new Predicate[predicates.size()]));

    return predicate;
  };
}

---

¹ As @blagerweij pointed out in his comment, the use of POST instead of GET will prevent in a certain way the use of caching at HTTP (web server, Spring MVC) level.

Nevertheless, it is necessary to indicate two important things here:

• one, you can safely use a GET or POST HTTP verb to handle your search, the solution provided will be valid with both verbs with minimal modifications.
• two, the use of one or other HTTP method will be highly dependent in your actual use case:
  • If, for example, you are dealing with a lot of parameters, the URL limit can be a problem if you use GET verb. I faced the problem myself several times.
  • If that is not the case and your application is mostly analytical, or at least you are working with static information or data that does not change often, use GET, HTTP level cache can provide you great benefits.
  • If your information is mostly operational, with numerous changes, you can always rely on server side cache, at database or service layer level, with Redis, Caffeine, etcetera, to provide caching functionally. This approach will usually provide you a more granular control about cache eviction and, in general, cache management.

---

² @blagerweij suggested in his comment the use of Slice as well. If you do not need to know the total number of elements of your record set - think, for example, in the typical use case in which you scroll a page and it fires the fetch of a new record set, in a fixed amount, that will be shown in the page - the use of Slice instead of Page can provide you great performance benefits. Please, consider review this SO question, for instance.

In a typical use case, in order to use Slice with findAll your repository cannot extend JpaRepository because in turn it extends PagingAndSortingRepository and that interface already provides the method you are using now, findAll(Pageable pageable).

Probably you can expend CrudRepository instead and the define a method similar to:

Slice<Product> findAll(Pageable pageable);

But, I am not sure if you can use Slices with Specifications: please, see this Github issue: I am afraid that it is still a WIP.

Answer 2

There are two ways to achieve this:

1. Examples. Simple but can only be used for simple searches

    @GetMapping("products")
    public Page<ProductFullDTO> findProducts(Product example,
Pageable pageable) {
   return productService.findProductsByExample(example, pageable);
}

To be able to search by example, you need make sure your ProductRepository interface extends from QueryByExampleExecutor:

public interface ProductRepository 
  extends JpaRepository<Product, Long>, QueryByExampleExecutor<Product> {}

In your service method, you can simply pass the example object:

public Page<ProductFullDTO> findProductsByExample(Product exampleProduct, Pageable pageable) {
    return productRepository.findByExample(Example.of(exampleProduct), pageable).map(ProductFullDTO::new);
}

See https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#query-by-example.matchers for more information

2. Using JPA Specification. This is useful in case the search parameters are more complex (e.g. when you need to join other entities)

    @GetMapping("products")
    public Page<ProductFullDTO> findProducts(@Valid ProductSearchParams params,
Pageable pageable) {
   return productService.findProducts(params, pageable);
}

The ProductSearchParams class could look like this:

public class ProductSearchParams {
   private String type;
   private Date dateAdded;
}

To use the search params, we need a Specification. To use this, you need make sure your ProductRepository interface extends from JpaSpecificationExecutor:

public interface ProductRepository 
  extends JpaRepository<Product, Long>, JpaSpecificationExecutor<Product> {}

Since the Specification interface has only one method, we can use a lambda in our service:

public Page<ProductFullDTO> findProducts(ProductSearchParams params, Pageable pageable) {
    Specification<Product> spec = (root, query, cb) -> {
        List<Predicate> predicates = new ArrayList<>();
        if (params.getType() != null) {
            predicates.add(cb.equal(root.get("type"), params.getType()));
        }
        if (params.getDateAdded() != null) {
            predicates.add(cb.greaterThan(root.get("dateAdded"), params.getDateAdded()));
        }
        return cb.and(predicates.toArray(new Predicate[predicates.size()]));
    };
    return productRepository.findAll(spec, pageable).map(ProductFullDTO::new);
}

Using Specification is more powerful, but also a bit more work. See https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#specifications for information on using Specifications. Using Specifications also allows you to write really elegant expressions, e.g. customerRepository.findAll( isLongTermCustomer().or(hasSalesOfMoreThan(amount)));

Answer 3

You want to make an implementation of the Specification interface that you can pass to your JPA repository. The implementation of the toPredicate method is where you can build your selection logic.

There is an overload to in the JpaSpecificationExecutor:

Page<T> findAll(@Nullable Specification<T> spec, Pageable pageable); 

There's a good example of using the Specification/Repository interfaces here.