turkraft / springfilter Goto Github PK

Hi @torshid,
I have a problem with this rest api example:
[http://localhost:8080/schedaSempl?q=identificazione.collezione:'D'arco']
the error is in the input text using apex.
Console error:
com.turkraft.springfilter.exception.OutOfInputException: Reached end of input before end of text
at com.turkraft.springfilter.compiler.token.input.TextMatcher.match(TextMatcher.java:33)
at com.turkraft.springfilter.compiler.token.input.TextMatcher.match(TextMatcher.java:9)
at com.turkraft.springfilter.compiler.Tokenizer.tokenize(Tokenizer.java:45)
at com.turkraft.springfilter.compiler.Parser.parse(Parser.java:31)
at com.turkraft.springfilter.SpringFilterUtils.getFilterFromInputs(SpringFilterUtils.java:43)
at com.turkraft.springfilter.boot.SpecificationFilterArgumentResolver.getSpecification(SpecificationFilterArgumentResolver.java:44)
at com.turkraft.springfilter.boot.SpecificationFilterArgumentResolver.resolveArgument(SpecificationFilterArgumentResolver.java:37)
at org.springframework.web.method.support.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:121)
at org.springframework.web.method.support.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:167)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:134)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:878)

Can you help me, please?

As stated in the title, predicates on @OneToMany and @ManyToMany relations should use exists coupled with subqueries instead of using inner joins.

This would be more logical and would also prevent duplicate results.

@OneToOne and @ManyToOne relations can continue to use left joins.

While playing with comparators, i found a problem with the ~ comparator.
The query LIKE syntax doesn't work as expected and the problem is not the query itself, but the expression:
catalog.name ~ 'electronic%'.
Without a proper url encoding of the % character to the compliant %25, the query fails because the % doesn't get past the request.
Looking at the rsql example from https://www.baeldung.com/rest-api-search-language-rsql-fiql, you can see that the LIKE comparator is the * in the query string.
We all know that url encoding is a must have, but maybe it could be easily avoided this way.

Anyway, i wanted to give you my congratulations for this library, i've been looking for a REST API with a model driven approach fully integrated with Spring and this is the first in months of research which allows to setup a nice query language with JPA 👯‍♂️ !

It would be nice if you could search for Elements in a collection.

Hey!

I went thru the documentation and didn't find any solution.
Is there any chance to cast a numerical field to the string and make the 'like'-query?
For instance, there is an integer field and I want to get the result by using query <field_name>~'%3007%'

Thanks.

For like queries, the value for CASE_SENSITIVE_LIKE_OPERATOR is false by default, which generated queries like

select * from users where upper(name) like "a%";

Having an upper function prohibits the query from using indexes and slows the query considerably. I had to set CASE_SENSITIVE_LIKE_OPERATOR as true to stop the query from using the upper function and make use of the indexes I had defined.

ExpressionGeneratorParameters.CASE_SENSITIVE_LIKE_OPERATOR = true

Can I know the rationale behind having it as false by default?

Thanks for this good library!
It would be great to have support for security expressions similarly to Spring in @query (https://docs.spring.io/spring-security/site/docs/4.0.1.RELEASE/reference/htmlsingle/#data).

maritalStatus or payslips is empty end up as ( generatedAlias0.maritalStatus = true ) or ( generatedAlias0.payslips = true ) is null

The method
default Page<T> findAll(@Nullable Document doc, Pageable pageable)
should return a PageImpl object with informations regarding pageable objects (count and page number).
In order to achieve this, the default implementation should be the following:

@Override
    default Page<T> findAll(Document doc, Pageable pageable) {
        final Query query = BsonGeneratorUtils.getQueryFromDocument(doc).with(pageable);
        final long count = getMongoOperations().count(query, getMetadata().getJavaType());
        final List<T> content = getMongoOperations().find(query, getMetadata().getJavaType());
        return new PageImpl<>(content, pageable, count);
    }

I tried and tested it, so it can be directly implemented.

Ok so, I'm opening this issue for further investigations over the ability to query a date field via MongoRepository.

I've found this project to start from:
https://github.com/RutledgePaulV/rsql-mongodb

This is using the Visitor pattern.
A bit different, but same result. Maybe it could be inspiring.

Hi,

I'm just trying to filter entities on a fields containing number but I get this error:
2021-11-03 21:33:48.454 ERROR 19900 --- [nio-8080-exec-5] o.a.c.c.C.[.[.[.[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [/sffCasesPartsLists] threw exception [Request processing failed; nested exception is com.turkraft.springfilter.exception.BadFilterSyntaxException: mismatched input '25' expecting {<EOF>, '~', ':', '!', '>', '>:', '<', '<:', AND, OR, IN, IS_NULL, IS_NOT_NULL, IS_EMPTY, IS_NOT_EMPTY, '.'}] with root cause

I'm using this filter:
caseInformation.drive25.min >: 2

So Could you please add support for any valid Java variable name.

Thanks!

By the way thanks for the amazing work on this library!

Hey!

Entity class can map Db field of timestamp format to Date field.
It would be great to have the possibility to filter by that kind of field not only by date but by date and time both.
So for such cases, it would be needed to set the Date pattern as dd-MM-yyyy'T'HH:mm:ss.

Could you improve this?
Thanks.

Hi!

I have problem while integrating this library with my project (Spring boot 2.7.2, Spring Data Rest (last stable)).
I need to write custom controller for search logic but with all of advantages of the Spring Data Rest library and this library, so I used @RepositoryRestController annotation - and as a result I got error "No primary or default constructor found for interface org.springframework.data.jpa.domain.Specification".
Signature of Controller method looks like the example https://github.com/turkraft/spring-filter :

@RequestMapping(method = RequestMethod.GET, value = "/someUrl")
public @responsebody ResponseEntity search(@filter Specification specification);

Will be grateful for any idea!

If I have two or more fields, be able to search in all of them at the same time (or in the selected fields).

My problem is the next one: You have a table with Names and Surnames in two Columns (or paramters in the returned object):

NAME SURNAME

JOHN DOE
JOHN SMITH
PAMELA DOE
PAMELA SMITH

Now if I search "Smith", with AND/OR operators, the result will be "John Smith, Pamela Smith", but if I want to be more specific and I search "John%Smith", (name~'John%Smith' AND surname~'John%Smith') the result will be all of them, so the results are worst as more parameters I search, because more words can appear on those fields.

In my proposition, I think something about: "globalFilter:name,surname~'John%Smith'" only the "John Smith will be returned, as it is the only one which contains the search term in the parameters selected.

I don't know if I explained myself enough, english is not my native language.

Your project has made me want to build APIs again with Spring, It's wonderful and this is the only think that I really missed, I hope it's possible to code... Thank you!

Hey :)
I've been very busy lately, couldn't follow the project as I loved to.
Anyway, while working with the library I had to face the problem of building like filters that had to be case sensitive in some circumstances.
The FilterParameters.CASE_SENSITIVE_LIKE_OPERATOR cannot be changed directly without affecting all the application behaviour, or at least it cannot be done without adding some logic, like a servlet Filter, before passing through the implementation of the HandlerMethodArgumentResolver that parses the query string and applies the operators, but it's static too, so its logic is shared around, concurrency won't be nice this way.

Maybe another token in the grammar could be useful...I don't know, something like name ~~ 'Mario', where the ~~ stands for the strict sensitive match?

Could someone post a usage with MongoDB? I've looking everywhere but, not able to find any example.
I saw the example with MongoDB in readme, but, wasn't sure what to do with the following part.

Bson bson = BsonGenerator.run(Filter.from(query), Entity.class);
Document doc = BsonUtils.getDocumentFromBson(bson);
Query query = BsonUtils.getQueryFromDocument(doc);
// ...

Hi there,

when i was testing you library, nice work btw, I was trying to filter entity ID by like '~' but wasn't working.

/filter?filter=id : '612f8b56800fb96a21e7dc23' // this work (filtering with ':')
/filter?filter=id ~ '612f8b56800fb96a21e7dc23' // this doesn't work (return empty list)
/filter?filter=id ~ '612f8b' // this doesn't work (return empty list)

logging the document value, this is how it's written (when using like '~' ):

doc: Document{{id=BsonRegularExpression{pattern='612f8b56800fb96a21e7dc23', options=''}}}

btw, here is my entity:

@Document(collection = "person")
@Data
@NoArgsConstructor
@AllArgsConstructor
@Builder(toBuilder = true)
public class Person{

    @Id
    private String id ;
    
    ........
}

I have made some search, and i find out that i can't search by id portion, since it's saved as ObjectId, and not as string.
link: https://stackoverflow.com/a/32777342/4771750

... but do you know a way to do it?

Hello, long time no see 👯!
So, I've just upgraded the library to the latest version, but it seems there's a regression due to this commit: 5117fec#diff-e5984ad41ce0a8624b70f0a28079f35ab37392bfe1fa7b986d813728826eb714

The configuration has changed from a plain Java Configuration to a Spring-like using beans injected in the context here:

@Autowired(required = false)
SpecificationFilterArgumentResolver specificationFilterArgumentResolver;

@Autowired(required = false)
BsonFilterArgumentResolver bsonFilterArgumentResolver;

@Override
public void addArgumentResolvers(List<HandlerMethodArgumentResolver> argumentResolvers) {

    if (FilterUtils.isSpecificationDependencyPresent()) {
      argumentResolvers.add(specificationFilterArgumentResolver);
    }

    if (FilterUtils.isSpringDataMongoDbDependencyPresent()) {
      argumentResolvers.add(bsonFilterArgumentResolver);
    }
  }

the issue is that both class properties beans are not yet injected in the context when calling the method addArgumentResolvers(...),
therefore producing an Unchecked Exception (NullPointerException) when performing calls from the Spring DispatcherServlet, during method arguments resolvers used internally by Spring.

Both Beans are declared as @Component, but without any explicit @ComponentScan declaration in your library, Spring won't create Bean Definitions and these two beans won't be present inside the ApplicationContext.
I tried adding it in one of my Configuration classes, like this: @ComponentScan("com.turkraft.springfilter") and doing so, the component scannig is triggered and another issue is raised:

Caused by: java.lang.IllegalStateException: Failed to introspect Class [com.turkraft.springfilter.boot.BsonFilterArgumentResolver] from ClassLoader [sun.misc.Launcher$AppClassLoader@18b4aac2]
	at org.springframework.util.ReflectionUtils.getDeclaredFields(ReflectionUtils.java:739)
	at org.springframework.util.ReflectionUtils.doWithFields(ReflectionUtils.java:704)
	at org.springframework.util.ReflectionUtils.doWithFields(ReflectionUtils.java:689)
	at org.springframework.boot.test.mock.mockito.DefinitionsParser.parse(DefinitionsParser.java:65)
	at org.springframework.boot.test.mock.mockito.MockitoPostProcessor.postProcessBeanFactory(MockitoPostProcessor.java:135)
	at org.springframework.boot.test.mock.mockito.MockitoPostProcessor.postProcessBeanFactory(MockitoPostProcessor.java:128)
	at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:325)
	at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:191)
	at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:746)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:564)
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:765)
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:445)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:338)
	at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:120)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124)
	... 62 more
Caused by: java.lang.NoClassDefFoundError: Lorg/springframework/data/mongodb/CodecRegistryProvider;
	at java.lang.Class.getDeclaredFields0(Native Method)
	at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
	at java.lang.Class.getDeclaredFields(Class.java:1916)
	at org.springframework.util.ReflectionUtils.getDeclaredFields(ReflectionUtils.java:734)
	... 77 more
Caused by: java.lang.ClassNotFoundException: org.springframework.data.mongodb.CodecRegistryProvider
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 81 more

That's a side effect due to the fact that now the configuration is not programmatic anymore. The BsonFilterArgumentResolver is another declared component, and during component scanning, it declares a bean that is not required (and that's fine), but the problem is that without mongo dependency in the classpath, even the bean definition fails at PostProcessorRegistrationDelegate class level, because the class itself might not be in the java classpath.

I'm happy to help if you need additional support/infos.

Hello, I've tried to add the library to my project with version 1.0.5 as is in the readme but maven couldn't find it so I checked mvnrepository here https://mvnrepository.com/artifact/com.turkraft/spring-filter and the last version is 1.0.4. Is this a known issue?

Hi, I would like to query date fields on the mongo db.
Does mongo support the date format? I'm trying to enrich the library.

I'm opening this issue to discuss about a like operator supporting case sensitive function.
Sometimes it could be useful to provide a like query which matches exactly with the case provided.
Tell me what do you think about it!

Is there a way to map properties?

I use DTOs, and on the frontend for example I have:

interface ThisDto {
  id: number,
  otherId: number,
  ....
}

and on the backend I have:

ThisEntity {
  Long id;
  OtherEntity otherEntity;
}

OtherEntity {
  Long id;
  ...
}

If I want to use a filter on the otherId, I have to extend my dto with this on the frontend:

interface ThisDto {
  id: number,
  otherId: number,
  OtherEntity otherEntity
}

interface OtherEntity {
  id: number
}

But it would be much simpler and cleaner if I could configure it, like in the modelMapper:

TypeMap<ThisDto, OtherEntity> typeMap = modelMapper.createTypeMap(ThisDto.class, OtherEntity.class);
typeMap.addMappings(mapper -> {
    mapper.map(ThisDto::getOtherId, OtherEntity::setId);
});

In some cases, I sometimes need to choose the type of join, but I can't do it. I would like to see something similar to this.

package ru.rshb.intech.mortgage.engine.customFilter;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
public @interface JoinType {

	javax.persistence.criteria.JoinType type() default javax.persistence.criteria.JoinType.INNER;

}

package com.turkraft.springfilter.parser.generator.expression;

public static Path<?> getDatabasePath(
			Root<?> root,
			Map<String, Join<?, ?>> joins,
			Object payload,
			String fieldPath,
			BiFunction<Path<?>, Object, Boolean> authorizer) {

		if (!fieldPath.contains(".")) {
			return authorize(authorizer, root.get(fieldPath), payload, fieldPath);
		}

		if (!FilterUtils.isHibernateCoreDependencyPresent()) {
			throw new UnsupportedOperationException(
					"The Hibernate Core dependency should be added in order to filter nested fields");
			// TODO: instead of throwing an exception, try to join with JPA only
		}

		Path<?> path = root;
		From<?, ?> from = root;

		String[] fields = fieldPath.split("\\.");

		String chain = null;

		for (int i = 0; i < fields.length; i++) {

			String field = fields[i];

			path = from.get(field);

			if (chain == null) {
				chain = field;
			} else {
				chain += "." + field;
			}

			authorize(authorizer, path, payload, chain);

			JoinType join = Arrays.stream(((RootImpl<?>) root).getEntityType().getJavaType().getDeclaredFields())
					.filter(classField -> classField.getName().equals(field))
					.findFirst()
					.flatMap(classField -> Arrays.stream(classField.getAnnotations())
							.filter(annotation -> ru.rshb.intech.mortgage.engine.customFilter.JoinType.class.getName()
									.equals(annotation.annotationType().getName()))
							.findFirst()
							.map(annotation -> ((ru.rshb.intech.mortgage.engine.customFilter.JoinType) annotation).type()))
					.orElse(path instanceof PluralAttributePath ? JoinType.INNER
							: (path instanceof SingularAttributePath
									&& ((SingularAttributePath<?>) path).getAttribute().getPersistentAttributeType()
									!= PersistentAttributeType.BASIC
									? JoinType.LEFT
									: null));

			if (join != null && i < fields.length - 1) {
				if (!joins.containsKey(chain)) {
					joins.put(chain, from.join(field, join));
				}
				from = joins.get(chain);
			}

		}

		return path;

	}

Hi is it possible to enable @filter on POST endpoints?

Reason im asking is because for GET we have care about url encoding, which can make the front-end clients quite verbose for example

GET http://test.com?page=0&sort=id&filter=ssn:'11111'

Instead we could have

POST

{
   "query": "ssn: '111' and name: '123'"
}

the LIKE filter isn't working with ignore case as wanted.
Version: 1.1.0

let's say I have FALL value in document link property. (my database is mongo) rest apis, ignore case isn't working.

// document in database
{link: 'FALL'}

//rest apis
URL?&filter=link~'FALL*' //working
URL?&filter=link~'Fall*' // NOT working
URL?&filter=link~'fall*' // NOT working

quoting from your readme docs file:

By default, this comparator is case insensitive, the behavior can be changed with ExpressionGeneratorParameters.CASE_SENSITIVE_LIKE_OPERATOR.

I have tried changing that value but still didn't work. here is my code:

@Configuration
public class SpringFilterConfiguration {
    SpringFilterConfiguration() {
        ExpressionGeneratorParameters.CASE_SENSITIVE_LIKE_OPERATOR = true;
    }
}

Debugging my spring boot app, all the above API calls result building the same Document:
(with and without CASE_SENSITIVE_LIKE_OPERATOR configuration)

URL?&filter=link~'FALL*' ===>  Document{{link=BsonRegularExpression{pattern='FAll*', options=''}}}
URL?&filter=link~'Fall*' ===>  Document{{link=BsonRegularExpression{pattern='Fall*', options=''}}}
URL?&filter=link~'fall*' ===>  Document{{link=BsonRegularExpression{pattern='fall*', options=''}}}

How we could add i in $regex options?

testing my logic code directly in @Query,

{ link: {$regex:/FALL/, $options: 'i'}
{ link: {$regex:/Fall/, $options: 'i'}
{ link: {$regex:/fall/, $options: 'i'}

adding i in regex options solve it and make them all return the specified document: {link: 'FALL'}.

We could solve the problem by building this regex Document:

Document{{link=BsonRegularExpression{pattern='fall*', options='i'}}} (with i in $regex options)

could you please check it out?

Hi,
I'm trying to integrate your library into one of my projects.
Since my base JPA entity has a LocalTime, I want to apply filter on this field but this type is not supported.
Could you please help me?
Best Regards.

Hi everyone 👋

I am planning to create a JavaScript filter builder which I think will be helpful when calling APIs with frontend frameworks such as Vue or Angular. It should be similar to the FilterBuilder class.

Example:

// lastName : 'Doe' and birthDate > '1-01-2000'
const filter =
    and(
        equal('lastName', 'Doe'),
        greaterThan('birthDate', '1-01-2000')
    ).generate();

const req = await fetch('http://api/person?filter=' + filter);

Let me know if you're interested, if you have suggestions or things you would like to see. :)

Edit: we now have our builder https://github.com/sisimomo/Spring-Filter-Query-Builder!

Please how to do a 'not like' filter?
I wanted to do 'not contains' filter (in SQl it's NOT LIKE '%value%') but I failed, I tested many formulas :

• !'%value%'
• !~'%value%'
• not(~'%value%')

But it didn't work.
Thank you.

Why no sorting in the library?
I see in history that it was deleted.

Hi turkraft,

In some cases, I want to prohibit users from using advanced filtering features.
Is there any way to do it?
Is there any way to disable certain operators or set the available operators explicitly for the entities?

Hi turkraft,
I have problems with the ~ operator. In particular, in the README it is written that "the * character can also be used instead of % when using the ~ comparator" but it functions only with *.
Thank you.

Hello,

I happily found your spring-filter lib, and tried to use it for my project.
Unfortunately, I came accross a problem when filtering on some related objects.

I use spring-boot 2.5 with spring-data-jpa
I have an entity like

@Entity
UserEntity {
	private String username;
	
	@OneToOne(fetch = FetchType.EAGER)
	private RoleEntity role;
	[...]
}

and use your spec builder in my service with something like
Page<UserEntity> entities = userRepo.findAll(new FilterSpecification<UserEntity>(search), pageable);

And I have some cases that always fails.

search=role.name:'ADMIN'
-> always works
search=role.name:'EDITOR'
-> always fails with the error below, that happens when spring data builds its count query
search=role.name:'EDITOR' and username~'something%'
-> always works when adding another criteria that filters something and reduce the results count
search=role.name:'EDITOR' and username~'noimpact%'
-> always fails with same error when adding another criteria that filters nothing and does not change the results count

From the last 2 cases, I believe the problem comes only when spring chooses to build a new count query.
But when I look at the first 2, I really do not see why one fails and not the other...

Anyway, after debugging failing cases, I found that the RootImpl used in the count query had no Joins, and so that is why RoleEntity is missing in the "from" part of the count query.
So looking at your spec builder, I found that sometimes joins building is bypassed when a join of the same key exists.
Bypassing the check to rebuild joins every time solves the problem, as shown below.

But being rather new on this spring-data-jpa API, I cannot tell if it is a good solution...
So could you have a look please and see if there is something better to do ?
And of course make a fix available if this is a real bug.
Let me know.
Thanks

--- the error I got with the search filters
org.hibernate.hql.internal.ast.QuerySyntaxException: Invalid path: 'generatedAlias1.name' [select count(generatedAlias0) from fr.edf.ssie.api.entity.UserEntity as generatedAlias0 where ( generatedAlias1.name=:param0 ) or ( generatedAlias1.name=:param1 )]; nested exception is java.lang.IllegalArgumentException: org.hibernate.hql.internal.ast.QuerySyntaxException: Invalid path: 'generatedAlias1.name' [select count(generatedAlias0) from fr.edf.ssie.api.entity.UserEntity as generatedAlias0 where ( generatedAlias1.name=:param0 ) or ( generatedAlias1.name=:param1 )]]

--- the code I changed to solve my problem

ExpressionGeneratorUtils.getDatabasePath(...)
[...]
	if (join != null && i < fields.length - 1) {
		// FIXME even if the joins map contains the field, call from.join() method to add a join in root.joins, else join will be missing when building query
		// if (!joins.containsKey(chain)) {
		joins.put(chain, from.join(field, join));
		//}
		from = joins.get(chain);
	}
[...]

ENHANCEMENT:

Spring boot mongo repositories aggregation offer means to interact with the aggregation framework via annotated repository query methods. Similar to the JSON based queries, you can define a pipeline using the org.springframework.data.mongodb.repository.Aggregation annotation.

I'm suggesting that you should enhance your library to support aggregation pipeline:

now your library could be used like this: (working example)

// controller
@GetMapping("")
public Page<Entity> filter(@Filter(entityClass = Entity.class) Document document, Pageable pageable) {
    return repository.filter(document, pageable);
}
// repository
@Query("?0")
Page<Entity> filter(Document document, Pageable pageable);

mongo repositories aggregation annotation @Aggregation accepts 2 types of arguments:

• String: simple aggregation query. (manu examples are available in official doc, the link above).
• String[]: aggregation queries pipeline.

example format of aggregation queries pipeline

// it returns list, can't return page. limitation by official library
@Aggregation({
  "{ $match : {.......},",
  "{$addFields:{.......}",
  "{'$unwind':'$field'}",
  "{$group:{.......}",
  "{'$sort':{'_id':-1}}",
  "{$project:{......}",
})
List<dataDTO> exampleOfAggregationPipeline(ParamTypeObject params... , Pageable pageable);

current usage of your library with Aggregation:

your library could be used currently with aggregation to the $match only.

@Aggregation({
  "{ $match : ?0 ,",
  "{$addFields:{.......}",
  "{'$unwind':'$field'}",
  "{$group:{.......}",
  "{'$sort':{'_id':-1}}",
  "{$project:{......}",
})
List<dataDTO> exampleOfAggregationPipeline(Document document, ParamTypeObject params... , Pageable pageable);

enhancement of your library with Aggregation: (suggestions)

IMPORTANT: be aware that @Aggregation("?0") is referring to simple aggregation query which accepts String type and not activating aggregation pipelines.

@Aggregation() or @Aggregation("[0]") or @Aggregation("#0") or @Aggregation("#0")
List<dataDTO> exampleOfAggregationPipeline(List<Document> document, ParamTypeObject params... , Pageable pageable);
// or to simplify it, params could be added from the query builder.
List<dataDTO> exampleOfAggregationPipeline(List<Document> document , Pageable pageable);

This should be outstanding, and more flexible solution.

Hope you like it, and work on it :)

is there a way to provide a function whose result is the computation of an expression?
eg:
/car?filter=size(wheels.diameter>:10)>3
which means "find every car with at least 4 wheels with a diameter >=10".

@Configuration
public class FilterWebConfig implements WebMvcConfigurer {

  @Override
  public void addArgumentResolvers(List<HandlerMethodArgumentResolver> argumentResolvers) {
    argumentResolvers.add(new SpecificationFilterArgumentResolver());
    argumentResolvers.add(new DocumentFilterArgumentResolver());
  }

}

While I was integrating the library in a mongo only project, i noticed that at startup both resolvers are automatically added to the runtime environment, causing a ClassNotFoundException, if either JPA or Mongo dependency is not in the classpath.
The pom.xml states those dependecy with <optional>true</optional> so i guess these resolvers must be registered dinamically according to this.
This is not a bug...it's a feature👯‍♂️ , since now the library both supports spring-data-mongodb and spring-data-jpa.

It would be great if you could implement filtering of PostgreSQL hstore.
For an example EXISTS(some_hstore.key) or some_hstore.key:value or some_hstore.key IN (value1, value2)

Can you please add functionality(annotation on field) for disabled or permission to search by one field or more fields

In the company API, is it possible to search among the employee names?

The below syntax does not work.

employees.firstName : "Johana"

It would be nice have filter functionality on filelds which uses attributeconvertor.

I have a convertor class

public class MasterAttributeConvertor implements AttributeConverter<TestEntity, Integer> {

private final RedisAdaptor<Integer, TestCacheEntity> masterModelCacheAdaptor;

@Override public Integer convertToDatabaseColumn(){}

public MasterCacheEntity convertToEntityAttribute(Integer testId) (){}

using redis, this converts

id to TestEntiy when giving as response.
converts TestEntity to id when saving in db.

Right now if I pass testId to filter the convertor field. it would throw "cannot cast integer to testCacheEntity"

It would be a nice feature or I doest find it yet to filter nested collection.

Like in the Live Example to filter Company with

(id < 10 and employees.maritalStatus: "unknown")
curl -X 'GET'
'https://spring-filter.herokuapp.com/company?filter=id%20%3C%2010%20and%20employees.maritalStatus%3A%22unknown%22'
-H 'accept: /'

Expected Output

[
  {
    "id": 4,
    "name": "Huel Inc",
    "industry": {
      "id": 3,
      "name": "Capital Markets"
    },
    "employees": [
      {
        "id": 11,
        "firstName": "Johana",
        "lastName": "Crona",
        "birthDate": "13-09-1980",
        "maritalStatus": "unknown",
        "salary": 2301
      },
      {
        "id": 16,
        "firstName": "Cora",
        "lastName": "Lebsack",
        "birthDate": "18-10-1961",
        "maritalStatus": "unknown",
        "salary": 5783
      }
    ]
  }
]

In my own example with one Entity and 3 Elements in nested collection i got 3 times the same response

First of all, I want to thank you for such a great tool! It is not clear to me why in such a developed Spring ecosystem there is no tool that provides REST query lanquage out-of-the-box. I have tried RSQL with extentions, QueryDSL with value oparators and now see that this tool has great potential to be the first choice.

Secondly, I want to ask you to add more examples of the use-cases various elements and their interaction to the swagger

I can't use spring-filter now because it lacks a critical functionality for filtering by nested collections. I try to explain ruge by examples, from more critical to less (by swagger):

1. /employee?filter=stuff.firstName:Maurine - return that employees who get in stuff person with firstName "Maurine"
   1.1) /employee?filter=stuff.salary > 10000 - return that employees who got any stuff with salary more then 10000
   1.2) /employee?filter=all(stuff.salary) > 10000 - return that employees who got all stuff with salary more then 10000

2. /employee?filter=sum(salary) < 20000 - return employees globally limited by sum of their salary by 20000

like in postgres:

SELECT *
FROM
(SELECT *, sum(salary) OVER (ORDER BY salary) AS sumSalary FROM employee) AS emp
WHERE sumSalary < 20000 ;

2.1) /employee?filter=staff.sum(salary) < 15000 - return that employees whos all staff sum sallary less 15000
2.2) /employee?filter=sum(staff.salary) < 15000 - return employees globally limited by sum of their staff salary by 20000

Hi,
I'm trying to integrate your library into one of my projects.
Since my base JPA entity has a PK UUID, and @UpdateTimestamp @CreationTimestamp declared as OffsetDateTime
I could find a way to search for them.
Could you please give me some hints.
Best Regards.

Will it also work in sWAGGER?
Can user test out in swagger?

Can this library be adapted for use with mongo repository?

Hi,

I'm just trying to filter entities on a BigDecimal field but I get this error:

com.turkraft.springfilter.exception.InternalFilterException: The input '5' could not be converted to class java.math.BigDecimal at com.turkraft.springfilter.parser.generator.expression.ExpressionGenerator.visitInput(ExpressionGenerator.java:471) ~[spring-filter-2.0.2.jar:na]

Could you please add support for BigDecimal.

When parallel requests are sent via filter, the sharing NumberFormat instance parse randomly different numbers. Sometimes it fails , sometime its correct and sometimes it parses a double out.

The issue is caused by your StringConverter.class

An simple example that causes the issue:

public class Scratch extends Thread {

    public static final NumberFormat NUMBER_FORMAT ;

    public static void main(String args[]) {
        for (int i = 0; i < 3; i++) {
            Scratch thread = new Scratch();
            thread.start();
        }
    }

    public void run() {
        ParsePosition position = new ParsePosition(0);
        Number number = NUMBER_FORMAT.parse("1011", position);
        System.out.println(number);
    }

    static {
        NUMBER_FORMAT = NumberFormat.getInstance(Locale.US);
        NUMBER_FORMAT.setGroupingUsed(false);
    }
}

The issue disapers, if we use a new instance of Numberformat in every Thread:

public class Scratch extends Thread {


    public static void main(String args[]) {
        for (int i = 0; i < 5; i++) {
            Scratch thread = new Scratch();
            thread.start();
        }
    }

    public void run() {
        ParsePosition position = new ParsePosition(0);
        NumberFormat format = NumberFormat.getInstance(Locale.US);
        format.setGroupingUsed(false);
        Number number = format.parse("1011", position);
        System.out.println(number);
    }
  
}

#75

concat(firstName, concat(' ', lastName)) will then be simplified to concat(firstName, ' ', lastName).

With version 1.1.0, a bad input like
email~manu.%
used to throw a com.turkraft.springfilter.exception.InvalidInputException

Now with 2.0.0, it passes silently, util the spring DAO throws its org.springframework.dao.InvalidDataAccessApiUsageException
Unable to locate Attribute with the the given name [manu] on this ManagedType [...]

I think the previous behavior was more explicit and debug friendly, but maybe the change was needed in your new implementation.
Let me know what you think.