-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ToQueryString method to RedisCollection, RedisAggregationSet #487
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @tgmoore, first thanks for opening this! I think we just need to wrap all the arguments and the command names in double quotes - and this should be pretty much good to go. To your questions:
ToCommandString
is probably the more correct name insofar as defining what it does, however given that EF Core usesToQueryString
, I think it's very probably better to leave it asToQueryString
- This is a perfectly reasonable level of duplication
- You probably don't need to do all the myriad set of different things, but it's not wrong to do it.
serializedArgs.Add(_chunkSize.ToString()); | ||
} | ||
|
||
return $"FT.AGGREGATE {string.Join(" ", serializedArgs)}"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we also need to do a select and wrap all the arguments in double quotes "
(as well as wrapping the FT.AGGREGATE
in quotes)
There's two reasons for this:
-
Syntax / parameterization: the way that Redis protocol works, it requires it's arguments to be parameterized, so all of the serialized arguments above are actually parameterized when they are sent over the wire to Redis (e.g. the protocol tells Redis how many arguments to expect, and then a precise length for each so the query
FT.SEARCH people "@name:{Steve} @city:Satellite" LIMIT 0 100
is actually 6 parameters over the wire:FT.SEARCH
,people
,@name:{Steve} @city:Satellite
,LIMIT
,0
,100
. Particularly in the case of the query string@name:{Steve} @city:Satellite
, it needs to be made very clear to the RESP serializer (e.g. the redis CLI) that that is the entire parameter, which leads to the ergonomics of it. -
Ergonomics: I think what's likely to be the most common usage of this feature is to dump a string into a log / console for a user to just copy/paste into the redis-cli / Redis insights to see what it pulls back. Surrounding everything with quotes is how the MONITOR works, and this is quite deliberate as it makes it trivial to copy/paste whatever the command was.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do. Thank you for your clear answers to my questions and additional comprehensive feedback! I really appreciate your time and attention.
public void TestToQueryString() | ||
{ | ||
var collection = new RedisAggregationSet<Person>(_substitute, true, chunkSize: 10000); | ||
var command = "FT.AGGREGATE person-idx @Salary:[(30.55 inf] LOAD * APPLY @Address_HouseNumber + 4 AS house_num_modified SORTBY 2 @Age DESC LIMIT 0 10 WITHCURSOR COUNT 10000"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See this is what I was worried about above, this command needs quotes to be properly parameterized (from the CLI's perspective)
127.0.0.1:6379> FT.AGGREGATE person-idx @Salary:[(30.55 inf] LOAD * APPLY @Address_HouseNumber + 4 AS house_num_modified SORTBY 2 @Age DESC LIMIT 0 10 WITHCURSOR COUNT 10000
(error) Unknown argument `inf]` at position 1 for <main>
vs (mind you I've not added any data):
127.0.0.1:6379> FT.AGGREGATE person-idx "@Salary:[(30.55 inf]" LOAD * APPLY "@Address_HouseNumber + 4" AS house_num_modified SORTBY 2 @Age DESC LIMIT 0 10 WITHCURSOR COUNT 10000
1) 1) (integer) 0
2) (integer) 0
IMO the expected output should be:
"FT.AGGREGATE" "person-idx" "@Salary:[(30.55 inf]" "LOAD" "*" "APPLY" "@Address_HouseNumber + 4" "AS" "house_num_modified" "SORTBY" "2" "@Age" "DESC" "LIMIT" "0" "10" "WITHCURSOR" "COUNT" "10000"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense. Thanks for the link to MONITOR. This feedback helped me to understand some of the conversations I've seen on the Discord and in other issues about serialization and escaping special characters.
{ | ||
var query = ExpressionTranslator.BuildQueryFromExpression(Expression, typeof(T), BooleanExpression, RootType); | ||
query.Limit ??= new SearchLimit { Number = ChunkSize, Offset = 0 }; | ||
return $"FT.SEARCH {string.Join(" ", query.SerializeQuery())}"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same comment regarding quotations surrounding the arguments here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've made this change as well. Thank you!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tgmoore LGTM 👍 - thank you for submitting this!
Exposes ToQueryString from RedisCollection and RedisAggregationSet.
Resolves #486
The following questions came up while putting this together:
ToQueryString
a reasonable name? IsToCommandString
more accurate for Redisearch?AggregationEnumerator.SerializedArgs
andRedisCollectionEnumerator.RedisCollectionEnumerator
. Is that reasonable, or should there be a common source for that behavior?