Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The PR introduce a new proc macro for command registrations. #326

Merged
merged 13 commits into from
May 15, 2023

Conversation

MeirShpilraien
Copy link
Collaborator

@MeirShpilraien MeirShpilraien commented May 7, 2023

The PR introduce a new proc macro for command registrations with the new command registration API that was introduced on Redis 7.

The new proc macro are called command, The following command will register the test_command function as a Redis command called foo:

#[command(
    {
        name: "test",
        flags: [ReadOnly],
        arity: -2,
        key_spec: [
            {
                notes: "test command that define all the arguments at even possition as keys",
                flags: [ReadOnly, Access],
                begin_search: Keyword({ keyword : "foo", startfrom : 1 }),
                find_keys: Range({ last_key :- 1, steps : 2, limit : 0 }),
            }
        ]
    }
)]
fn test_command(_ctx: &Context, _args: Vec<RedisString>) -> RedisResult {
    Ok(RedisValue::SimpleStringStatic("OK"))
}

The supported properties are:

  • name - The command name,
  • flags - An array of RedisCommandFlags.
  • summary (optional) - Command summary
  • complexity (optional) - Command compexity
  • since (optional) - At which module version the command was first introduce
  • tips (optional) - Command tips for proxy, for more information please refer to https://redis.io/topics/command-tips
  • arity - Number of arguments, including the command name itself. A positive number specifies an exact number of arguments and a negative number specifies a minimum number of arguments.
  • key_spec - A list of specs representing how to find the keys that the command might touch. the following options are available:
    • notes (optional) - Some note about the key spec.
    • flags - List of flags reprenting how the keys are accessed, the following options are available:
      • Readonly - Read-Only. Reads the value of the key, but doesn't necessarily return it.
      • ReadWrite - Read-Write. Modifies the data stored in the value of the key or its metadata.
      • Overwrite - Overwrite. Overwrites the data stored in the value of the key.
      • Remove - Deletes the key.
      • Access - Returns, copies or uses the user data from the value of the key.
      • Update - Updates data to the value, new value may depend on the old value.
      • Insert - Adds data to the value with no chance of modification or deletion of existing data.
      • Delete - Explicitly deletes some content from the value of the key.
      • NotKey - The key is not actually a key, but should be routed in cluster mode as if it was a key.
      • Incomplete - The keyspec might not point out all the keys it should cover.
      • VariableFlags - Some keys might have different flags depending on arguments.
    • begin_search - Represents how Redis should start looking for keys.There are 2 possible options:
      • Index - start looking for keys from a given position.
      • Keyword - Search for a specific keyward and start looking for keys from this keyword
    • FindKeys - After Redis finds the location from where it needs to start looking for keys, Redis will start finding keys base on the information in this struct. There are 2 possible options:
      • Range - An object of three elements last_key, steps, limit.
        • last_key - Index of the last key relative to the result of the begin search step. Can be negative, in which case it's not relative. -1 indicates the last argument, -2 one before the last and so on.
        • steps - How many arguments should we skip after finding a key, in order to find the next one.
        • limit - If lastkey is -1, we use limit to stop the search by a factor. 0 and 1 mean no limit. 2 means 1/2 of the remaining args, 3 means 1/3, and so on.
      • Keynum - An object of 3 elements keynumidx, firstkey, keystep.
        • keynumidx - Index of the argument containing the number of keys to come, relative to the result of the begin search step.
        • firstkey - Index of the fist key relative to the result of thebegin search step. (Usually it's just after keynumidx, inwhich case it should be set to keynumidx + 1.) * keystep - How many arguments should we skip after finding a key, in order to find the next one?

Notice, by default Redis does not validate the command spec. User should validate the command keys on the module command code. The command spec is used for validation on cluster so Redis can raise a cross slot error when needed.

Ideas for future extensions:

  • The proc macro can analyze the function inputs and generate a code for parsing the command arguments so that the command function will not have to deal with it.

The PR introduce a new proc macro for command registrations with the new command registration API that was introduced on Redis 7.

The new proc macro are called `redis_command`, The following command will register the `test_command` function as a Redis command called `foo`:

```rust
#[redis_command(
{
   name: "foo",
   arity: 3,
   key_spec: [
       {
           notes: "some notes",
           flags: ["RW", "ACCESS"],
           begin_search: Keyword(("foo", 1)),
           find_keys: Range((1, 2, 3)),
       }
   ]
}
)]
fn test_command(_ctx: &Context, _args: Vec<RedisString>) -> RedisResult {
    Ok(RedisValue::SimpleStringStatic("OK"))
}
```

The supported properties are:

* name - The command name,
* flags (optional) - Command flags such as `readonly`, for the full list please refer to https://redis.io/docs/reference/modules/modules-api-ref/#redismodule_createcommand
* summary (optional) - Command summary
* complexity (optional) - Command compexity
* since (optional) - At which module version the command was first introduce
* tips (optional) - Command tips for proxy, for more information please refer to https://redis.io/topics/command-tips
* arity - Number of arguments, including the command name itself. A positive number specifies an exact number of arguments and a negative number specifies a minimum number of arguments.
* key_spec - A list of specs representing how to find the keys that the command might touch. the following options are available:
   * notes (optional) - Some note about the key spec.
   * flags - List of flags reprenting how the keys are accessed, the following options are available:
      * RO - Read-Only. Reads the value of the key, but doesn't necessarily return it.
      * RW - Read-Write. Modifies the data stored in the value of the key or its metadata.
      * OW - Overwrite. Overwrites the data stored in the value of the key.
      * RM - Deletes the key.
      * ACCESS - Returns, copies or uses the user data from the value of the key.
      * UPDATE - Updates data to the value, new value may depend on the old value.
      * INSERT - Adds data to the value with no chance of modification or deletion of existing data.
      * DELETE - Explicitly deletes some content from the value of the key.
      * NOT_KEY - The key is not actually a key, but should be routed in cluster mode as if it was a key.
      * INCOMPLETE - The keyspec might not point out all the keys it should cover.
      * VARIABLE_FLAGS - Some keys might have different flags depending on arguments.
   * begin_search - Represents how Redis should start looking for keys.There are 2 possible options:
      * Index - start looking for keys from a given position.
      * Keyword - Search for a specific keyward and start looking for keys from this keyword
   * FindKeys - After Redis finds the location from where it needs to start looking for keys, Redis will start finding keys base on the information in this struct. There are 2 possible options:
      * Range - A tuple represent a range of `(last_key, steps, limit)`.
         * last_key - Index of the last key relative to the result of the begin search step. Can be negative, in which case it's not relative. -1 indicates the last argument, -2 one before the last and so on.
         * steps - How many arguments should we skip after finding a key, in order to find the next one.
         * limit - If `lastkey` is -1, we use `limit` to stop the search by a factor. 0 and 1 mean no limit. 2 means 1/2 of the remaining args, 3 means 1/3, and so on.
      * Keynum -  A tuple of 3 elements `(keynumidx, firstkey, keystep)`.
         * keynumidx - Index of the argument containing the number of keys to come, relative to the result of the begin search step.
         * firstkey - Index of the fist key relative to the result of thebegin search step. (Usually it's just after `keynumidx`, inwhich case it should be set to `keynumidx + 1`.)
         * keystep - How many arguments should we skip after finding a key, in order to find the next one?

**Notice**, by default Redis does not validate the command spec. User should validate the command keys on the module command code. The command spec is used for validation on cluster so Redis can raise a cross slot error when needed.

Ideas for future extension:

* The proc macro can analyze the function inputs and generate a code for parsing the command arguments so that the command function will not have to deal with it.
Copy link
Collaborator

@iddm iddm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work! I was doing almost the same behind the scenes, and as a user, I would personally love this approach compared to the long-long redis_macro. The more things we can split out, the better, in my opinion, just as it is done with other projects, for example, social network bots. I hope we continue adding more procedural macros in the future to make it much easier for the user to create modules, at the same time lowering the boundary for creating it by greatly simplifying the code and making it much more concise.

I suggest a few changes though, which, I think, are reasonable.

examples/proc_macro_commands.rs Show resolved Hide resolved
examples/proc_macro_commands.rs Outdated Show resolved Hide resolved
examples/proc_macro_commands.rs Show resolved Hide resolved
examples/proc_macro_commands.rs Outdated Show resolved Hide resolved
examples/proc_macro_commands.rs Outdated Show resolved Hide resolved
examples/proc_macro_commands.rs Show resolved Hide resolved
examples/proc_macro_commands.rs Outdated Show resolved Hide resolved
src/context/mod.rs Outdated Show resolved Hide resolved
/// to the number of keys that will follow. Then it will start
/// from `firstkey` and jump each `keystep` to find the keys.
pub enum FindKeys {
Range((i32, i32, i32)), // (last_key, steps, limit)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be too easy to mess up with the range values. I suggest removing the comments and having strict names for these fields instead of the unnamed ones within a tuple. As another alternative, we can provide transparent types, but this would be overcomplicating the situation, compared to just having named fields.

/// 2. Keyword - Search for a specific keyward and start looking for keys from this keyword
pub enum BeginSearch {
Index(i32),
Keyword((String, i32)), // (keyword, startfrom)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the named fields here instead of the comment, please?

src/context/commnads.rs Outdated Show resolved Hide resolved
src/context/commnads.rs Outdated Show resolved Hide resolved
src/context/commnads.rs Outdated Show resolved Hide resolved
src/context/commnads.rs Outdated Show resolved Hide resolved
src/context/commnads.rs Outdated Show resolved Hide resolved
src/context/commnads.rs Outdated Show resolved Hide resolved
@MeirShpilraien MeirShpilraien requested a review from iddm May 14, 2023 06:19
src/context/commands.rs Outdated Show resolved Hide resolved
@MeirShpilraien MeirShpilraien merged commit 0c708c5 into master May 15, 2023
@MeirShpilraien MeirShpilraien deleted the new_redis_command_proc_macro branch May 15, 2023 14:13
MeirShpilraien added a commit to RedisGears/RedisGears that referenced this pull request May 16, 2023
Added `RG.FCALLASYNC` to run asynchronous functions.

Solved: #895

The PR adds a new command `RG.FCALLASYNC`. Run asynchronous functions can only be done using this new command. Trying to run asynchronous functions using `RG.FCALL` will fail with an error.

In addition, the PR introduce a new `JS` api, `redis.register_async_function` which is the same as `redis.register_function` but also accept async function (coroutine). Trying to register async function using `redis.register_function` will fail with an error.

Require: RedisLabsModules/redismodule-rs#326
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants