Subkey expiration

[optiman]

KeyDB has this interesting feature called subkey expiration for SET and HASH data types.
Is it:

• possible/desirable in Dragonfly (maybe after Redis API is completely implemented)
• out of the scope of this project
• unrealistic due to current architectural decisions / excessive memory usage

The EXPIREMEMBER command

KeyDB implements subkey expires with a new command, EXPIREMEMBER. This command takes in a key along with a subkey and an expire time, and removes just the subkey from the data structure mapped to the key at the specifiied time leaving the rest of the value intact.

Keydb-cli> HSET myhash f1 v1 f2 v2 f3 v3
(integer) 3
Keydb-cli> EXPIREMEMBER myhash f3 10
(integer) 1
Keydb-cli> TTL myhash f3
(integer) 10

10 seconds later

Keydb-cli> HGET myhash f1
"v1"
Keydb-cli> HGET myhash f2
"v2"
Keydb-cli> HGET myhash f3
(nil)

Blog post with more details:
https://docs.keydb.dev/blog/2021/06/08/blog-post/

[romange]

So, first of all, if someone from Ukraine requests a subkey expiration - we will deliver the subkey expiration!

Seriously, it is on our roadmap. Roadblocks to this feature are not technical. We need design partners for this feature to make a high quality work.

1. We need to learn about use-cases. How people are planning to use it.
2. One of the questions - is how to implement SCARD (if we talk about SET type). Do you need exact SCARD semantics when you use expirations? Can you live with the upper bound? If you need exact semantics, it may take O(N) worst complexity.

[optiman]

That's great! Indeed, it is better to plan everything thoroughly, so no hurry.
If you ask me, I'm for exact SCARD answer even if it takes more time, but it depends on the implementation - how much more time, maybe it will be just impractical with very large sets :)
And thanks for the quick reply!

[romange]

Sure. Do you use very large sets in your app? what's the maximal cardinality?

[optiman]

No, currently not, most of the time they are tiny.
It depends on the workload. I assume a couple tens of thousands elements max, but that's not really that large either.