Params to store values local

[liukonen]

I wrote a Docker file to try to use Garnet as a replacement Redis server. As of late, it's meeting all of my goals, however, I am running into some difficulties in storing values on disk

I created a volume mount point and am using the following entry point, which I know is probably overkill

ENTRYPOINT ["/app/GarnetServer", "-i", "128m", "-c", "/data/checkpoint", "-r", "true", "-l", "/data/logs", "--storage-tier", "true", "--on-demand-checkpoint", "true", "--incsnap", "true", "--main-memory-replication", "true", "--checkpoint-throttle-delay", "1800000", "--aof", "true", "--aof-commit-freq", "300000", "--aof-size-limit", "100MB"]

The only things I see on disk are 2 subfolders in /data/logs, a Store and Objectstore folder, which are both empty
I'd like a periodic record stored so that if I relaunch or update my container, it can pull from the stored values. Any help would be appreciated

[kingcomxu]

my try:
var dir = @"z:\garnet_files\"; GarnetServerOptions options = new GarnetServerOptions(); options.Address = "0.0.0.0"; options.CheckpointDir = dir; /c options.EnableAOF = true; /aof options.Recover = true; /r using var server = new GarnetServer(options);

can reconver data on relaunch
But I still do not understand the persistence mechanism of garnet, I hope the official document can be issued
For example, What does the redis rdb file correspond to garnet

[liukonen]

will give this a try... thanks 👍

[liukonen]

This keeps persistence. Would still like some flavor of rdb, which is a direct instance of the db on disk if possible, but that might be a bit outside the scope of the project at the time :)

[liukonen]

As a side, wouldn't need to be a direct 1/1, just a way to store the data in db for persistence that auto cleans up as keys expire

[badrishc]

If you call SAVE or BGSAVE then data will be checkpointed.
For data between checkpoints, you can enable AOF.
You can configure Garnet to take periodic checkpoints.
When spilling to disk, you can define disk space and compaction strategy.

All this is exposed as options when you start the server.

[ceisele-r]

@badrishc I tried to find the configuration option to take periodic checkpoints but I couldn't find it in the docs.
Could you please give me a hint how I can configure periodic checkpoints?

[badrishc]

When you enable AOF, you can specify AOF size limit (--aof-size-limit). When the AOF size limit is reached, Garnet will take an automatic checkpoint in order to truncate the AOF.

[nightroman]

@badrishc Nice tip, thank you, the "automatic checkpoint" part works. But I am not sure that "truncate the AOF" works (as I expect at least). The file keeps growing. Here is what I see in logs:

04::04::55 info: StoreWrapper[0] Enforcing AOF size limit currentAofSize: 1369804 >  AofSizeLimit: 1048576
04::04::55 info: StoreWrapper[0] Initiating checkpoint; full = False, type = Snapshot, tryIncremental = False, storeType = All
04::04::55 info: StoreWrapper[0] Will truncate AOF to 84193748 after checkpoint (files deleted after next commit)
04::04::55 info: StoreWrapper[0] Completed checkpoint

I set --aof-size-limit 1m. Garnet correctly sees this

04::04::55 info: StoreWrapper[0] Enforcing AOF size limit currentAofSize: 1369804 >  AofSizeLimit: 1048576

But then what does this mean? "truncate AOF to 84193748" is not what I expect setting the limit to 1048576

04::04::55 info: StoreWrapper[0] Will truncate AOF to 84193748 after checkpoint (files deleted after next commit)

[badrishc]

File deletion/truncation happens at the granularity of segment size, which is by default 1GB. Above shows you have only truncated until 84MB. For these smaller scales, you could use a smaller segment size specified via SegmentSize argument in options.

Segment size identifies the size of each file created as part of the AOF. We add and remove at this file granularity.

[nightroman]

@badrishc Thank you for the useful bits of information, difficult to guess without the detailed manuals.
AOF still keeps growing... I use these options:

--port 3333 --aof --aof-size-limit 1m --page 1m --segment 1m --compaction-max-segments 1 --checkpointdir C:\Users\...\checkpointdir --index 64m --obj-index 8m -q --recover --logger-level Information

(I am just testing waters, small numbers are for testing, not practice)