Skip to content

Stream the key/value-pairs of a judy array into an arbitrary data structure

License

Notifications You must be signed in to change notification settings

tkvogt/streamly-judy

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

streamly-judy

Stream the key/value-pairs of a judy array, dumping it to a file.

In the judy library the functions

keys :: JudyImmutable a -> IO [Key]
elems :: JE a => JudyImmutable a -> IO [a]
toList :: JE a => JudyImmutable a -> IO [(Key, a)] 

return lists, which can use quite a lot of memory. This is problematic when the array is big. It makes much more sense to stream a judy array into an arbitrary data structure. This way much bigger judy arrays can be loaded from/stored into a file, which is also in my use case faster than the construction of the judy array and reduces startup time.

Instead of adding these functions to judy or streamly, we make a new library to not force new dependencies on either of the two.

About

Stream the key/value-pairs of a judy array into an arbitrary data structure

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published