You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are not consistent about the path attribute of stores right now. localstore has a root attribute that's basically a path (with type pathlib.path), remotestore has a v2-style explicit path attribute (with type str), zipstore has a path attribute but I think its semantics are not like remotestore.path, since zipstore.path points to a location outside the zip file system, etc.
I think we should normalize the store.path attribute along the following lines:
store.path is a string that names a location inside the filesystem modeled by the store class. All stores have such a path attribute.
new store class instances can be created via methods like store_instance.with_path('new_path') (completely new path) or store_instance.join_path('relative_path') (old path + relative_path), or store_instance / 'relative_path'` as an abbreviation of the last one.
The StorePath class goes away entirely, because we are giving stores their own path.
(This API is largely influenced by the yarl.URL class, which I have been using quite a bit recently)
Thoughts?
The text was updated successfully, but these errors were encountered:
after looking into this a bit more and starting a draft implementation of this idea, I noticed that the semantics of some of the store operations varies too much across the different store implementations. If we compare MemoryStore and RemoteStore:
MemoryStore has no path attribute; MemoryStore.list returns all keys in the store. This is not scalable for stores with many keys.
RemoteStore has a path attribute; RemoteStore.list only returns the keys that have Remotestore.path as a leading prefix. This makes sense, because attempting to list all the keys in s3 would not be very useful.
I'm increasingly convinced that all stores should have an interface that works for RemoteStore, i.e. a path attribute that serves to narrow the scope of list / create / get actions in the key space of the store.
We are not consistent about the
path
attribute of stores right now.localstore
has aroot
attribute that's basically a path (with typepathlib.path
),remotestore
has a v2-style explicitpath
attribute (with typestr
),zipstore
has a path attribute but I think its semantics are not likeremotestore.path
, sincezipstore.path
points to a location outside the zip file system, etc.I think we should normalize the
store.path
attribute along the following lines:store.path
is a string that names a location inside the filesystem modeled by the store class. All stores have such apath
attribute.store_instance.with_path('new_path')
(completely new path) orstore_instance.join_path('relative_path') (old path + relative_path), or
store_instance / 'relative_path'` as an abbreviation of the last one.StorePath
class goes away entirely, because we are giving stores their own path.(This API is largely influenced by the
yarl.URL
class, which I have been using quite a bit recently)Thoughts?
The text was updated successfully, but these errors were encountered: