Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle named graphs #43

Open
jmatsushita opened this issue Jan 13, 2017 · 9 comments
Open

Handle named graphs #43

jmatsushita opened this issue Jan 13, 2017 · 9 comments

Comments

@jmatsushita
Copy link
Collaborator

jmatsushita commented Jan 13, 2017

With the recent version update, it is now possible to submit JSON-LD docs which includes an @graph key. However, named graphs are not handled yet. (More precisely, if there is a top level @id then the @graph value is just ignored).

Given the discussion on levelgraph/levelgraph#43 I'm thinking of first implementing this using a graph triple property for put, and a slow filter JS-land approach for get.

The performance penalty will be big on anything that tries to use named graphs but it's probably good to start here before doing anything more involved. Things will probably take shape as I implement but does this make sense @mcollina ?

@jmatsushita
Copy link
Collaborator Author

To clarify, I'm not looking at the sublevel approach yet because I'm interested in provenance cross graph queries and graph per resource or graph per aspect patterns

@mcollina
Copy link
Collaborator

@jmatsushita I have not so much time to dig into this, so I would leave it to you. As a general rule, start simple and then optimize.
I think the filter should happen as soon as possible in search() at the clause level. In that way we can get thing faster.
Another option might be to write levelsubgraph, as a thing that maps multiple levelgraphs based on the graph. Use https://www.npmjs.com/package/subleveldown and not sublevel.

@jmatsushita
Copy link
Collaborator Author

Thanks for the input. I'll document my progress here.

@jacoscaz
Copy link

jacoscaz commented Mar 7, 2017

@jmatsushita @mcollina we've just published the first proof of concept of node-quadstore, an attempt at using LevelGraph's LevelDB-based storage approach with a quad-oriented indexing strategy to implement a graph database with native support for cross-graph queries. It still uses 6 indexes. As we do not want to re-invent the wheel, assuming our solution works out we'd be more than happy to contribute to LevelGraph in any way you might deem fit.

@mcollina
Copy link
Collaborator

mcollina commented Mar 7, 2017

feel free to send a PR to LevelGraph if you want to! :)

@BigBlueHat
Copy link
Member

Thinking through @jmatsushita's work in #54--which you can demo in a fork of the playground--it's not possible to keep the provenance of a single triple if it exists (or arrives via) two different named graphs.

Try the example in that playground fork, and change 73 to 74...there will be a triple left behind so it's clear that 73 was inserted, but you'd not be able to re-compute the @graph as all its references are lost relative to the triples--all being replaced by the identical graph with different name (i.e. 74).

If the graph property were stored as an array (or object for easier uniqueness), that would solve for this issue...but I'm not sure it'd scale so great...

Thoughts? 💭

@jmatsushita
Copy link
Collaborator Author

jmatsushita commented Apr 18, 2017

Hi @BigBlueHat thanks for your continued interest! Indeed this is a bug, or a design shortcoming of my fork, in the sense that it cannot work with levelgraph's current indexing strategy. Basically the graph property is just metadata added to triples so 2 triples with a differing graph are in fact the same triple for levelgraph, hence the behavior you see on the playground.

There would need to be a change (or an option) in levelgraph in order for quads to be indexed. There was some chat about this on the levelgraph issue tracker levelgraph/levelgraph#43

It seems that node-quadstore manages to do quads with 6 indices and now is aligning with recent work on RDF/JS. It also has a LevelUp compatible storage backend and its Graph API (not the RDF/JS one) is fairly close to the LevelGraph one. So it should be possible to write a quadstore-jsonld module, or even better a rdfjs-jsonld module.

As @elf-pavlik suggested in levelgraph/levelgraph#43 (and if quad indexing is added into levelgraph) creating a levelgraph-rdf layer would mean that more robust and interoperable approaches to levelgraph-jsonld and levelgraph-n3 would be able to exist. Adding something like clownface would also allow something very similar to navigation API from LevelGraph as well.

Given all of this recent good work I would be inclined to watch that issue levelgraph/levelgraph#43 for possible addition of quad indexing into levelgraph and consider my fork of levelgraph-jsonld as a deadend. I will also definitely give a try to @jacoscaz's work on node-quadstore which looks very promising!

@jacoscaz
Copy link

@jmatsushita @BigBlueHat I think a levelgraph-rdf layer would make the most sense as it would allow levelgraph to become a part of the growing RDF/JS ecosystem. A rdfjs-jsonld module would then be a wonderful contribution to the whole ecosystem. We ourselves are thinking about extracting the query layer of node-quadstore into a dedicated rdfjs-* module.

@BigBlueHat
Copy link
Member

I'm definitely down for more collaboration! 😁 I'm already lurking in all the places. 😉 Exciting times ahead!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants