Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

End-to-end resolution testing #27

Closed
11 of 13 tasks
jmsfltchr opened this issue Apr 1, 2020 · 2 comments
Closed
11 of 13 tasks

End-to-end resolution testing #27

jmsfltchr opened this issue Apr 1, 2020 · 2 comments
Assignees
@jmsfltchr
Labels
priority: high status: solved type: feature

Comments

@jmsfltchr
Copy link
Contributor

jmsfltchr commented Apr 1, 2020 •
edited
Loading

Objective

Broadly, the generation of a query from the explanations of an inferred answer is complete #24, and forward chaining to complete a KB is also working #23.

What remains is to interface the two, and iron out any issues that arise. This includes:

  • Enforce that all data have keys Enforce that all data have keys #25
  • Generate random keys when forward chaining (because inserting data requires keys)
  • Add explicit variables to relations with anonymous variables when building the resolution query
  • Upgrade the framework to work for any pattern, not just conjunctive statements Handle all types of pattern in resolution testing #26
  • Investigate whether we do need unification
  • Use different variables for each application of a rule in the case that we don't unify
  • Write test for the full procedure
  • Enforce that all entity and relation types have keys
  • Implement completeness check
  • De-duplication of inferred concepts during forward chaining
  • Test server architecture
  • BDD integration with client-java (only to be implemented in Grakn Core)
  • BDD integration with grakn-core
@jmsfltchr jmsfltchr self-assigned this Apr 1, 2020
@jmsfltchr jmsfltchr mentioned this issue Apr 1, 2020
Closed
@jmsfltchr jmsfltchr mentioned this issue Apr 30, 2020
Closed
@alexjpwalker
Copy link
Member

client-java needs to be fixed in order to run the graql reasoner tests: typedb/typedb-driver@78613e7

@jmsfltchr jmsfltchr mentioned this issue Jun 29, 2020
Merged
lolski pushed a commit to typedb/typedb that referenced this issue Jul 1, 2020
@jmsfltchr
Resolution Testing Framework (#5798)
5e30fa1 
## What is the goal of this PR?

We introduce a Resolution Testing framework for Grakn's reasoner that can automatically verify Grakn's reasoning correctness for a given test case. 

## What are the changes implemented in this PR?

Implements solution no. 5 from our original design issue typedb/typedb-behaviour#20

Solves typedb/typedb-behaviour#27
This issue contains a breakdown of many of the work items

Solves typedb/typedb-behaviour#23
Solves typedb/typedb-behaviour#24

The approach has some limitations, including:
- The framework only considers queries over data and not schema, so mixed data and schema queries using `type` and/or `sub` are not supported.
- Attribute variables must be given explicitly 
  - in the queries tested
  - in the rules
- Certain `VarProperty`s haven't been implemented yet, such as equality properties. This requires minor effort.
@jmsfltchr
Copy link
Contributor Author

Solved by typedb/typedb#5798

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jmsfltchr jmsfltchr

Labels
priority: high status: solved type: feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Resolution Testing jmsfltchr/verification-old
Resolution Testing jmsfltchr/typedb-behaviour
3 participants
@alexjpwalker @jmsfltchr @grabl