Skip to content

Commit

Permalink
add v4
Browse files Browse the repository at this point in the history
  • Loading branch information
yaacov committed Feb 17, 2020
1 parent 93d5360 commit 97a2cc4
Show file tree
Hide file tree
Showing 44 changed files with 7,580 additions and 31 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
#

# Major version directory
major:=v4
major:=v5

# Details of the version of 'antlr' that will be downloaded to process the
# grammar and generate the parser:
Expand Down
56 changes: 28 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@

Tree Search Language (TSL) is a wonderful human readable filtering language.

[![Go Report Card](https://goreportcard.com/badge/github.com/yaacov/tree-search-language/v4)](https://goreportcard.com/report/github.com/yaacov/tree-search-language/v4)
[![Go Report Card](https://goreportcard.com/badge/github.com/yaacov/tree-search-language/v5)](https://goreportcard.com/report/github.com/yaacov/tree-search-language/v5)
[![Build Status](https://travis-ci.org/yaacov/tree-search-language.svg?branch=master)](https://travis-ci.org/yaacov/tree-search-language)
[![GoDoc](https://img.shields.io/static/v1?label=godoc&message=reference&color=blue)](https://pkg.go.dev/github.com/yaacov/tree-search-language/v4/pkg/tsl?tab=doc)
[![GoDoc](https://img.shields.io/static/v1?label=godoc&message=reference&color=blue)](https://pkg.go.dev/github.com/yaacov/tree-search-language/v5/pkg/tsl?tab=doc)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

The TSL language grammar is human readable and similar to SQL syntax.
Expand All @@ -30,9 +30,9 @@ The TSL language grammar is human readable and similar to SQL syntax.

You can use the TSL package to add uniform and powerful filtering to your RESTful API or GraphQL services, implement brewing-recipe searches on your smart tea brewer, or even make your own memory based "SQL like" server as we do in our `tsl_mem` CLI example.

([more examples](/v4/cmd/))
([more examples](/v5/cmd/))

Here is our `tsl_mem`CLI tool ([code](/v4/cmd/tsl_mem)), it's an in-memory search engine, it is using the TSL package to filter through an in-memory array of books using "SQL like" `tsl phrases`:
Here is our `tsl_mem`CLI tool ([code](/v5/cmd/tsl_mem)), it's an in-memory search engine, it is using the TSL package to filter through an in-memory array of books using "SQL like" `tsl phrases`:

``` bash
$ ./tsl_mem -i "spec.rating is not null and author ~= 'Joe'" -o prettyjson
Expand Down Expand Up @@ -127,28 +127,28 @@ go get -u github.com/onsi/ginkgo/ginkgo

``` bash
# Install the base package
go get "github.com/yaacov/tree-search-language/v4/pkg/tsl"
go get "github.com/yaacov/tree-search-language/v5/pkg/tsl"

# Install all walkers
go get "github.com/yaacov/tree-search-language/v4/pkg/walkers/..."
go get "github.com/yaacov/tree-search-language/v5/pkg/walkers/..."

# Or pick the walker needed
go get "github.com/yaacov/tree-search-language/v4/pkg/walkers/sql"
go get "github.com/yaacov/tree-search-language/v4/pkg/walkers/mongo"
go get "github.com/yaacov/tree-search-language/v4/pkg/walkers/ident"
go get "github.com/yaacov/tree-search-language/v4/pkg/walkers/graphviz"
go get "github.com/yaacov/tree-search-language/v5/pkg/walkers/sql"
go get "github.com/yaacov/tree-search-language/v5/pkg/walkers/mongo"
go get "github.com/yaacov/tree-search-language/v5/pkg/walkers/ident"
go get "github.com/yaacov/tree-search-language/v5/pkg/walkers/graphviz"
```

#### Installing the command line examples using `go get`

See CLI tools usage [here](https://github.com/yaacov/tree-search-language#cli-tools).

``` bash
go get -v "github.com/yaacov/tree-search-language/v4/cmd/tsl_parser"
go get -v "github.com/yaacov/tree-search-language/v4/cmd/tsl_mongo"
go get -v "github.com/yaacov/tree-search-language/v4/cmd/tsl_sqlite"
go get -v "github.com/yaacov/tree-search-language/v4/cmd/tsl_gorm"
go get -v "github.com/yaacov/tree-search-language/v4/cmd/tsl_graphql"
go get -v "github.com/yaacov/tree-search-language/v5/cmd/tsl_parser"
go get -v "github.com/yaacov/tree-search-language/v5/cmd/tsl_mongo"
go get -v "github.com/yaacov/tree-search-language/v5/cmd/tsl_sqlite"
go get -v "github.com/yaacov/tree-search-language/v5/cmd/tsl_gorm"
go get -v "github.com/yaacov/tree-search-language/v5/cmd/tsl_graphql"
```

## Syntax examples
Expand Down Expand Up @@ -202,12 +202,12 @@ dot file.dot -Tpng > image.png

## Code examples

For complete working code examples, see the CLI tools [directory](/v4/cmd)
For complete working code examples, see the CLI tools [directory](/v5/cmd)
( see more on TSL's CLI tools usage [here](https://github.com/yaacov/tree-search-language#cli-tools) ).

##### ParseTSL

The `tsl` package include the ParseTSL [code](/v4/pkg/tsl/tsl.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v4/pkg/tsl#ParseTSL) method for parsing TSL into a search tree:
The `tsl` package include the ParseTSL [code](/v5/pkg/tsl/tsl.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v5/pkg/tsl#ParseTSL) method for parsing TSL into a search tree:
``` go
tree, err := tsl.ParseTSL("name in ('joe', 'jane') and grade not between 0 and 50")
```
Expand All @@ -218,13 +218,13 @@ After parsing the TSL tree will look like this (image created using the `tsl_par

##### sql.Walk

The `walkers` `sql` package include a helper sql.Walk ([code](/v4/pkg/walkers/sql/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v4/pkg/walkers/sql#Walk)) method that adds search to [squirrel](https://github.com/Masterminds/squirrel)'s SelectBuilder object:
The `walkers` `sql` package include a helper sql.Walk ([code](/v5/pkg/walkers/sql/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v5/pkg/walkers/sql#Walk)) method that adds search to [squirrel](https://github.com/Masterminds/squirrel)'s SelectBuilder object:

``` go
import (
...
sq "github.com/Masterminds/squirrel"
"github.com/yaacov/tree-search-language/v4/pkg/walkers/sql"
"github.com/yaacov/tree-search-language/v5/pkg/walkers/sql"
...
)

Expand Down Expand Up @@ -253,12 +253,12 @@ SELECT name, city, state FROM users WHERE (name IN (?,?) AND grade NOT BETWEEN ?

##### mongo.Walk

The `walkers` `mongo` package include a helper mongo.Walk ([code](/v4/pkg/walkers/mongo/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v4/pkg/walkers/mongo#Walk)) method that adds search bson filter to [mongo-go-driver](https://github.com/mongodb/mongo-go-driver):
The `walkers` `mongo` package include a helper mongo.Walk ([code](/v5/pkg/walkers/mongo/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v5/pkg/walkers/mongo#Walk)) method that adds search bson filter to [mongo-go-driver](https://github.com/mongodb/mongo-go-driver):

``` go
import (
...
"github.com/yaacov/tree-search-language/v4/pkg/walkers/mongo"
"github.com/yaacov/tree-search-language/v5/pkg/walkers/mongo"
...
)

Expand All @@ -274,12 +274,12 @@ cur, err := collection.Find(ctx, filter)

##### graphviz.Walk

The `walkers` `graphviz` package include a helper graphviz.Walk ([code](/v4/pkg/walkers/graphviz/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v4/pkg/walkers/graphviz#Walk)) method that exports `.dot` file nodes :
The `walkers` `graphviz` package include a helper graphviz.Walk ([code](/v5/pkg/walkers/graphviz/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v5/pkg/walkers/graphviz#Walk)) method that exports `.dot` file nodes :

``` go
import (
...
"github.com/yaacov/tree-search-language/v4/pkg/walkers/graphviz"
"github.com/yaacov/tree-search-language/v5/pkg/walkers/graphviz"
...
)

Expand All @@ -295,12 +295,12 @@ s = fmt.Sprintf("digraph {\n%s\n}\n", s)

##### ident.Walk

The `walkers` `ident` package include a helper ident.Walk ([code](/v4/pkg/walkers/ident/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v4/pkg/walkers/ident#Walk)) method that checks and mapps identifier names:
The `walkers` `ident` package include a helper ident.Walk ([code](/v5/pkg/walkers/ident/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v5/pkg/walkers/ident#Walk)) method that checks and mapps identifier names:

``` go
import (
...
"github.com/yaacov/tree-search-language/v4/pkg/walkers/ident"
"github.com/yaacov/tree-search-language/v5/pkg/walkers/ident"
...
)
...
Expand Down Expand Up @@ -336,12 +336,12 @@ tree, err = ident.Walk(tree, checkColumnName)

##### semantics.Walk

The `walkers` `semantics` package include a helper semantics.Walk ([code](/v4/pkg/walkers/semantics/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v4/pkg/walkers/semantics#Walk)) method that helps filter a list of objects using a `tsl tree`, and a `type` `semantics.EvalFunc` ([code](/v4/pkg/walkers/semantics/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v4/pkg/walkers/semantics#EvalFunc)) that return a record's value for a record key:
The `walkers` `semantics` package include a helper semantics.Walk ([code](/v5/pkg/walkers/semantics/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v5/pkg/walkers/semantics#Walk)) method that helps filter a list of objects using a `tsl tree`, and a `type` `semantics.EvalFunc` ([code](/v5/pkg/walkers/semantics/walk.go), [doc](https://pkg.go.dev/github.com/yaacov/tree-search-language/v5/pkg/walkers/semantics#EvalFunc)) that return a record's value for a record key:

``` go
import (
...
"github.com/yaacov/tree-search-language/v4/pkg/walkers/semantics"
"github.com/yaacov/tree-search-language/v5/pkg/walkers/semantics"
...
)
...
Expand Down Expand Up @@ -379,7 +379,7 @@ compliance, err = semantics.Walk(tree, eval)

## CLI tools

The example CLI tools showcase the TSL language and `tsl` golang package, see the [cmd](/v4/cmd) directory for code.
The example CLI tools showcase the TSL language and `tsl` golang package, see the [cmd](/v5/cmd) directory for code.

##### tsl_parser

Expand Down
4 changes: 2 additions & 2 deletions TSL.g4
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ tableName
;

columnName
: ( ( databaseName '.' )? tableName '.' )? IDENTIFIER
: IDENTIFIER
;

literalValue
Expand Down Expand Up @@ -85,7 +85,7 @@ IDENTIFIER
: '"' (~'"' | '""')* '"'
| '`' (~'`' | '``')* '`'
| '[' ~']'* ']'
| [a-zA-Z_] [a-zA-Z_0-9]* // TODO check: needs more chars in set
| [a-zA-Z_] [a-zA-Z_./0-9]* [a-zA-Z_0-9]* // TODO check: needs more chars in set
;

NUMERIC_LITERAL
Expand Down
7 changes: 7 additions & 0 deletions v5/cmd/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@


<p align="center">
<img src="https://raw.githubusercontent.com/yaacov/tree-search-language/master/img/search-248.png" alt="TSL Logo">
</p>

# Tree Search Language (TSL) Command Line Interface (CLI) Examples
42 changes: 42 additions & 0 deletions v5/cmd/model/model.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// Copyright 2018 Yaacov Zamir <kobi.zamir@gmail.com>
// and other contributors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package model for demo.
package model

// BookSpecs for demo.
type BookSpecs struct {
Pages uint `bson:"pages,omitempty" json:"pages,omitempty"`
Rating uint `bson:"rating,omitempty" json:"rating,omitempty"`
}

// Book for demo.
type Book struct {
Title string `bson:"title,omitempty" json:"title,omitempty"`
Author string `bson:"author,omitempty" json:"author,omitempty"`
Spec BookSpecs `bson:"spec,omitempty" json:"spec,omitempty"`
}

// Books are the demo list of books.
var Books = []interface{}{
Book{Title: "Book", Author: "Joe", Spec: BookSpecs{Pages: 100, Rating: 4}},
Book{Title: "Other Book", Author: "Jane", Spec: BookSpecs{Pages: 200, Rating: 3}},
Book{Title: "Some Book", Author: "Jane", Spec: BookSpecs{Pages: 50, Rating: 5}},
Book{Title: "Some Other Book", Author: "Jane", Spec: BookSpecs{Pages: 50}},
Book{Title: "Good Book", Author: "Joe", Spec: BookSpecs{Pages: 150, Rating: 4}},
Book{Title: "Some Great Book", Author: "Jane", Spec: BookSpecs{Pages: 550, Rating: 2}},
Book{Title: "Other Great Book", Author: "Jane", Spec: BookSpecs{Pages: 250}},
Book{Title: "My Big Book", Author: "Joe", Spec: BookSpecs{Pages: 15, Rating: 5}},
}
126 changes: 126 additions & 0 deletions v5/cmd/tsl_gorm/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
// Copyright 2018 Yaacov Zamir <kobi.zamir@gmail.com>
// and other contributors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package main.
package main

import (
"context"
"encoding/json"
"flag"
"fmt"
"log"

"github.com/yaacov/tree-search-language/v5/pkg/tsl"
"github.com/yaacov/tree-search-language/v5/pkg/walkers/ident"
"github.com/yaacov/tree-search-language/v5/pkg/walkers/sql"

"github.com/yaacov/tree-search-language/v5/cmd/model"
)

func check(err error) {
if err != nil {
log.Fatal(err)
}
}

// columnNamesMap mapps between user namespace and the SQL column names.
var columnNamesMap = map[string]string{
"title": "title",
"author": "author",
"spec.pages": "pages",
"spec.rating": "rating",
}

// checkColumnName checks if a coulumn name is valid in user space replace it
// with the mapped column name and returns and error if not a valid name.
func checkColumnName(s string) (string, error) {
// Chekc for column name in map.
if v, ok := columnNamesMap[s]; ok {
return v, nil
}

// If not found return string as is, and an error.
return s, fmt.Errorf("column \"%s\" not found", s)
}

func main() {
// Setup the input.
inputPtr := flag.String("i", "title is not null", "the tsl string to parse (e.g. \"title = 'Book'\")")
preparePtr := flag.Bool("p", false, "prepare a book collection for queries")
filePtr := flag.String("f", "./sqlite.db", "the sqlite database file name")
flag.Parse()

// Set context.
ctx := context.Background()

// Try to connect to sqlite driver.
tx, err := connect(ctx, *filePtr)
check(err)
defer tx.Commit()

// Create a clean books collection.
if *preparePtr {
err = prepareCollection(tx)
check(err)
}

// Parse input string into a TSL tree.
tree, err := tsl.ParseTSL(*inputPtr)
check(err)

// Check and replace user identifiers with the SQL table column names.
tree, err = ident.Walk(tree, checkColumnName)
check(err)

// Prepare SQL filter.
filter, err := sql.Walk(tree)
check(err)

// Create SQL query.
sql, args, err := filter.ToSql()
check(err)

// Query SQL table.
rows, err := tx.Model(&Book{}).Where(sql, args...).Select("*").Rows()
check(err)
defer rows.Close()

for rows.Next() {
var elem Book

// ScanRows scan a row into our gorm Book element.
err = tx.ScanRows(rows, &elem)
check(err)

// Conver gorm Book type to model Book type.
book := model.Book{
Title: elem.Title,
Author: elem.Author,
Spec: model.BookSpecs{
Pages: elem.Pages,
Rating: elem.Rating,
},
}

// Print out one Book in json format.
b, _ := json.Marshal(book)
fmt.Printf("%s\n", string(b))
}

// Check for errors and exit.
err = rows.Err()
check(err)
}
Loading

0 comments on commit 97a2cc4

Please sign in to comment.