file-storage

Objects storage in file system.

Lets suppose you need to:

• save an object in files systems in straigthforward manner, in readable files (i.e. JSON);
• have a well defined interface to store and return these objects;
• have type polimorphism returning the concrete class you persisted;
• have ways to perform parametrized queries on this repository;
• have the possibility to extend Object concept with 'resources' which are files associated to a given type (new concept);
• and more.

Depending on your needs, the use of a database with file storage is not necessary, and moreover easy access to these files does not mean that you can easily find usefull information just by crawling file system.

These are some of the capabilities provided by file-storage.

Usage

Include latest version to your project.

		<dependency>
			<groupId>io.github.thiagolvlsantos</groupId>
			<artifactId>file-storage</artifactId>
			<version>${latestVersion}</version>
		</dependency>

Add @EnableFileStorage to you app

It will provide an instance of IStorageFile which is the generic interface for objects storage.

...
@EnableFileStorage
public class Application {
	...main(String[] args) {...}
}

Add @FileRepo related annotations to your object

Its kind of Hibernate annotations, but in this case we are annotating information to describe how a given object will be persisted to the file system.

Bellow the example of an object of type Project with a set of possible annotations. The semantics of each annotation is explained bellow in sequence.

...

@FileRepo("projects")
public class Project {
	// file id field, objects can be found by keys (@FileKey) or ids(@FileIds).
	@FileId 
	private Long id; 

	// concurrency control, increases every save action
	@FileRevision
	private Long revision; 

	//creation time, only set once on first save action
	@FileCreated
	private LocalDateTime created = LocalDateTime.now(); 

	// update time, defaul behavior LocalDateTime.now() when changes are made.
	@FileChanged
	private LocalDateTime changed = LocalDateTime.now(); 

	// project name, if there are multipl keys, attribute 'order' can be used.
	@FileKey(order = 0)
	private String name; 

	// alias for project parent
	private ProjectAlias parent; 

	@FileChanged(UUIInitializer.class)
	private UUID uuid; // example of custom changed info

	...
}

The usage of annotations above has the following semantics:

Annotation	Semantics
@FileRepo	REQUIRED. Stands for the entity repository name on file system. In the example above, supposing a base directory /base, the Project objects will be persisted to directory /base/@projects acording to the value in the annotation.
@FileName	Stands for the entity file name in system. The full name is calculated by IFileSerializer based on presence of this annotation or not. In the example above, supposing: a base directory /base; a JSON serializer; a project named alpha, the resulting file name will be /base/@projects/alpha/data.json. This annotation replaces the data substring when present.
@FileId	REQUIRED. Stands for id field(s), this field is set automatically on saving actions, similar to @Id in Hibernate
@FileRevision	Stands for a concurrency control on saving actions in order to avoid rewrite of data without having set the right revision (version behind current), similar to @Revision in Hibernate
@FileCreate	Stands for an attribute that will be keeped unchanged after the first saving action. Kind of Hibernate audit annotations.
@FileChanged	Stands for an attribute that will be changed on every saving action. Kind of Hibernate audit annotations. Both @FileCreated and @FileChanged admit custom generators.
@FileKey	REQUIRED. Stands for attributes that will be used as directory structure in file system. In the previous example, if we have a Project named example and we send a IFileStorage save it a directory with name /data/@projects/example will be created where the serialized version will lay and its resources will reside under folder /data/@project/example/data@resources.

IFileStorage will refuse saving objects without minimal annotations: @FileRepo, @FileId and @FileKey. The other annotations are optional.

Using IFileStorage to persist objects

Application using a service...

...
@EnableFileStorage
public class Application {
	...main(String[] args) {
		...
		ProjetService service = ...;
		service.save(new Project("example"));
		...
	}
}

to avoid duplicaton and using a repository...

...
@Service
public class ProjectService {
	private @Autowired ProjectRepository repository;

	public void save(Project p) {
		if(!repository.exists(p)) {
			respository.save(p);
		} else {
			throw new ObjectAlreadExists("choose you exception!");
		}
	}
}

to save object in file system.

...
@Respository
public class ProjectRespository ... {

private @Value("#{storage.directory:/base}") String dir;
private @Autowired IFileStorage storage;

private File baseDir() {
	return new File(dir);
}

public boolean exists(Project p) {
	return storage.exists(baseDir(),p);
}

public void save(Project p) {
	storage.write(baseDir(),p);
}

...

After this call, supposing an initially empty folder /base, we`ll have the structure:

dir	content
/base/@projects	With data related to all projects
/base/@projects/example/data.json	With data related to the specific project example
/base/@projects/example/data@resources	With all project example resource files
/base/@projects/.data.current	Controll file for ids sequencial.
/base/@projects/.data.index	Controll directory for mapping keys to ids and vice-versa.

If the annotation @FileEntityName is present, the structure will be the same above with the informed name replacing all ocurrences of substring data. i.e. using @FileEntityname("meta") the object JSON will be /base/@projects/example/meta.json and so on.

Interface IFileSerializer abstraction

The serializer in IFileStorage is reponsible for preparing and saving the object itself to the file system.

The default implementation saves/restores objects in JSON format which can be easily read.

You can change it using setSerializer(...) on IFileStorage interafce.

Performing queries on objects

Call IFileStorage passing a Predicate object which will be used as filter for selection.

...
	//filtering project with name starting with 'm'
	Predicate<Project> p = (p)->p.getName().startsWith("m");
	respository.search(p);
...

The predicate can be as complex as you want. For the repository its complexity wil not matter, all projects will be verified.

...
private @Value("#{storage.directory:/data}") String dir;
private @Autowired IFileStorage storage;

private File baseDir() {
	return new File(dir);
}

private List<Project> query(Predicate<Project> filter) {
	return storage.list(baseDir(),Project.class,new FilePredicate(filter),null,null);
}
...

Pagination and Sorting are always optional

You can use, or not, FilePaging and FileSorting for paging and sorting on any search methods, for objects or resources.

Resources - new object dimension built-in service

An extension of object concept to cope also files as part of object elements besides attributes and methods.

For example, suppose an entity called Template, it can have a name attribute, and a content attribute. Depending on how much templates are possible with that name we could have to create a list attribute to keep track on then.

Another option is just have an attribute with a name, and the templates related to that template instance saved as resources in /base/@templates/mytemplate/data@resources.

Check IFileStorage.*Resource*() methods to save/update/query this feature.

Using file-storage in conjuction with git-transactions

Imagine a world without databases (I didn`t say without 'data'), a scenario where you already have object keys to access information in a straightforward manner. Yes, you can do it by using a NoSql database, but you already have your file system and can use it to navigate/edit your data.

Why not organizing you objects in directories that can be easily accessed? Furthermore use an API like git-transactions to automatically pull/commit/push this structure to a Git repository. It`s a perfect match, an object API to write data into file system in a simple structure, and an API to automatically have it persisted in your Git repository, there is your database with:

• Easy visualization;
• Resiliency;
• History.

Just to be clear, I`not saying that this is a 'database' for massive data, but most part of applications can use such a simpler structure where objects are stored in file systems as JSON without the majority of restrictions imposed by relational databases.

Build

Localy, from this root directory call Maven commands or bin/<script name> at your will...