JSON Mangler

JSON mangler is a compressor with name mangling for JSON objects. Using a schema, the compressor generates mappings for any given objects and compresses them which makes the objects smaller in size (and harder to read as a side-effect).

API

Compression:

const Compressor = require('json-mangler').Compressor;
const data = [
  { name: 'Jack', age: 43, occupation: 'Frontend Developer', employed: true },
  { name: 'Steve', age: 28, occupation: 'Backend Developer', employed: true },
  { name: 'Bob', age: 23, occupation: 'Frontend Developer', employed: false },
];
// Create compressor with schema
const compressor = new Compressor([{
  name: true,
  age: true,
  occupation: true,
  employed: true
}]);

// Compress data with mutation and compression info
const result = compressor.compress(data, true, true);

console.log(result.compressed); // [{ a: 'Jack', b: 43, c: 'Frontend Developer', d: true }, ...]
console.log(result.mappings); // a:*.name;b:*.age;c:*.occupation;d:*.employed; (Use for decompressing this object)
console.log(result.info); // { originalSize: 226, newSize: 163, diff: 63, time: 0 }

Decompression:

const Decompressor = require('json-mangler').Decompressor;
// Compressed object and mappings can be obtained from the compressor results
const compressed = [
  { a: 'Jack', b: 43, c: 'Frontend Developer', d: true },
  { a: 'Steve', b: 28, c: 'Backend Developer', d: true },
  { a: 'Bob', b: 23, c: 'Frontend Developer', d: false }
];
const mappings = 'a:*.name;b:*.age;c:*.occupation;d:*.employed;';

// Decompress a compressed object using its mappings
const decompressor = new Decompressor(mappings);

// Decompress data with mutation and decompression info
const results = decompressor.decompress(compressed, true, true);

console.log(results.decompressed); // The original object
console.log(results.info); // { originalSize: 163, newSize: 226, diff: -63, time: 0 }

Compressor

Commonjs:

const Compressor = require('json-mangler').Compressor;

ES6:

import { Compressor } from 'json-mangler';

Is a constructor that takes a schema for the objects. Any properties included in the schema will be considered in the compression. The constructed object has the following members:

• mappings: A read-only string which contains the mappings generated by the provided schema.
• compress(json, noClone, calcInfo): A method which takes an object to compress with two flags:
  • noClone (default: false): If true, compression will avoid cloning the object which makes the compression faster but mutates the original object by reference.
  • calcInfo (default: false): If true, the compression information will be calculated and available on the returning object with key info.

The compression result will contain the following properties:

• compressed: The compressed object.
• mappings: The mappings generated for the object.
• info: The compression info (only if calcInfo flag was true).

Decompressor

Commonjs:

const Decompressor = require('json-mangler').Decompressor;

ES6:

import { Decompressor } from 'json-mangler';

Is a constructor that takes a mappings string. The constructed object has the following members:

• mappings: A read-only string which contains the mappings provided in the constructor.
• decompress(json, noClone, calcInfo): A method which takes a compressed-object to decompress with two flags:
  • noClone (default: false): If true, decompression will avoid cloning the object which makes the decompression faster but mutates the original object by reference.
  • calcInfo (default: false): If true, the decompression information will be calculated and available on the returning object with key info.

The decompression result will contain the following properties:

• decompressed: The decompressed object.
• info: The decompression info (only if calcInfo flag was true).

Real-Life Application

1. A good application for this module is when a great amount of objects with the same structure is served to the client from a server. Let's consider the following example:
2. We have a service (web application + server) which is connected to a database of movies information
3. A complete movie object looks like the following:

{
    "Title": "Pulp Fiction",
    "Year": "1994",
    "Rated": "R",
    "Released": "14 Oct 1994",
    "Runtime": "154 min",
    "Genre": "Crime, Drama",
    "Director": "Quentin Tarantino",
    "Writer": "Quentin Tarantino (stories), Roger Avary (stories), Quentin Tarantino",
    "Actors": "Tim Roth, Amanda Plummer, Laura Lovelace, John Travolta",
    "Plot": "The lives of two mob hitmen, a boxer, a gangster & his wife, and a pair of diner bandits intertwine in four tales of violence and redemption.",
    "Language": "English, Spanish, French",
    "Country": "USA",
    "Awards": "Won 1 Oscar. Another 62 wins & 69 nominations.",
    "Poster": "https://m.media-amazon.com/images/M/MV5BNGNhMDIzZTUtNTBlZi00MTRlLWFjM2ItYzViMjE3YzI5MjljXkEyXkFqcGdeQXVyNzkwMjQ5NzM@._V1_SX300.jpg",
    "Ratings": [
      {
        "Source": "Internet Movie Database",
        "Value": "8.9/10"
      },
      {
        "Source": "Rotten Tomatoes",
        "Value": "92%"
      },
      {
        "Source": "Metacritic",
        "Value": "94/100"
      }
    ],
    "Metascore": "94",
    "imdbRating": "8.9",
    "imdbVotes": "1,655,375",
    "imdbID": "tt0110912",
    "Type": "movie",
    "DVD": "19 May 1998",
    "BoxOffice": "N/A",
    "Production": "Miramax Films",
    "Website": "N/A"
}

3. The JSON object of the movie title "Pulp Fiction" is 1031 bytes in size (with white spaces removed)

4. The database contains 500,000 movie titles (approximately 515.5 MB) and the service has on average 200,000 daily users

5. If each user fetches 50 movies in their visit (consider the frontpage showcasing new movies in a list), the server is serving ~300 GB of data each month

6. Compressing the database using this module would reduce the storage size to 434.5 MB and the monthly bandwidth to ~262.3 GB

7. This data would be decompressed on the client side before being displayed to the users

8. Tabular data converted to JSON would usually end up as an array of objects with the same properties. Mangling can reduce the total size greatly.

Tests

npm test