Perspective benchmark

[jlb6740]

Hi, I wanted to check out the perspective benchmark because I am interested in seeing the benefit wasm brings. I'd not heard of perspective before now but it looks to be some library and apis to help visualize streaming data in javascript and python. From the readme didn't see build instructions so I went to the developer guide and tried to follow those instructions. I've gotten to the point to where I am running to an error:

`$ node build.js
internal/modules/cjs/loader.js:1023
throw err;
^

Error: Cannot find module '@finos/perspective-build/wasm'
Require stack: ....`

This is after running 'yarn build' at the root and fixing a bunch of other build stoppages before now. Here the error seems straight forward and it looks like I am just missing a dependency and need to run npm install in some folder. However both "npm install wasm-pack" and "npm install wasm" are failing at the root. The installs are failing for different reasons. Also, I don't understand the significance of the directory structure in the cannot find module message. Note, wasm-pack was already installed on my system under ~/.cargo/bin/ and I went to the wasm-pack website to re-install it, but I take it that is not what is wanted here and that it should be installed via npm? Not knowing the purpose of wasm-pack here it is not clear to me the significance of picking either install method. Any suggestions would be helpful.

[texodus]

Apologies, the development documentation is out-of-date.

You have the correct wasm-pack, the one from the official install site; the build script just looks for wasm-pack on your PATH, so you may need to add ~/cargo/.bin to make this accessible (in bash which wasm-pack will tell you). A good place to look for hints if you have problems is the azure-pipelines.yml, as these are the exact install commands for our CI builds on Windows, OSX and Linux. Feel free to post in this thread where this gets you and I can help you get Perspective building.

However, you don't need to build Perspective from source to use it or benchmark it. Just follow the installation instructions on the installation section of the docs for JavaScript (or Python).

You may also want to check out the existing benchmark suite in tools/perspective-bench, which downloads descending versions of Perspective from NPM, benchmark them, then write the results to an Apache Arrow blob so Perspective feature version history relative to performance can be visualized in Perspective. It's a little cumbersome to use as its only for development, but it may help get you started.

I would love to start publishing Perspective benchmarks, at least relative to its own previous releases if not to other similar DBs, so very interested to see what you come up with!

[sv2]

Hi !
Following this discussion, I've put together a quick benchmark test, for loading and querying data using Perspective in node app:
https://github.com/slanatech/perspective-test

I'm interested in processing time series data, so data schema is very simple:

{ ts: 'datetime', value: 'integer' }

Test dataset is 864000 rows - a row for each second, representing 10 days of data.
I measure memory usage and timing for loading this dataset from the array, and from the arrow file,
and then a time to execute groupby query aggregating data per minute:

  const view = await pTable.view({
    columns: ['tb', 'value'],
    expressions: [`//tb\nbucket("ts", '1m')`],
    group_by: ['tb'],
    aggregates: { value: 'count' },
  });

For comparison, I've also added a test of executing the same query on the same data using Arquero

Here are my results:

$ npm start

> perspective-test@1.0.0 start
> node --expose-gc test.js


Perspective Table Test: data load from array
Memory increase: Perspective data load from array
  RSS      : 62.9 MB (211103744 -> 274001920)
  HeapTotal: 1.05 MB (107044864 -> 108093440)
  HeapUsed : -2.08 MB (74620320 -> 72540208)
  External : 14.49 MB (69707682 -> 84195990)
  arrayBuffers : -10.81 MB (21552851 -> 10742129)
  wasmHeap : 44.56 MB (16777216 -> 61341696)

Table load time: 1167 msec, rows: 864000
Aggregation query time: 1093 msec, result rows: 14401

Perspective Table Test: data load from Arrow dataset
Memory increase: Perspective data load from Arrow dataset
  RSS      : 5.68 MB (734044160 -> 739725312)
  HeapTotal: -1.31 MB (107782144 -> 106471424)
  HeapUsed : -236.06 KB (69892048 -> 69655984)
  External : 20.74 MB (479509142 -> 500245910)
  arrayBuffers : 10.37 MB (10742129 -> 21110513)
  wasmHeap : 0  (456654848 -> 456654848)

Table load time: 291 msec, rows: 864000
Aggregation query time: 871 msec, result rows: 14401

Arquero Table Test: data load from array
Memory increase: Arquero data load from array
  RSS      : 26.59 MB (740839424 -> 767430656)
  HeapTotal: 12.79 MB (107257856 -> 120045568)
  HeapUsed : 13.83 MB (69695096 -> 83527368)
  External : -10.37 MB (489877526 -> 479509142)
  arrayBuffers : 0  (10742129 -> 10742129)

Table load time: 39 msec, rows: 864000
Aggregation query time: 72 msec, result rows: 14401

Wanted to check if these results are in sync with expectations.
As far as I can see, they match with what is reported by tools/perspective-bench

Perhaps there are improvements I should make in data loading / querying code - kindly let me know

Thanks in advance!