General dataset ingestion for the Sustain ecosystem.
npm i
npm install variety-cli -g
- Download whatever data you are importing into some directory.
./run.sh <name_of_new_collection> <data_format> <filepath> <indexes> [-n]
, wherein:
Name of the new collection you are creating within mongodb.
Format of data you are ingesting. Accepted values are:
census
, format downloaded from nhgis.org, csv format, one header line.census_w_descriptive_header
, format downloaded from nhgis.org, csv format, two header lines. This will make a file inout/
which maps the first header to the descriptive one. This also will make a new parallel copy of the csv with two headers, but delete the second header in the original.neon
, Neon format, WIP.
Path to whatever file you are ingesting.
Space seperated string of indexes to create.
Don't re-build. Useful when not developing.
./run.sh county_race census ./path/to/censuscsv.csv "GISJOIN index2 index3"