Skip to content

Commit

Permalink
MMGIS 3.0.0 (#592)
Browse files Browse the repository at this point in the history
* Bump 2.11 (#515)

* Docs - Time Tiles - Clarify tagging

* Fix css misaligned panel arrows due to browser update

* Bugfix: Send options to WMS layers (#517)

* Bugfix: Send options to WMS layers

* remove console.log

* Bugfix: TimeUI zoom - use wheel instead of mousewheel event for ff

* Bugfix: Catch errors in api/shorten/expand

* #519 fix mmgisAPI.setTime start (#520)

* #518 time={starttime} in a WMS URL is filled with endtime (#522)

* ts-518 Inject time tile params better

* #518 Stop wms layer options for override wms parameters

* Bugfix: LayersTool - Fix collapse all headers

* #523 Chronice fix second loss (#524)

* Info.go

* #270 Fix nested header expansion - quickfix

* Fix Chronice overflow, update spice-kernel-conf-example and fix TimeUI startInPointMode

* #525 Make dynamicExtent requery timeEnabled aware. (#526)

* #525 Make dynamicExtent requery timeEnabled aware.

* #525 Make dynamicExtent requery controlled aware.

* #527 Bugfix: Vector layers with headings/bearings do not toggle on/off based on zoom cutoffs (#528)

* #529 Viewshed set initial target and observer heights, also doc improvements (#530)

* #531 Viewshed in-app docs (#532)

* #531 partial

* #531 Viewshed in-app docs

* gdal2customtiles.py remove downscale for dem tiles

* #533 Allow adding relative seconds to initial times (#534)

* #533 Allow adding relative seconds to initial times

* #533 Make sure not to overwrite config time value for initialend

* #535 dynamicExtent vector layer and filtering issues (#536)

* Spelling fixes

* #452 Updated Identifier tool to be a separated floating tool

- Need to test different UI combinations with other separated tools
- We may possibly want an option to make the tool not be separated to
maintain compatibility

* Fixed Identifier tool bug when using old config with non-separated tool

Bug was causing other tools to not open. This makes it backwards
compatible.

* JS-API - setTime: add customtimes

* Fix docs typo

* Allow Identifier and Legend tools to be on different sides if specified

* Bug fix - MeasureTool without dem config variable set

* DrawTool - include Created and Modified time part in file info

* Fix no data dynamicExtent responses

* Remove minor debug field

* Added downloadURL variable to allow users to download raster layers #546

* #544 Fix setTime currentTime timezone, default layer time objects (#548)

* #539 MeasureTool - linear x-axis (#554)

* Made downloadURL specific to tile and data layers, and updated docs #546

* #557 Viewer Panel - PDFs (#558)

* #557 Viewer Panel Pdfs - support with draw layers

* Fix issue with reloadLayer overwriting url

* Add CurtainTool

* Viewer Panel reads from all feature properties

* Improvements to bulk_tiles and added quantize_colormap #561

- quantize_colormaps will produce quantized colormaps using standard MPL
palettes for a single tiff or directory of files
- New option to use continuous/discrete colors for bulk_tiles
- Other minor improvements to bulk_tiles

* Enabled scalefactor and improved text readout for Identifier tool

Resolves:
- Consistent text alignment in Identifier Tool readouts #555
- Apply scale factor to Identifier Tool source data readouts #556

* #568 fix Deeplink to selected point or polygon (#569)

* #568 fix Deeplink to selected point or polygon

* #568 Give useKeyAsId priority over latlng

* #570 Info with dynamicExtent (#571)

* #572 mmgisAPI.getActiveTools (#573)

* #464 /configure-beta (#574)

* #464 Early COnfigure Remake

* #464 Add styles to new configure

* #464 New Configure 1

* #464 Rework Configure 2

* #464 New mission modals, general modals and snackbar

* #464 Integrate new config page into MMGIS server, start of layers tab

* #464 Start metaconfigs - home and ui

* #464 Start Layers

* #464 Map, Tile layer modal, settable config

* #464 Change layer params, vector-conf, colorpickers, start tools tab

* #464 Tools Tab

* #464 Tool Modal partial

* #464 Partial updates

* #464 More Tool configs

* #464 More work

* #464 Vector layer components

* #464 Preview Modal

* #464 Layer Organization

* #464 Pages and API Keys

* #464 Early Geodatasets

* Merge dev into new config feature branch (#541)

* Fix click on vectortile

* Fix missing Missions/mission path on tile layers

* #465 Disable Globe more thoroughly when off (#466)

* #467 GENERATE_SOURCEMAP ENV (#468)

* #469 Fix Tool Drag Handle Remains on Screen (#470)

* #471 Additional Body Metadata for Draw Webhooks (#472)

* #471 Webhook fields 1

* #471 Additional Body Metadata for Draw Webhooks

* #471 Add raw_file_description

* #474 ShadeTool - Fix Entity height calculation (#475)

* #474 ShadeTool- Fix Entity height calculation

* #474 Remove comment of bad calculation line

* #473 MeasureTool and IdentifierTool Layer Vars (#476)

* #473 Layer Specific IdentifierTool vars 1

* #473 MeasureTool and IdentifierTool Layer Vars

* #477 DrawTool - Filter state management (#478)

* #479 Remove restriction on Layer names (#480)

* #479 Remove Layer Name restrictions

* #479 Remove console.log

* #481 ShadeTool Touchups (#482)

* #481 ShadeTool Touchups

* #481 Fix docs

* #483 MeasureTool Fix nodata issue (#484)

* Minor ShadeTool Fixes - hide point and regen at med res

* Check for empty time configs in TimeControl

* #489 DrawTool - Move (#490)

* #489 Draggable DrawTool Features part 1

* #489 DrawTool - Move 2

* #494 Dropdown in the topbar for a selected feature’s properties links (#495)

* #485 Rover Image Overlay Improvements (#486)

* #485 Rover Image Overlay Improvments

* #485 Prevent reload/updates to layers through the mmgisAPI if one is already taking place

* #485 Support initialOpacity for imageOverlays w/ docs

* Remove dev consolelog

* #485 Rover Improvement fixes

* #485 Image overlay chache works (but is offset badly now)

* #485 Fix image_overlay updateVectorLayer flickering

* #485 Fix layerObj ref

* #496 Support Symbols in DrawTool Folders and Tags (#497)

* #491 panorama angle of view wonky near poles (#499)

* #492 updateVectorLayer calls causes flat panorama view to revert to wrapped panorama view (#501)

* Fix time replacement in IdentifierTool

* #500 getFiles if deeplinked to draw tool (#502)

* Update docker installation docs

* Fix typo in docs

* mmgisAPI.appendLineString returns false if layer was in a blocking state

* Touch up, move extraneous DrawTool destory

* If Globe Panel disabled, do not even run empty animate loop

* Bug: Deeplinked DrawTool layers are now immediately clickable

* Bug: Webhooks identify symbols in draw tags

* Minor bug fixes - missing deeplink layer error gone, search bar recenters and can force makelayer

* #505 Dynamic Vector Layers (#506)

* #505 Dynamic Vector Layer querying 1

* #505 Time queries, reselect improvement, download extent or full

* #505 Dynamic vector layers

* #504 ShadeTool Improvements and Multi-missionize (#507)

* #504 ShadeTool - User settable AzElRange

* #504 SPICE kernel scheduled downloads

* #504 update shadetool css

* #504 Update docs for SPICE conf

* #505 [Bug]: Support dynamicExtentMoveThresholf for non-geodatasets

* #508 Upgrade Dockerfile to Node.JS v20+ (#509)

* #508 Upgrade to NodeJS v20.11.1

* #508 Update Dockerfile 2, update nupmy

* Geodatasets to use spatial index, also DB_POOL envs

* #510 Temporal Geodatasets (#511)

* #510 Temporal Geodatasets

* #510 Fix new doc page path

* #270 Fix nested header expansion (#512)

* #498 LayersTool - Prevent toggle of loading layer (#513)

* Bump 2.11 (#515)

* Docs - Time Tiles - Clarify tagging

* Fix css misaligned panel arrows due to browser update

* Bugfix: Send options to WMS layers (#517)

* Bugfix: Send options to WMS layers

* remove console.log

* Bugfix: TimeUI zoom - use wheel instead of mousewheel event for ff

* Bugfix: Catch errors in api/shorten/expand

* #519 fix mmgisAPI.setTime start (#520)

* #518 time={starttime} in a WMS URL is filled with endtime (#522)

* ts-518 Inject time tile params better

* #518 Stop wms layer options for override wms parameters

* Bugfix: LayersTool - Fix collapse all headers

* #523 Chronice fix second loss (#524)

* Info.go

* #270 Fix nested header expansion - quickfix

* Fix Chronice overflow, update spice-kernel-conf-example and fix TimeUI startInPointMode

* #525 Make dynamicExtent requery timeEnabled aware. (#526)

* #525 Make dynamicExtent requery timeEnabled aware.

* #525 Make dynamicExtent requery controlled aware.

* #527 Bugfix: Vector layers with headings/bearings do not toggle on/off based on zoom cutoffs (#528)

* #529 Viewshed set initial target and observer heights, also doc improvements (#530)

* #531 Viewshed in-app docs (#532)

* #531 partial

* #531 Viewshed in-app docs

* gdal2customtiles.py remove downscale for dem tiles

* #533 Allow adding relative seconds to initial times (#534)

* #533 Allow adding relative seconds to initial times

* #533 Make sure not to overwrite config time value for initialend

* #535 dynamicExtent vector layer and filtering issues (#536)

* Spelling fixes

* #452 Updated Identifier tool to be a separated floating tool

- Need to test different UI combinations with other separated tools
- We may possibly want an option to make the tool not be separated to
maintain compatibility

* Fixed Identifier tool bug when using old config with non-separated tool

Bug was causing other tools to not open. This makes it backwards
compatible.

* JS-API - setTime: add customtimes

* Fix docs typo

* Allow Identifier and Legend tools to be on different sides if specified

* Bug fix - MeasureTool without dem config variable set

* DrawTool - include Created and Modified time part in file info

* Fix no data dynamicExtent responses

* Remove minor debug field

---------

Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov>
Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com>

* #464 More Geodataset page

* #464 More Geodataset work

* #464 Geodatasets page

* #464 More new configuration site work

* #464 More new config fixes

* #464 Serve both configs, saving, websockets, early datasets

* #464 Remove extra image

* #464 Home page among many other things

* #464 Configure-beta - Finish webhooks and dataset

* #464 Touch ups

* Update Feature Branch #464 (#550)

* Fix click on vectortile

* Fix missing Missions/mission path on tile layers

* #465 Disable Globe more thoroughly when off (#466)

* #467 GENERATE_SOURCEMAP ENV (#468)

* #469 Fix Tool Drag Handle Remains on Screen (#470)

* #471 Additional Body Metadata for Draw Webhooks (#472)

* #471 Webhook fields 1

* #471 Additional Body Metadata for Draw Webhooks

* #471 Add raw_file_description

* #474 ShadeTool - Fix Entity height calculation (#475)

* #474 ShadeTool- Fix Entity height calculation

* #474 Remove comment of bad calculation line

* #473 MeasureTool and IdentifierTool Layer Vars (#476)

* #473 Layer Specific IdentifierTool vars 1

* #473 MeasureTool and IdentifierTool Layer Vars

* #477 DrawTool - Filter state management (#478)

* #479 Remove restriction on Layer names (#480)

* #479 Remove Layer Name restrictions

* #479 Remove console.log

* #481 ShadeTool Touchups (#482)

* #481 ShadeTool Touchups

* #481 Fix docs

* #483 MeasureTool Fix nodata issue (#484)

* Minor ShadeTool Fixes - hide point and regen at med res

* Check for empty time configs in TimeControl

* #489 DrawTool - Move (#490)

* #489 Draggable DrawTool Features part 1

* #489 DrawTool - Move 2

* #494 Dropdown in the topbar for a selected feature’s properties links (#495)

* #485 Rover Image Overlay Improvements (#486)

* #485 Rover Image Overlay Improvments

* #485 Prevent reload/updates to layers through the mmgisAPI if one is already taking place

* #485 Support initialOpacity for imageOverlays w/ docs

* Remove dev consolelog

* #485 Rover Improvement fixes

* #485 Image overlay chache works (but is offset badly now)

* #485 Fix image_overlay updateVectorLayer flickering

* #485 Fix layerObj ref

* #496 Support Symbols in DrawTool Folders and Tags (#497)

* #491 panorama angle of view wonky near poles (#499)

* #492 updateVectorLayer calls causes flat panorama view to revert to wrapped panorama view (#501)

* Fix time replacement in IdentifierTool

* #500 getFiles if deeplinked to draw tool (#502)

* Update docker installation docs

* Fix typo in docs

* mmgisAPI.appendLineString returns false if layer was in a blocking state

* Touch up, move extraneous DrawTool destory

* If Globe Panel disabled, do not even run empty animate loop

* Bug: Deeplinked DrawTool layers are now immediately clickable

* Bug: Webhooks identify symbols in draw tags

* Minor bug fixes - missing deeplink layer error gone, search bar recenters and can force makelayer

* #505 Dynamic Vector Layers (#506)

* #505 Dynamic Vector Layer querying 1

* #505 Time queries, reselect improvement, download extent or full

* #505 Dynamic vector layers

* #504 ShadeTool Improvements and Multi-missionize (#507)

* #504 ShadeTool - User settable AzElRange

* #504 SPICE kernel scheduled downloads

* #504 update shadetool css

* #504 Update docs for SPICE conf

* #505 [Bug]: Support dynamicExtentMoveThresholf for non-geodatasets

* #508 Upgrade Dockerfile to Node.JS v20+ (#509)

* #508 Upgrade to NodeJS v20.11.1

* #508 Update Dockerfile 2, update nupmy

* Geodatasets to use spatial index, also DB_POOL envs

* #510 Temporal Geodatasets (#511)

* #510 Temporal Geodatasets

* #510 Fix new doc page path

* #270 Fix nested header expansion (#512)

* #498 LayersTool - Prevent toggle of loading layer (#513)

* Bump 2.11 (#515)

* Docs - Time Tiles - Clarify tagging

* Fix css misaligned panel arrows due to browser update

* Bugfix: Send options to WMS layers (#517)

* Bugfix: Send options to WMS layers

* remove console.log

* Bugfix: TimeUI zoom - use wheel instead of mousewheel event for ff

* Bugfix: Catch errors in api/shorten/expand

* #519 fix mmgisAPI.setTime start (#520)

* #518 time={starttime} in a WMS URL is filled with endtime (#522)

* ts-518 Inject time tile params better

* #518 Stop wms layer options for override wms parameters

* Bugfix: LayersTool - Fix collapse all headers

* #523 Chronice fix second loss (#524)

* Info.go

* #270 Fix nested header expansion - quickfix

* Fix Chronice overflow, update spice-kernel-conf-example and fix TimeUI startInPointMode

* #525 Make dynamicExtent requery timeEnabled aware. (#526)

* #525 Make dynamicExtent requery timeEnabled aware.

* #525 Make dynamicExtent requery controlled aware.

* #527 Bugfix: Vector layers with headings/bearings do not toggle on/off based on zoom cutoffs (#528)

* #529 Viewshed set initial target and observer heights, also doc improvements (#530)

* #531 Viewshed in-app docs (#532)

* #531 partial

* #531 Viewshed in-app docs

* gdal2customtiles.py remove downscale for dem tiles

* #533 Allow adding relative seconds to initial times (#534)

* #533 Allow adding relative seconds to initial times

* #533 Make sure not to overwrite config time value for initialend

* #535 dynamicExtent vector layer and filtering issues (#536)

* Spelling fixes

* #452 Updated Identifier tool to be a separated floating tool

- Need to test different UI combinations with other separated tools
- We may possibly want an option to make the tool not be separated to
maintain compatibility

* Fixed Identifier tool bug when using old config with non-separated tool

Bug was causing other tools to not open. This makes it backwards
compatible.

* JS-API - setTime: add customtimes

* Fix docs typo

* Allow Identifier and Legend tools to be on different sides if specified

* Bug fix - MeasureTool without dem config variable set

* DrawTool - include Created and Modified time part in file info

* Fix no data dynamicExtent responses

* Remove minor debug field

* #544 Fix setTime currentTime timezone, default layer time objects (#548)

---------

Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov>
Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com>

* #464 Final touches

* #464 Fix missing /

* #464 Update installation docs

* #464 Add downloadURL config var

* #464 Add documentation for configure dev and explain a idTool field better

* #464 Add IdTool scalefactor param to config-beta

* #464 Add useNameAsKey to configure-beta

* #464 Final touches, bump version

---------

Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov>
Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com>

* #542 Add Unity tf (#543)

* #542 Add Unity tf

* #542 Add Unity tf 2

* #542 Add all ENVs to tf

* #542 Update unity terraform

* #576 Vector Layer shapeIcon (#577)

* Update sds tf

* #578 Next/Previous Feature (#579)

* #578 Next/Previous part 1 - working next previous on index, menu ui that does nothing

* Quick fix

* #578 Some work on dropdown

* #578 Current Extent, Pan to and touchups

* #578 Arrow keys, first/last, bug fixes

* #578 dynamicExtent and time - partial

* #578 Small fixes, geom precision, ui tweaks

* #578 Better dynamicExtent next-preving

* #578 next/prev improvements

* #578 Next/Prev final touchups

* Minor fix - Description Nav dropdown properties order

* Create OPEN-SOURCE-CONTRIBUTIONS.md (#582)

* #584 Doc Link to Slack Channel (#585)

* Ts 584 (#586)

* #584 Doc Link to Slack Channel

* #584 Add more links to slack channel

* Use ROOT_PATH for configure-beta paths

* Fix ROOT_PATH in configure-beta

* Remove setup log

* Use relative configure-beta calls

* #587 COMPOSITE_TILE_DIR_STORE_MAX_AGE_MS and per layer refreshIntervals (#588)

* #587 COMPOSITE_TILE_DIR_STORE_MAX_AGE_MS and per layer refreshIntervals

* #587 Raster and Vector Refresh Intervals

* #587 Update more docs

* #587 Update more docs 2

* configure-beta calls / logic

* #583 Starting in Point Mode for Time doesn't work (#589)

* 3.0.0 Changelog

---------

Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov>
Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com>
  • Loading branch information
3 people authored Oct 14, 2024
1 parent 605694f commit 31714fe
Show file tree
Hide file tree
Showing 201 changed files with 53,260 additions and 1,819 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,7 @@
*__pycache__

sessions
.terraform/
.terraform.lock.hcl

#tools
25 changes: 25 additions & 0 deletions API/Backend/Config/setup.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,31 @@ let setup = {
});
}
);

s.app.get(
s.ROOT_PATH + "/configure-beta",
s.ensureGroup(s.permissions.users),
s.ensureAdmin(true),
(req, res) => {
const user = process.env.AUTH === "csso" ? req.user : null;
res.render("../configure/build/index.pug", {
user: user,
AUTH: process.env.AUTH,
NODE_ENV: process.env.NODE_ENV,
PORT: process.env.PORT || "8888",
ENABLE_CONFIG_WEBSOCKETS: process.env.ENABLE_CONFIG_WEBSOCKETS,
ENABLE_CONFIG_OVERRIDE: process.env.ENABLE_CONFIG_OVERRIDE,
ROOT_PATH:
process.env.NODE_ENV === "development"
? ""
: process.env.ROOT_PATH || "",
WEBSOCKET_ROOT_PATH:
process.env.NODE_ENV === "development"
? ""
: process.env.WEBSOCKET_ROOT_PATH || "",
});
}
);
}

s.app.use(
Expand Down
8 changes: 6 additions & 2 deletions API/Backend/Config/uuids.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,17 @@ const populateUUIDs = (config) => {

// Track of all of the previously defined UUIDs (i.e. ignore the UUIDs of the newly added layers)
Utils.traverseLayers(config.layers, (layer) => {
if (layer.uuid != null && !layer.proposed_uuid) {
if (
layer.uuid != null &&
typeof layer.uuid !== "number" &&
!layer.proposed_uuid
) {
definedUUIDs.push(layer.uuid);
}
});

Utils.traverseLayers(config.layers, (layer) => {
if (layer.uuid == null) {
if (layer.uuid == null || typeof layer.uuid === "number") {
layer.uuid = uuidv4();
newlyAddedUUIDs.push({
name: layer.name,
Expand Down
114 changes: 109 additions & 5 deletions API/Backend/Datasets/routes/datasets.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ const inspect = require("util").inspect;

const { sequelize } = require("../../../connection");

const Utils = require("../../../utils.js");
const logger = require("../../../logger");
const datasets = require("../models/datasets");
const csvtojson = require("csvtojson");
Expand Down Expand Up @@ -83,15 +84,68 @@ function get(req, res, next) {
router.post("/entries", function (req, res, next) {
Datasets.findAll()
.then((sets) => {
if (sets && sets.length > 0) {
if (sets) {
let entries = [];
for (let i = 0; i < sets.length; i++) {
entries.push({ name: sets[i].name, updated: sets[i].updatedAt });
}
res.send({
status: "success",
body: { entries: entries },
});
// For each entry, list all occurrences in latest configuration objects
sequelize
.query(
`
SELECT t1.*
FROM configs AS t1
INNER JOIN (
SELECT mission, MAX(version) AS max_version
FROM configs
GROUP BY mission
) AS t2
ON t1.mission = t2.mission AND t1.version = t2.max_version ORDER BY mission ASC;
`
)
.then(([results]) => {
// Populate occurrences
results.forEach((m) => {
Utils.traverseLayers(m.config.layers, (layer, path) => {
entries.forEach((entry) => {
entry.occurrences = entry.occurrences || {};
entry.occurrences[m.mission] =
entry.occurrences[m.mission] || [];
if (layer?.variables?.datasetLinks?.length != null) {
layer.variables.datasetLinks.forEach((d) => {
if (d.dataset === entry.name) {
entry.occurrences[m.mission].push({
name: layer.name,
uuid: layer.uuid,
path: path,
});
}
});
}
});
});
});

res.send({
status: "success",
body: { entries: entries },
});
return null;
})
.catch((err) => {
logger(
"error",
"Failed to find missions.",
req.originalUrl,
req,
err
);
res.send({
status: "failure",
message: "Failed to find missions.",
});
return null;
});
} else {
res.send({
status: "failure",
Expand Down Expand Up @@ -175,6 +229,56 @@ router.post("/search", function (req, res, next) {
});
});

/*
* req.query.layer
*/
router.get("/download", function (req, res, next) {
//First Find the table name
Datasets.findOne({ where: { name: req.query.layer } })
.then((result) => {
if (result) {
let table = result.dataValues.table;

sequelize
.query("SELECT * FROM " + table)
.then(([results]) => {
res.send({
status: "success",
body: results,
});

return null;
})
.catch((err) => {
logger(
"error",
"SQL error downloading dataset.",
req.originalUrl,
req,
err
);
res.send({
status: "failure",
message: "SQL error.",
});
});
} else {
res.send({
status: "failure",
message: "Layer not found.",
});
}

return null;
})
.catch((err) => {
logger("error", "Failure finding dataset.", req.originalUrl, req, err);
res.send({
status: "failure",
});
});
});

router.post("/upload", function (req, res, next) {
// Disable timeout
req.setTimeout(0);
Expand Down
Loading

0 comments on commit 31714fe

Please sign in to comment.