#464 /configure-beta (#574)

* #464 Early COnfigure Remake * #464 Add styles to new configure * #464 New Configure 1 * #464 Rework Configure 2 * #464 New mission modals, general modals and snackbar * #464 Integrate new config page into MMGIS server, start of layers tab * #464 Start metaconfigs - home and ui * #464 Start Layers * #464 Map, Tile layer modal, settable config * #464 Change layer params, vector-conf, colorpickers, start tools tab * #464 Tools Tab * #464 Tool Modal partial * #464 Partial updates * #464 More Tool configs * #464 More work * #464 Vector layer components * #464 Preview Modal * #464 Layer Organization * #464 Pages and API Keys * #464 Early Geodatasets * Merge dev into new config feature branch (#541) * Fix click on vectortile * Fix missing Missions/mission path on tile layers * #465 Disable Globe more thoroughly when off (#466) * #467 GENERATE_SOURCEMAP ENV (#468) * #469 Fix Tool Drag Handle Remains on Screen (#470) * #471 Additional Body Metadata for Draw Webhooks (#472) * #471 Webhook fields 1 * #471 Additional Body Metadata for Draw Webhooks * #471 Add raw_file_description * #474 ShadeTool - Fix Entity height calculation (#475) * #474 ShadeTool- Fix Entity height calculation * #474 Remove comment of bad calculation line * #473 MeasureTool and IdentifierTool Layer Vars (#476) * #473 Layer Specific IdentifierTool vars 1 * #473 MeasureTool and IdentifierTool Layer Vars * #477 DrawTool - Filter state management (#478) * #479 Remove restriction on Layer names (#480) * #479 Remove Layer Name restrictions * #479 Remove console.log * #481 ShadeTool Touchups (#482) * #481 ShadeTool Touchups * #481 Fix docs * #483 MeasureTool Fix nodata issue (#484) * Minor ShadeTool Fixes - hide point and regen at med res * Check for empty time configs in TimeControl * #489 DrawTool - Move (#490) * #489 Draggable DrawTool Features part 1 * #489 DrawTool - Move 2 * #494 Dropdown in the topbar for a selected feature’s properties links (#495) * #485 Rover Image Overlay Improvements (#486) * #485 Rover Image Overlay Improvments * #485 Prevent reload/updates to layers through the mmgisAPI if one is already taking place * #485 Support initialOpacity for imageOverlays w/ docs * Remove dev consolelog * #485 Rover Improvement fixes * #485 Image overlay chache works (but is offset badly now) * #485 Fix image_overlay updateVectorLayer flickering * #485 Fix layerObj ref * #496 Support Symbols in DrawTool Folders and Tags (#497) * #491 panorama angle of view wonky near poles (#499) * #492 updateVectorLayer calls causes flat panorama view to revert to wrapped panorama view (#501) * Fix time replacement in IdentifierTool * #500 getFiles if deeplinked to draw tool (#502) * Update docker installation docs * Fix typo in docs * mmgisAPI.appendLineString returns false if layer was in a blocking state * Touch up, move extraneous DrawTool destory * If Globe Panel disabled, do not even run empty animate loop * Bug: Deeplinked DrawTool layers are now immediately clickable * Bug: Webhooks identify symbols in draw tags * Minor bug fixes - missing deeplink layer error gone, search bar recenters and can force makelayer * #505 Dynamic Vector Layers (#506) * #505 Dynamic Vector Layer querying 1 * #505 Time queries, reselect improvement, download extent or full * #505 Dynamic vector layers * #504 ShadeTool Improvements and Multi-missionize (#507) * #504 ShadeTool - User settable AzElRange * #504 SPICE kernel scheduled downloads * #504 update shadetool css * #504 Update docs for SPICE conf * #505 [Bug]: Support dynamicExtentMoveThresholf for non-geodatasets * #508 Upgrade Dockerfile to Node.JS v20+ (#509) * #508 Upgrade to NodeJS v20.11.1 * #508 Update Dockerfile 2, update nupmy * Geodatasets to use spatial index, also DB_POOL envs * #510 Temporal Geodatasets (#511) * #510 Temporal Geodatasets * #510 Fix new doc page path * #270 Fix nested header expansion (#512) * #498 LayersTool - Prevent toggle of loading layer (#513) * Bump 2.11 (#515) * Docs - Time Tiles - Clarify tagging * Fix css misaligned panel arrows due to browser update * Bugfix: Send options to WMS layers (#517) * Bugfix: Send options to WMS layers * remove console.log * Bugfix: TimeUI zoom - use wheel instead of mousewheel event for ff * Bugfix: Catch errors in api/shorten/expand * #519 fix mmgisAPI.setTime start (#520) * #518 time={starttime} in a WMS URL is filled with endtime (#522) * ts-518 Inject time tile params better * #518 Stop wms layer options for override wms parameters * Bugfix: LayersTool - Fix collapse all headers * #523 Chronice fix second loss (#524) * Info.go * #270 Fix nested header expansion - quickfix * Fix Chronice overflow, update spice-kernel-conf-example and fix TimeUI startInPointMode * #525 Make dynamicExtent requery timeEnabled aware. (#526) * #525 Make dynamicExtent requery timeEnabled aware. * #525 Make dynamicExtent requery controlled aware. * #527 Bugfix: Vector layers with headings/bearings do not toggle on/off based on zoom cutoffs (#528) * #529 Viewshed set initial target and observer heights, also doc improvements (#530) * #531 Viewshed in-app docs (#532) * #531 partial * #531 Viewshed in-app docs * gdal2customtiles.py remove downscale for dem tiles * #533 Allow adding relative seconds to initial times (#534) * #533 Allow adding relative seconds to initial times * #533 Make sure not to overwrite config time value for initialend * #535 dynamicExtent vector layer and filtering issues (#536) * Spelling fixes * #452 Updated Identifier tool to be a separated floating tool - Need to test different UI combinations with other separated tools - We may possibly want an option to make the tool not be separated to maintain compatibility * Fixed Identifier tool bug when using old config with non-separated tool Bug was causing other tools to not open. This makes it backwards compatible. * JS-API - setTime: add customtimes * Fix docs typo * Allow Identifier and Legend tools to be on different sides if specified * Bug fix - MeasureTool without dem config variable set * DrawTool - include Created and Modified time part in file info * Fix no data dynamicExtent responses * Remove minor debug field --------- Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov> Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com> * #464 More Geodataset page * #464 More Geodataset work * #464 Geodatasets page * #464 More new configuration site work * #464 More new config fixes * #464 Serve both configs, saving, websockets, early datasets * #464 Remove extra image * #464 Home page among many other things * #464 Configure-beta - Finish webhooks and dataset * #464 Touch ups * Update Feature Branch #464 (#550) * Fix click on vectortile * Fix missing Missions/mission path on tile layers * #465 Disable Globe more thoroughly when off (#466) * #467 GENERATE_SOURCEMAP ENV (#468) * #469 Fix Tool Drag Handle Remains on Screen (#470) * #471 Additional Body Metadata for Draw Webhooks (#472) * #471 Webhook fields 1 * #471 Additional Body Metadata for Draw Webhooks * #471 Add raw_file_description * #474 ShadeTool - Fix Entity height calculation (#475) * #474 ShadeTool- Fix Entity height calculation * #474 Remove comment of bad calculation line * #473 MeasureTool and IdentifierTool Layer Vars (#476) * #473 Layer Specific IdentifierTool vars 1 * #473 MeasureTool and IdentifierTool Layer Vars * #477 DrawTool - Filter state management (#478) * #479 Remove restriction on Layer names (#480) * #479 Remove Layer Name restrictions * #479 Remove console.log * #481 ShadeTool Touchups (#482) * #481 ShadeTool Touchups * #481 Fix docs * #483 MeasureTool Fix nodata issue (#484) * Minor ShadeTool Fixes - hide point and regen at med res * Check for empty time configs in TimeControl * #489 DrawTool - Move (#490) * #489 Draggable DrawTool Features part 1 * #489 DrawTool - Move 2 * #494 Dropdown in the topbar for a selected feature’s properties links (#495) * #485 Rover Image Overlay Improvements (#486) * #485 Rover Image Overlay Improvments * #485 Prevent reload/updates to layers through the mmgisAPI if one is already taking place * #485 Support initialOpacity for imageOverlays w/ docs * Remove dev consolelog * #485 Rover Improvement fixes * #485 Image overlay chache works (but is offset badly now) * #485 Fix image_overlay updateVectorLayer flickering * #485 Fix layerObj ref * #496 Support Symbols in DrawTool Folders and Tags (#497) * #491 panorama angle of view wonky near poles (#499) * #492 updateVectorLayer calls causes flat panorama view to revert to wrapped panorama view (#501) * Fix time replacement in IdentifierTool * #500 getFiles if deeplinked to draw tool (#502) * Update docker installation docs * Fix typo in docs * mmgisAPI.appendLineString returns false if layer was in a blocking state * Touch up, move extraneous DrawTool destory * If Globe Panel disabled, do not even run empty animate loop * Bug: Deeplinked DrawTool layers are now immediately clickable * Bug: Webhooks identify symbols in draw tags * Minor bug fixes - missing deeplink layer error gone, search bar recenters and can force makelayer * #505 Dynamic Vector Layers (#506) * #505 Dynamic Vector Layer querying 1 * #505 Time queries, reselect improvement, download extent or full * #505 Dynamic vector layers * #504 ShadeTool Improvements and Multi-missionize (#507) * #504 ShadeTool - User settable AzElRange * #504 SPICE kernel scheduled downloads * #504 update shadetool css * #504 Update docs for SPICE conf * #505 [Bug]: Support dynamicExtentMoveThresholf for non-geodatasets * #508 Upgrade Dockerfile to Node.JS v20+ (#509) * #508 Upgrade to NodeJS v20.11.1 * #508 Update Dockerfile 2, update nupmy * Geodatasets to use spatial index, also DB_POOL envs * #510 Temporal Geodatasets (#511) * #510 Temporal Geodatasets * #510 Fix new doc page path * #270 Fix nested header expansion (#512) * #498 LayersTool - Prevent toggle of loading layer (#513) * Bump 2.11 (#515) * Docs - Time Tiles - Clarify tagging * Fix css misaligned panel arrows due to browser update * Bugfix: Send options to WMS layers (#517) * Bugfix: Send options to WMS layers * remove console.log * Bugfix: TimeUI zoom - use wheel instead of mousewheel event for ff * Bugfix: Catch errors in api/shorten/expand * #519 fix mmgisAPI.setTime start (#520) * #518 time={starttime} in a WMS URL is filled with endtime (#522) * ts-518 Inject time tile params better * #518 Stop wms layer options for override wms parameters * Bugfix: LayersTool - Fix collapse all headers * #523 Chronice fix second loss (#524) * Info.go * #270 Fix nested header expansion - quickfix * Fix Chronice overflow, update spice-kernel-conf-example and fix TimeUI startInPointMode * #525 Make dynamicExtent requery timeEnabled aware. (#526) * #525 Make dynamicExtent requery timeEnabled aware. * #525 Make dynamicExtent requery controlled aware. * #527 Bugfix: Vector layers with headings/bearings do not toggle on/off based on zoom cutoffs (#528) * #529 Viewshed set initial target and observer heights, also doc improvements (#530) * #531 Viewshed in-app docs (#532) * #531 partial * #531 Viewshed in-app docs * gdal2customtiles.py remove downscale for dem tiles * #533 Allow adding relative seconds to initial times (#534) * #533 Allow adding relative seconds to initial times * #533 Make sure not to overwrite config time value for initialend * #535 dynamicExtent vector layer and filtering issues (#536) * Spelling fixes * #452 Updated Identifier tool to be a separated floating tool - Need to test different UI combinations with other separated tools - We may possibly want an option to make the tool not be separated to maintain compatibility * Fixed Identifier tool bug when using old config with non-separated tool Bug was causing other tools to not open. This makes it backwards compatible. * JS-API - setTime: add customtimes * Fix docs typo * Allow Identifier and Legend tools to be on different sides if specified * Bug fix - MeasureTool without dem config variable set * DrawTool - include Created and Modified time part in file info * Fix no data dynamicExtent responses * Remove minor debug field * #544 Fix setTime currentTime timezone, default layer time objects (#548) --------- Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov> Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com> * #464 Final touches * #464 Fix missing / * #464 Update installation docs * #464 Add downloadURL config var * #464 Add documentation for configure dev and explain a idTool field better * #464 Add IdTool scalefactor param to config-beta * #464 Add useNameAsKey to configure-beta * #464 Final touches, bump version --------- Co-authored-by: Joe Roberts <joe.t.roberts@jpl.nasa.gov> Co-authored-by: Joe T. Roberts <5315956+jtroberts@users.noreply.github.com>
NASA-AMMOS · Jul 8, 2024 · 2984c1d · 2984c1d
1 parent 95742cc
commit 2984c1d
Show file tree

Hide file tree

Showing 117 changed files with 46,140 additions and 155 deletions.
diff --git a/API/Backend/Config/setup.js b/API/Backend/Config/setup.js
@@ -32,6 +32,30 @@ let setup = {
           });
         }
       );
+      s.app.get(
+        s.ROOT_PATH + "/configure-beta",
+        s.ensureGroup(s.permissions.users),
+        s.ensureAdmin(true),
+        (req, res) => {
+          const user = process.env.AUTH === "csso" ? req.user : null;
+          res.render("../configure/build/index.pug", {
+            user: user,
+            AUTH: process.env.AUTH,
+            NODE_ENV: process.env.NODE_ENV,
+            PORT: process.env.PORT || "8888",
+            ENABLE_CONFIG_WEBSOCKETS: process.env.ENABLE_CONFIG_WEBSOCKETS,
+            ENABLE_CONFIG_OVERRIDE: process.env.ENABLE_CONFIG_OVERRIDE,
+            ROOT_PATH:
+              process.env.NODE_ENV === "development"
+                ? ""
+                : process.env.ROOT_PATH || "",
+            WEBSOCKET_ROOT_PATH:
+              process.env.NODE_ENV === "development"
+                ? ""
+                : process.env.WEBSOCKET_ROOT_PATH || "",
+          });
+        }
+      );
     }
 
     s.app.use(

diff --git a/API/Backend/Config/uuids.js b/API/Backend/Config/uuids.js
@@ -9,13 +9,17 @@ const populateUUIDs = (config) => {
 
   // Track of all of the previously defined UUIDs (i.e. ignore the UUIDs of the newly added layers)
   Utils.traverseLayers(config.layers, (layer) => {
-    if (layer.uuid != null && !layer.proposed_uuid) {
+    if (
+      layer.uuid != null &&
+      typeof layer.uuid !== "number" &&
+      !layer.proposed_uuid
+    ) {
       definedUUIDs.push(layer.uuid);
     }
   });
 
   Utils.traverseLayers(config.layers, (layer) => {
-    if (layer.uuid == null) {
+    if (layer.uuid == null || typeof layer.uuid === "number") {
       layer.uuid = uuidv4();
       newlyAddedUUIDs.push({
         name: layer.name,

diff --git a/API/Backend/Datasets/routes/datasets.js b/API/Backend/Datasets/routes/datasets.js
@@ -10,6 +10,7 @@ const inspect = require("util").inspect;
 
 const { sequelize } = require("../../../connection");
 
+const Utils = require("../../../utils.js");
 const logger = require("../../../logger");
 const datasets = require("../models/datasets");
 const csvtojson = require("csvtojson");
@@ -83,15 +84,68 @@ function get(req, res, next) {
 router.post("/entries", function (req, res, next) {
   Datasets.findAll()
     .then((sets) => {
-      if (sets && sets.length > 0) {
+      if (sets) {
         let entries = [];
         for (let i = 0; i < sets.length; i++) {
           entries.push({ name: sets[i].name, updated: sets[i].updatedAt });
         }
-        res.send({
-          status: "success",
-          body: { entries: entries },
-        });
+        // For each entry, list all occurrences in latest configuration objects
+        sequelize
+          .query(
+            `
+            SELECT t1.*
+            FROM configs AS t1
+            INNER JOIN (
+                SELECT mission, MAX(version) AS max_version
+                FROM configs
+                GROUP BY mission
+            ) AS t2
+            ON t1.mission = t2.mission AND t1.version = t2.max_version ORDER BY mission ASC;
+            `
+          )
+          .then(([results]) => {
+            // Populate occurrences
+            results.forEach((m) => {
+              Utils.traverseLayers(m.config.layers, (layer, path) => {
+                entries.forEach((entry) => {
+                  entry.occurrences = entry.occurrences || {};
+                  entry.occurrences[m.mission] =
+                    entry.occurrences[m.mission] || [];
+                  if (layer?.variables?.datasetLinks?.length != null) {
+                    layer.variables.datasetLinks.forEach((d) => {
+                      if (d.dataset === entry.name) {
+                        entry.occurrences[m.mission].push({
+                          name: layer.name,
+                          uuid: layer.uuid,
+                          path: path,
+                        });
+                      }
+                    });
+                  }
+                });
+              });
+            });
+
+            res.send({
+              status: "success",
+              body: { entries: entries },
+            });
+            return null;
+          })
+          .catch((err) => {
+            logger(
+              "error",
+              "Failed to find missions.",
+              req.originalUrl,
+              req,
+              err
+            );
+            res.send({
+              status: "failure",
+              message: "Failed to find missions.",
+            });
+            return null;
+          });
       } else {
         res.send({
           status: "failure",
@@ -175,6 +229,56 @@ router.post("/search", function (req, res, next) {
     });
 });
 
+/*
+ * req.query.layer
+ */
+router.get("/download", function (req, res, next) {
+  //First Find the table name
+  Datasets.findOne({ where: { name: req.query.layer } })
+    .then((result) => {
+      if (result) {
+        let table = result.dataValues.table;
+
+        sequelize
+          .query("SELECT * FROM " + table)
+          .then(([results]) => {
+            res.send({
+              status: "success",
+              body: results,
+            });
+
+            return null;
+          })
+          .catch((err) => {
+            logger(
+              "error",
+              "SQL error downloading dataset.",
+              req.originalUrl,
+              req,
+              err
+            );
+            res.send({
+              status: "failure",
+              message: "SQL error.",
+            });
+          });
+      } else {
+        res.send({
+          status: "failure",
+          message: "Layer not found.",
+        });
+      }
+
+      return null;
+    })
+    .catch((err) => {
+      logger("error", "Failure finding dataset.", req.originalUrl, req, err);
+      res.send({
+        status: "failure",
+      });
+    });
+});
+
 router.post("/upload", function (req, res, next) {
   // Disable timeout
   req.setTimeout(0);