Build hdf-converters with Rollup for html-mapper #6004

kemley76 · 2024-07-17T15:28:49Z

As a way to simplify the usage of hdf-converter's hdf to html mapper, I have setup hdf-converters to be built using Rollup. This allows the template files (html, css, js) necessary for the html mapper to be bundled directly in with the hdf-converter's package. This removes the need for extra config for users of this mapper (no need to setup tailwind and move tailwind-elements files). Also, the main purpose of these changes is to allow for the mapper to work within the SAF CLI, which is now possible because it doesn't rely on Axios.

Signed-off-by: Kaden Emley <kemley@mitre.org>

…rters Signed-off-by: Kaden Emley <kemley@mitre.org>

mergify · 2024-07-25T20:35:46Z

This pull request has a conflict. Could you fix it @kemley76?

Signed-off-by: Kaden Emley <kemley@mitre.org>

mergify · 2024-07-26T00:40:57Z

This pull request has a conflict. Could you fix it @kemley76?

Signed-off-by: Kaden Emley <kemley@mitre.org>

Amndeep7 · 2024-07-29T19:38:16Z

apps/frontend/package.json

-    "prebuild": "rimraf ../../dist/frontend",
-    "build:tailwind": "tailwindcss -i ./src/tailwind.css -o ./public/static/export/style.css --minify",
-    "build": "yarn build:tailwind && vue-cli-service build",
+    "build": "nx run build",


please swap out all lerna commands for nx. this will also require updating the release docs. i would also make sure everything works with the docker builds.

Amndeep7 · 2024-07-29T19:38:41Z

apps/frontend/project.json

please put all the project information into the package.json file instead of being a standalone config file

Amndeep7 · 2024-07-29T19:40:01Z

package.json

@@ -8,7 +8,7 @@
  ],
  "scripts": {
    "backend": "yarn workspace heimdall-server",
-    "build": "lerna run build",
+    "build": "nx run-many --target=build --all",


if it's possible to swap out the lerna config file for nx as well, that'd be great

Amndeep7 · 2024-07-29T19:43:40Z

libs/hdf-converters/rollup.config.cjs

ideally this still works if you rename it as .js instead of .cjs

Amndeep7 · 2024-07-29T19:46:31Z

libs/hdf-converters/tsconfig.json

@@ -10,6 +10,7 @@
    "outDir": "./lib",
    "strict": true,
    "esModuleInterop": true,
-    "resolveJsonModule": true
+    "resolveJsonModule": true,


unnecessary since already defined in the top level tsconfig.json

Amndeep7 · 2024-07-29T19:51:09Z

libs/hdf-converters/rollup.config.cjs

+    input: 'index.ts',
+    output: {
+        file: 'lib/bundle.js',
+        format: 'cjs'


something to keep in mind for us moving wholesale to esm is any explicit format specifications like this.

Amndeep7 · 2024-07-29T19:52:35Z

libs/hdf-converters/src/converters-from-hdf/html/reverse-html-mapper.ts

@@ -24,6 +23,9 @@ import {
  IResultSeverity,
  IResultStatus
 } from './html-types';
+import tailwindStyles from './style.css';
+import htmlTemplate from './template.html';
+import twElementsScript from './tw-elements.umd.min.js.txt';


add a comment explaining why this needs to be .txt here or wherever is most appropriate

Comments have been added in several places.

Signed-off-by: Kaden Emley <kemley@mitre.org>

mergify · 2024-07-30T21:04:36Z

This pull request has a conflict. Could you fix it @kemley76?

sonarqubecloud · 2024-08-16T20:14:12Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

kemley76 · 2024-08-16T20:46:55Z

State as of 8/16/24

These changes are all to get HTML export working in the SAF CLI: PR

Features added

I've made all the functional changes needed:

setting up rollup for hdf-converters
setting up the whole monorepo to handle rollup
ensuring HTML mapper uses bundled template files
setting up tailwind and tw-elements to generate bundled files
removing said file generation from frontend
swapping out lerna commands for nx commands to better manage the monorepo

What is left to do

ensure that docker containers can still build and function properly after changes to build system
ensure that the removal of postcss does not pose any issues. I think this might just require testing on various browsers?
replace lerna release system with nx release system and update documentation accordingly. Currently lerna is only in this project as part of the release process, but replacing it with nx is not trivial for getting nx to do automatic changelog generation for the first time using it.

mergify · 2024-08-19T16:30:47Z

This pull request has a conflict. Could you fix it @kemley76?

kemley76 added 6 commits July 16, 2024 15:52

setup hdf-converters to use rollup for refactoring of html export

5bf602f

Signed-off-by: Kaden Emley <kemley@mitre.org>

add rollup typescript package

68b494d

Signed-off-by: Kaden Emley <kemley@mitre.org>

ensure hdf-converters is rebuilt before running frontend

72c1768

Signed-off-by: Kaden Emley <kemley@mitre.org>

point package json to actual generated types

19ea07f

Signed-off-by: Kaden Emley <kemley@mitre.org>

fix unknown file type errors in hdf-converters jest tests

3f5b943

Signed-off-by: Kaden Emley <kemley@mitre.org>

remove unused dev dependency

4e72734

Signed-off-by: Kaden Emley <kemley@mitre.org>

kemley76 mentioned this pull request Jul 17, 2024

Update hdf2html mapper #5910

Closed

kemley76 marked this pull request as draft July 17, 2024 15:39

kemley76 added 11 commits July 17, 2024 11:46

add accidently gitignored tailwind file

414cada

Signed-off-by: Kaden Emley <kemley@mitre.org>

move rollup dev deps into normal deps

fd42bab

Signed-off-by: Kaden Emley <kemley@mitre.org>

transform html export tempalte files for frontend tests

2855915

Signed-off-by: Kaden Emley <kemley@mitre.org>

build hdf-converters before frontend test

0acdb73

Signed-off-by: Kaden Emley <kemley@mitre.org>

move rollup dev deps to deps

a84cf1a

Signed-off-by: Kaden Emley <kemley@mitre.org>

build hdf-converters before building frontend

a961cf5

Signed-off-by: Kaden Emley <kemley@mitre.org>

include tsx as non-dev dependency

635e936

Signed-off-by: Kaden Emley <kemley@mitre.org>

modify dependencies and build order

ffa6c21

Signed-off-by: Kaden Emley <kemley@mitre.org>

build lerna with include dependencies

cdc68af

Signed-off-by: Kaden Emley <kemley@mitre.org>

build hdf-converters before frontend

8714213

Signed-off-by: Kaden Emley <kemley@mitre.org>

added implicit dependency relationship between frontend and hdf-conve…

779de05

…rters Signed-off-by: Kaden Emley <kemley@mitre.org>

kemley76 added 2 commits July 25, 2024 16:43

Merge branch 'master' into rollup-html-mapper

93ffcbf

setup nx for prebuild, build, and test for frontend

f07ee17

Signed-off-by: Kaden Emley <kemley@mitre.org>

kemley76 added 4 commits July 26, 2024 09:19

build tailwind before hdf-converters tests

c86852c

Signed-off-by: Kaden Emley <kemley@mitre.org>

Merge branch 'master' into rollup-html-mapper

7274fd5

build hdf-converters before frontend tests

410c5fa

Signed-off-by: Kaden Emley <kemley@mitre.org>

move tsx back to dev dependencies

c68f7df

Signed-off-by: Kaden Emley <kemley@mitre.org>

kemley76 mentioned this pull request Jul 26, 2024

HTML Export Support mitre/saf#1689

Open

kemley76 marked this pull request as ready for review July 26, 2024 16:32

Merge branch 'master' into rollup-html-mapper

73e2e8d

Amndeep7 requested changes Jul 29, 2024

View reviewed changes

kemley76 and others added 7 commits July 29, 2024 16:25

remove redeclaration of resolveJsonModule: true

2037b69

Signed-off-by: Kaden Emley <kemley@mitre.org>

rename rollup.config.cjs to rollup.config.js

9658559

Signed-off-by: Kaden Emley <kemley@mitre.org>

add comments for what the html template files are

cdabb7d

Signed-off-by: Kaden Emley <kemley@mitre.org>

migrate from lerna to nx completely

826dd13

Signed-off-by: Kaden Emley <kemley@mitre.org>

ensure each package minus the frontend doesnt need nx

8796c90

Signed-off-by: Kaden Emley <kemley@mitre.org>

install nx before trying to use nx in frontend test action

70d4ba8

Signed-off-by: Kaden Emley <kemley@mitre.org>

Merge branch 'master' into rollup-html-mapper

22b9627

Merge branch 'master' into rollup-html-mapper

90eff2c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build hdf-converters with Rollup for html-mapper #6004

Build hdf-converters with Rollup for html-mapper #6004

kemley76 commented Jul 17, 2024

mergify bot commented Jul 25, 2024

mergify bot commented Jul 26, 2024

Amndeep7 Jul 29, 2024

Amndeep7 Jul 29, 2024

kemley76 Jul 30, 2024

Amndeep7 Jul 29, 2024

kemley76 Jul 30, 2024

Amndeep7 Jul 29, 2024

kemley76 Jul 29, 2024

Amndeep7 Jul 29, 2024

kemley76 Jul 29, 2024

Amndeep7 Jul 29, 2024

Amndeep7 Jul 29, 2024

kemley76 Jul 29, 2024

mergify bot commented Jul 30, 2024

sonarqubecloud bot commented Aug 16, 2024

kemley76 commented Aug 16, 2024 •

edited

Loading

mergify bot commented Aug 19, 2024

Build hdf-converters with Rollup for html-mapper #6004

Are you sure you want to change the base?

Build hdf-converters with Rollup for html-mapper #6004

Conversation

kemley76 commented Jul 17, 2024

mergify bot commented Jul 25, 2024

mergify bot commented Jul 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mergify bot commented Jul 30, 2024

sonarqubecloud bot commented Aug 16, 2024

Quality Gate passed

kemley76 commented Aug 16, 2024 • edited Loading

State as of 8/16/24

Features added

What is left to do

mergify bot commented Aug 19, 2024

kemley76 commented Aug 16, 2024 •

edited

Loading