Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CoMPAS-Open-SCD should adhere to the new monorepo structure #266

Open
1 of 3 tasks
pascalwilbrink opened this issue Sep 13, 2023 · 6 comments
Open
1 of 3 tasks

CoMPAS-Open-SCD should adhere to the new monorepo structure #266

pascalwilbrink opened this issue Sep 13, 2023 · 6 comments
Labels

Comments

@pascalwilbrink
Copy link
Member

pascalwilbrink commented Sep 13, 2023

Intro

The Open-SCD project recently went from a monolith to a mono-repo architecture. In order to accomplish this a restructure was needed. This has a severe effect on the Compas-Open-SCD repository. The Compas-Open-SCD repository is a fork of the Open-SCD project and has added functionalities to it. If we want to apply the newest changes of the Open-SCD project, we first have to do some manual labor in order to prevent merge conflicts.

Changes made to the fork

As said above, the Compas-Open-SCD repository is a fork, but with alterations. Before we know how to resolve the coming merge conflicts. We need to make sure we know which files are added and which ones are altered, so we can take those into account. These files are noted in the lists below.

New files added to the project

  • src/file.ts
  • src/compas/CompasAutoAlignment.ts
  • src/compas/CompasChangeSetRadiogroup.ts
  • src/compas/CompasComment.ts
  • src/compas/CompasExistsIn.ts
  • src/compas/CompasImportFromApi.ts
  • src/compas/Compasing.ts
  • src/compas/CompasLabelsField.ts
  • src/compas/CompasLoading.ts
  • src/compas/CompasNsdoc.ts
  • src/compas/CompasOpen.ts
  • src/compas/CompasSave.ts
  • src/compas/CompasSclList.ts
  • src/compas/CompasSclTypeList.ts
  • src/compas/CompasSclTypeSelect.ts
  • src/compas/CompasSessionExpiredDialog.ts
  • src/compas/CompasSessionExpiringDialog.ts
  • src/compas/CompasSettings.ts
  • src/compas/CompasUploadVersion.ts
  • src/compas/CompasUserInfo.ts
  • src/compas/foundation.ts
  • src/compas/private.ts
  • src/compas-editors/autogen-substation.ts
  • src/compas-editors/CompasVersions.ts
  • src/compas-editors/foundation.ts
  • src/compas-editors/Sitipe.ts
  • src/compas-editors/sitipe/foundation.ts
  • src/compas-editors/sitipe/sitipe-bay.ts
  • src/compas-editors/sitipe/sitipe-service.ts
  • src/compas-editors/sitipe/sitipe-substation.ts
  • src/compas-services/CompasCimMappingService.ts
  • src/compas-services/CompasNSDocFileService.ts
  • src/compas-services/CompasSclAutoAlignmentService.ts
  • src/compas-services/CompasSclDataService.ts
  • src/compas-services/CompasUserInfoService.ts
  • src/compas-services/CompasValidatorService.ts
  • src/compas-services/foundation.ts
  • src/compas-services/Websocket.ts
  • src/compas-wizards/scl.ts
  • src/locamation/foundation.ts
  • src/locamation/LocamationIEDList.ts
  • src/locamation/LocamationLNEdit.ts
  • src/locamation/LocamationLNList.ts
  • src/menu/CompasAutoAlignment.ts
  • src/menu/CompasCimMapping.ts
  • src/menu/CompasCompareIED.ts
  • src/menu/CompasImportFromApi.ts
  • src/menu/CompasImportIEDs.ts
  • src/menu/CompasMerge.ts
  • src/menu/CompasOpen.ts
  • src/menu/CompasSave.ts
  • src/menu/CompasSaveAs.ts
  • src/menu/CompasSaveAsVersion.ts
  • src/menu/CompasSettings.ts
  • src/menu/CompasUpdateSubstation.ts
  • src/menu/ExportIEDParams.ts
  • src/menu/LocamationVMU.ts
  • src/validators/CompasValidateSchema.ts

Files from Open-SCD that are altered

  • src/Editing.ts
  • src/foundation.ts
  • src/Hosting.ts
  • src/Logging.ts
  • src/open-scd.ts
  • src/Plugging.ts
  • src/Setting.ts
  • src/wizard-dialog.ts [Component already moved out, could be extended at some point]
  • src/wizard-textfield.ts [Component already moved out, could be extended at some point]
  • src/editors/IED.ts
  • src/editors/communication/subnetwork-editor.ts
  • src/editors/protocol104/wizards/selectDo.ts
  • src/editors/publisher/data-set-editor.ts
  • src/editors/publisher/foundation.ts
  • src/editors/publisher/gse-control-editor.ts
  • src/editors/publisher/report-control-editor.ts
  • src/editors/publisher/sampled-value-control-editor.ts
  • src/editors/substation/ied-editor.ts
  • src/editors/templates/datype-wizards.ts
  • src/editors/templates/dotype-wizards.ts
  • src/editors/templates/enumtype-wizard.ts
  • src/editors/templates/lnodetype-wizard.ts
  • src/foundation/ied.ts
  • src/menu/CompareIED.ts
  • src/menu/ImportIEDs.ts
  • src/menu/Merge.ts
  • src/menu/OpenProject.ts
  • src/menu/UpdateDescriptionABB.ts
  • src/menu/UpdateDescriptionSEL.ts
  • src/menu/UpdateSubstation.ts
  • src/menu/VirtualTemplateIED.ts [Plugin already moved out, could be extended at some point]
  • src/translations/de.ts [How the translation will be handled is still an open issue]
  • src/translations/en.ts [How the translation will be handled is still an open issue]
  • src/wizards/clientln.ts
  • src/wizards/commmap-wizards.ts
  • src/wizards/connectedap.ts
  • src/wizards/dataset.ts
  • src/wizards/fcda.ts
  • src/wizards/gsecontrol.ts
  • src/wizards/lnode.ts
  • src/wizards/reportcontrol.ts
  • src/wizards/sampledvaluecontrol.ts
  • src/wizards/subnetwork.ts
  • src/wizards/foundation/dai-field-type.ts
  • src/wizards/foundation/enums.ts
  • src/wizards/foundation/finder.ts
  • src/wizards/foundation/limits.ts

Dependencies from Open-SCD

  • newLogEvent, newOpenDocEvent, newWizardEvent, LitElementConstructor, Mixin, newPendingStateEvent, Wizard, WizardInputElement, newUserInfoEvent, createElement, newActionEvent, isPublic, ComplexAction, SimpleAction, cloneElement, , EditorAction, getValue, WizardActor, getDescriptionAttribute, getInstanceAttribute, getNameAttribute, newSubWizardEvent, checkValidity, WizardAction, wizardInputSelector, compareNames, newIssueEvent from src/foundation.js
  • Nsdoc from src/foundation/nsdoc.js
  • SCL_NAMESPACE, isSCLNamespace from src/schemas.js
  • WizardTextField from src/wizard-textfield.js
  • EditingElement from src/Editing.js
  • mergeWizard from src/wizards.js
  • src/oscd-filter-button.js
  • SelectedItemsChangedEvent from src/oscd-filter-button.js
  • patterns from src/wizards/foundation/limits.js
  • src/WizardDivider.js
  • newLoadNsdocEvent from src/Setting.js
  • wizards from src/wizards/wizard-library.js
  • src/plain-compare-list.js
  • src/wizard-textfield.js [Will become oscd-textfield]
  • src/filtered-list [Will become oscd-filtered-list]
  • src/action-pane [Will become oscd-action-pane]
  • src/action-icon [Will become oscd-action-icon]

Suggestions to approach this issue

Step 1: Apply the restructure

Now that the file structure is clear a proof of concept can be made to see if the merge issues can be resolved. There are 2 options here to start.

  1. Apply the restructure manually to the entire Compas-Open-SCD project. This would mean to bump every file up 2 folders (packages/open-scd). This is probably the fastest way to implement the new structure. However the idea is to move out the compas related software to a different package. This can be more challenging with this approach
  2. Create a new fork of Open-SCD and start moving the software to the new package and see what we need from the Open-SCD package. This will probably take a bit more time, but it does leave out the manual labor that needs to be done first in order to get the new Open-SCD version.

Both approaches have merit and can be done. However approach #1 will keep the git history of Compas-Open-SCD in tact. If that's desirable the first approach would be best.

Step 2: Move out compas related software

The idea is that the open-scd package should be completely identical to the open-scd package we fork from. This will mean that The entire list from the new files should be moved to a packages/compas folder. Be sure that this folder has a dependency to the packages/open-scd folder.

Step 3: Move out tests for the Compas software

When the source code is succesfully moved out and the dependencies work correctly the tests can be moved out and run. Be sure all tests are succesful.

Step 4: Make sure the docker container is correctly build

The docker container should eventually be build from the compas package in order to make sure everything is included.

Open questions

  • How to create a dependency to a different package?
  • How to create the new default plugins that needs to be loaded in Compas-Open-SCD? (extend and override?)

Before starting

  • Update Compas-Open-SCD with the commit before the monorepo.
  • Recheck the file for differences
  • Check to see if changes between files can be a modified copy or can be extended into package/compas
@pascalwilbrink
Copy link
Member Author

We can split this story up into multiple substories

  1. Move everything from CoMPAS-OpenSCD up 1 folder (like has been done for OpenSCD)

  2. Extract all the CoMPAS code into /packages/compas (So that /packages/open-scd will be exactly the same as @openscd/open-scd/main:packages/open-scd

  3. Make packages/openscd be a dependency of packages/compas

  4. Make sure that the docker container is built from packages/compas

@pascalwilbrink
Copy link
Member Author

Changed files
ValidateTemplates.test.ts
ImportIedsPlugin.test.ts
IED.test.ts
datasets-container.test.snap.js
IED.test.snap.js
Setting.test.snap.js
open-scd.test.snap.js
wizard-textfield.ts
Wizard-dialog.ts
En.ts
de.ts
Setting.ts
Plugging.ts
open-scd.ts
UpdateSubstation.ts
OpenProject.ts
Merge.ts
ImportIEDs.ts
Logging.ts
Hosting.ts
foundation.ts
IED.ts
Editing.ts

@pascalwilbrink
Copy link
Member Author

Questions:

Should openSCD be able to import ISD files?
if so: Update OpenProject.ts and Merge.ts and ImportIEDs.ts in OpenSCD to allow ISD files.
if not: Should ComPas be able to?
if so: make fileExtensions configurable in OpenSCD (bigger refactoring)
if not: Remove ISD file from CoMPAS-OpenSCD/OpenProject.ts and Merge.ts and ImportIEDs.ts

@pascalwilbrink
Copy link
Member Author

pascalwilbrink commented Oct 19, 2023

Things to do:

@pascalwilbrink
Copy link
Member Author

pascalwilbrink commented Oct 25, 2023

How to migrate:

Steps that we need to take

  1. Move (almost) everything from (CoMPAS-Open-SCD)[https://github.com/com-pas/compas-open-scd] up 2 folders: packages/compas-open-scd.

Move the following:

  • public
  • src
  • test
  • .editorconfig
  • .eslintrc.cjs
  • .gitignore
  • .prettierignore
  • CHANGELOG.md
  • Dockerfile
  • ModHeader.png
  • README.md
  • RELEASING.md
  • browserconfig.xml
  • favicon.ico
  • index.html
  • manifest.json
  • package-lock.json
  • package.json
  • snowpack.config.mjs
  • sw.js
  • tsconfig.json
  • web-test-runner.config.mjs
  • workbox-config.cjs
  1. Update .github folder

A) Update .github/dependabot.yml
The package-ecosystem docker should have directory packages/compas-open-scd/, instead of /

B) Update .github/build-project.yml
Update the steps Test application with npm, Build application with npm, Build docker image: cd packages/compas-open-scd && ..

C) Update .github/release-project.yml
Update the steps Build application with npm, Build and push docker image to Docker Hub: cd packages/compas-open-scd && ..

D) Update .github/test-and-build.yml
Update the step Install and Build: cd packages/compas-open-scd && ..

E) Update .github/test.yml
Update the step Install and Test: cd packages/compas-open-scd && ..

  1. Create new Files in the repository root

A) Create a new README.md explaining about the monorepository and point to packages/compas-open-scd/README.md (See (OpenSCD README)[https://github.com/openscd/open-scd/blob/main/README.md] as an example)

B) Create a new .gitignore

C) Create a new package.json

{
  "name": "compas-openscd-monorepo",
  "version": "0.0.1",
  "description": "CoMPAS OpenSCD base distribution and plugins",
  "private": true,
  "workspaces": [
    "packages/*"
  ],
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/com-pas/compas-open-scd.git"
  },
  "license": "Apache-2.0"
}
  1. Sync com-pas/CoMPAS-Open-SCD with main from openscd/open-scd.

  2. Make sure that CoMPAS-Open-SCD and OpenSCD are build correctly. For CoMPAS-Open-SCD, we can easily check if it's build correctly (npm start).
    Since CoMPAS-Open-SCD will make use of plugins that are maintained by OpenSCD, we need to either Copy those (built) plugins over, or adjust the build process of CoMPAS-Open-SCD.

Copy plugins over:
5A) Create a build script in the root package.json
'build': 'cd packages/open-scd && npm run build && cd packages/compas-openscd && npm run build'
'postbuild': 'cp packages/open-scd/build'

Adjust build process:
5B) Update the snowpack.config.mjs inside packages/compas-openscd.
Add the
packageOptions.knownEntrypoints inside snowpack configuration to add @openscd/open-scd as a known dependency.

Step 5B will be easiest and cleanest, just some playing around with Snowpack.

  1. Update plugin.json in packages/compas-open-scd with OpenSCD plugins that have correct pointers (depending on step 5. See build folder)

  2. Remove identical files between packages/compas-open-scd and packages/open-scd.

Copy link

Hello there,

Thank you for opening this issue! We appreciate your interest in our project.
However, it seems that this issue hasn't had any activity for a while. To ensure that our issue tracker remains organized and efficient, we occasionally review and address stale issues.

If you believe this issue is still relevant and requires attention, please provide any additional context, updates, or details that might help us understand the problem better.
Feel free to continue the conversation here.

If the issue is no longer relevant, you can simply close it. If you're uncertain, you can always reopen it later.

Remember, our project thrives on community contributions, and your input matters. We're here to collaborate and improve.
Thank you for being part of this journey!

@github-actions github-actions bot added the stale label Dec 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Development

No branches or pull requests

1 participant