Merge dev into main (Sprint 5 and 6) (#24)

* Implement test orchestrator (#4)

* Initial work on test-orchestrator

* Ignore runtime folder

* Update runtime directory for test modules

* Fix logging
Add initial framework for running tests

* logging and misc cleanup

* logging changes

* Add a stop hook after all tests complete

* Refactor test_orc code

* Add arg passing
Add option to use locally cloned via install or remote via main project network orchestrator

* Fix baseline module
Fix orchestrator exiting only after timeout

* Add result file to baseline test module
Change result format to match closer to design doc

* Refactor pylint

* Skip test module if it failed to start

* Refactor

* Check for valid log level

---------

Co-authored-by: Jacob Boddey <boddey@google.com>

* Add issue report templates (#7)

* Add issue templates

* Update README.md

* Discover devices on the network (#5)

* Test run sync (#8)

* Initial work on test-orchestrator

* Ignore runtime folder

* Update runtime directory for test modules

* Fix logging
Add initial framework for running tests

* logging and misc cleanup

* logging changes

* Add a stop hook after all tests complete

* Refactor test_orc code

* Add arg passing
Add option to use locally cloned via install or remote via main project network orchestrator

* Fix baseline module
Fix orchestrator exiting only after timeout

* Add result file to baseline test module
Change result format to match closer to design doc

* Refactor pylint

* Skip test module if it failed to start

* Refactor

* Check for valid log level

* Add config file arg
Misc changes to network start procedure

* fix merge issues

* Update runner and test orch procedure
Add useful runtiem args

* Restructure test run startup process
Misc updates to work with net orch updates

* Refactor

---------

* Quick refactor (#9)

* Fix duplicate sleep calls

* Add net orc (#11)

* Add network orchestrator repository

* cleanup duplicate start and install scripts

* Temporary fix for python dependencies

* Remove duplicate python requirements

* remove duplicate conf files

* remove remote-net option

* cleanp unecessary files

* Add the DNS test module (#12)

* Add network orchestrator repository

* cleanup duplicate start and install scripts

* Temporary fix for python dependencies

* Remove duplicate python requirements

* remove duplicate conf files

* remove remote-net option

* cleanp unecessary files

* Add dns test module
Fix test module build process

* Add mac address of device under test to test container
Update dns test to use mac address filter

* Update dns module tests

* Change result output

* logging update

* Update test module for better reusability

* Load in module config to test module

* logging cleanup

* Update baseline module to new template
Misc cleanup

* Add ability to disable individual tests

* remove duplicate readme

* Update device directories

* Remove local folder

* Update device template
Update test module to work with new device config file format

* Change test module network config options
Do not start network services for modules not configured for network

* Refactor

---------

* Add baseline and pylint tests (#25)

* Discover devices on the network (#22)

* Discover devices on the network

* Add defaults when missing from config
Implement monitor wait period from config

* Add steady state monitor
Remove duplicate callback registrations

* Load devices into network orchestrator during testrun start

---------

Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com>

* Build dependencies first (#21)

* Build dependencies first

* Remove debug message

* Add depend on option to test modules

* Re-add single interface option

* Import subprocess

---------

Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com>

* Port scan test module (#23)

* Add network orchestrator repository

* cleanup duplicate start and install scripts

* Temporary fix for python dependencies

* Remove duplicate python requirements

* remove duplicate conf files

* remove remote-net option

* cleanp unecessary files

* Add dns test module
Fix test module build process

* Add mac address of device under test to test container
Update dns test to use mac address filter

* Update dns module tests

* Change result output

* logging update

* Update test module for better reusability

* Load in module config to test module

* logging cleanup

* Update baseline module to new template
Misc cleanup

* Add ability to disable individual tests

* remove duplicate readme

* Update device directories

* Remove local folder

* Update device template
Update test module to work with new device config file format

* Change test module network config options
Do not start network services for modules not configured for network

* Initial nmap test module add
Add device ip resolving to base module
Add network mounting for test modules

* Update ipv4 device resolving in test modules

* Map in ip subnets and remove hard coded references

* Add ftp port test

* Add ability to pass config for individual tests within a module
Update nmap module scan to run tests based on config

* Add full module check for compliance

* Add all tcp port scans to config

* Update nmap commands to match existing DAQ tests
Add udp scanning and tests

* logging cleanup

* Update TCP port scanning range
Update logging

* Merge device config into module config
Update device template

* fix merge issues

* Update timeouts
Add multi-threading for multiple scanns to run simultaneously
Add option to use scan scripts for services

* Fix merge issues

* Fix device configs

* Remove unecessary files

* Cleanup duplicate properties

* Cleanup install script

* Formatting (#26)

* Fix pylint issues in net orc

* more pylint fixes

* fix listener lint issues

* fix logger lint issues

* fix validator lint issues

* fix util lint issues

* Update base network module linting issues

* Cleanup linter issues for dhcp modules
Remove old code testing code

* change to single quote delimeter

* Cleanup linter issues for ntp module

* Cleanup linter issues for radius module

* Cleanup linter issues for template module

* fix linter issues with faux-dev

* Test results (#27)

* Collect all module test results

* Fix test modules without config options

* Add timestamp to test results

* Test results (#28)

* Collect all module test results

* Fix test modules without config options

* Add timestamp to test results

* Add attempt timing and device info to test results

* Ignore disabled test containers when generating results

* Fully skip modules that are disabled

* Fix pylint test and skip internet tests so CI passes (#29)

* disable internet checks for pass

* fix pylint test

* Increase pylint score (#31)

* More formatting fixes

* More formatting fixes

* More formatting fixes

* More formatting fixes

* Misc pylint fixes
Fix test module logger

---------

Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com>

* Pylint (#32)

* More formatting fixes

* More formatting fixes

* More formatting fixes

* More formatting fixes

* Misc pylint fixes
Fix test module logger

* remove unused files

* more formatting

* revert breaking pylint changes

* more formatting

* fix results file

* More formatting

* ovs module formatting

---------

Co-authored-by: Jacob Boddey <boddey@google.com>

---------

Co-authored-by: jhughesbiot <50999916+jhughesbiot@users.noreply.github.com>
Co-authored-by: jhughesbiot <jonathan.hughes@buildingsiot.com>
Co-authored-by: Noureddine <noureddine@saidi.uk>

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,32 @@
+---
+name: Bug report
+about: Create a report to help us identify and resolve bugs
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Error logs**
+If applicable, provide a log from https://gist.github.com/
+
+**Environment (please provide the following information about your setup):**
+ - OS: [e.g. Ubuntu]
+ - Version [e.g. 22.04]
+ - Additional hardware (network adapters)
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest a new feature or change request
+title: ''
+labels: request
+assignees: ''
+
+---
+
+**What is the problem your feature is trying to solve?**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you think would solve the problem**
+A clear and concise description of what you want to happen.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/workflows/testing.yml

@@ -0,0 +1,30 @@
+name: Testrun test suite
+
+on:
+  push:
+  pull_request:
+  schedule:
+    - cron: '0 13 * * *'
+
+jobs:
+  testrun:
+    name:  Baseline
+    runs-on: ubuntu-20.04
+    timeout-minutes: 20
+    steps:
+      - name: Checkout source
+        uses: actions/checkout@v2.3.4
+      - name: Run tests
+        shell: bash {0}
+        run: testing/test_baseline
+
+  pylint:
+    name:  Pylint
+    runs-on: ubuntu-20.04
+    timeout-minutes: 20
+    steps:
+      - name: Checkout source
+        uses: actions/checkout@v2.3.4
+      - name: Run tests
+        shell: bash {0}
+        run: testing/test_pylint

.gitignore

@@ -1,3 +1,7 @@
-venv/
-net_orc/
-.vscode/
+runtime/
+venv/
+.vscode/
+error
+pylint.out
+local/
+__pycache__/