Fix FitNesse architecture and comments.

esa · Oct 23, 2023 · 2c63a4e · 2c63a4e
1 parent fe0d1a9
commit 2c63a4e
Show file tree

Hide file tree

Showing 17 changed files with 371 additions and 90 deletions.
diff --git a/testbeds/testbed-mal/src/main/fitnesse/FitNesseRoot/MalTests/TestDocument/content.txt b/testbeds/testbed-mal/src/main/fitnesse/FitNesseRoot/MalTests/TestDocument/content.txt
@@ -36,16 +36,21 @@
 !see TestScenarios.Section4Scenarios.PubSubTestCase.SubscriptionTestProcedure
 !see TestScenarios.Section4Scenarios.PubSubTestCase.MultiTypeTestProcedure
 !see TestScenarios.Section4Scenarios.DataTypeTestScenario
+!see TestScenarios.Section4Scenarios.DataTypeTestScenario.ExplicitTypesTestProcedure
+!see TestScenarios.Section4Scenarios.DataTypeTestScenario.ElementParameterTestProcedure
+!see TestScenarios.Section4Scenarios.DataTypeTestScenario.MultiReturnTestProcedure
+!see TestScenarios.Section4Scenarios.DataTypeTestScenario.PolymorphicTypesTestProcedure
+!see TestScenarios.Section4Scenarios.DataTypeTestScenario.MoObjectTestProcedure
 !see TestScenarios.Section4Scenarios.ErrorTestScenario
 !see TestScenarios.Section4Scenarios.ErrorTestScenario.TransportErrors
 !see TestScenarios.Section4Scenarios.ErrorTestScenario.SecurityErrors
 !see TestScenarios.Section4Scenarios.AccessControlTestScenario
-!see TestScenarios.Section4Scenarios.TransportTestScenario
-!see TestScenarios.Section4Scenarios.TransportTestScenario.IsSupportedQosTestCase
-!see TestScenarios.Section4Scenarios.TransportTestScenario.IsSupportedIpTestCase
-!see TestScenarios.Section4Scenarios.TransportTestScenario.TransmitTestCase
-!see TestScenarios.Section4Scenarios.TransportTestScenario.TransmitErrorTestCase
-!see TestScenarios.Section4Scenarios.TransportTestScenario.TransmitMultipleTestCase
-!see TestScenarios.Section4Scenarios.TransportTestScenario.ReceiveTestCase
-!see TestScenarios.Section4Scenarios.TransportTestScenario.ReceiveMultipleTestCase
+#!see TestScenarios.Section4Scenarios.TransportTestScenario
+#!see TestScenarios.Section4Scenarios.TransportTestScenario.IsSupportedQosTestCase
+#!see TestScenarios.Section4Scenarios.TransportTestScenario.IsSupportedIpTestCase
+#!see TestScenarios.Section4Scenarios.TransportTestScenario.TransmitTestCase
+#!see TestScenarios.Section4Scenarios.TransportTestScenario.TransmitErrorTestCase
+#!see TestScenarios.Section4Scenarios.TransportTestScenario.TransmitMultipleTestCase
+#!see TestScenarios.Section4Scenarios.TransportTestScenario.ReceiveTestCase
+#!see TestScenarios.Section4Scenarios.TransportTestScenario.ReceiveMultipleTestCase
 !see TestScenarios.RegressionScenarios.FastProvider
diff --git a/...narios/Section4Scenarios/AllExceptPubSubTestCase/AllExceptPubSubTestProcedure/content.txt b/...narios/Section4Scenarios/AllExceptPubSubTestCase/AllExceptPubSubTestProcedure/content.txt
@@ -25,18 +25,47 @@
 |org.ccsds.moims.mo.mal.test.patterns|
 
 *******!
-!4 Test procedure: <ip>/<qos>/<session>/<transition list id>
-The test procedure does the following actions:
- 1 Call the operation <ip> provided by the IPTest service with the parameters passed to the test procedures: 'ip', 'qos', 'session' and 'transition list id'.
+!4 Test procedure: all patterns except Pub/Sub
+The test procedure iterates over the following actions:
+ 1 Call the proper pattern type operation provided by the IPTest service with the parameters passed to the test procedure.
  2 Wait for the expected transitions to be done, except the faulty ones that are ignored.
  3 Check the message headers
  4 Check the transitions
 
 It is necessary to go through all the transitions of the IP state charts, the faulty transitions are underlined.
-In the faulty case, the field 'errorCode' of the IPTestTransition is set to the value INCORRECT_STATE otherwise it
-is set to "-1". The field 'Transition list id' is used to identify the test procedure.
+In the faulty case, the field 'errorCode' of the IPTestTransition is set to the value INCORRECT_STATE otherwise it is set to "-1".
+
+The test procedure relies on parameterized FitNesse sub scenarios in order to iterate through all the tested behaviours.
+
+The consumer initiates the patterns by calling the following operations provided by the service IPTest:
+ * testSubmit
+ * request
+ * invoke
+ * progress
+Those operations accept a single IPTestDefinition parameter. This parameter is notably used by the provider to execute a predefined behaviour.
+Variants of those operations are defined by the service IPTest, with an empty body for the OUT messages, or with an additional parameter in the body of the initial IN message:
+ * submitMulti
+ * testRequestEmptyBody, requestMulti
+ * testInvokeEmptyBody, invokeMulti
+ * testProgressEmptyBody, progressMulti
+Those additional operations implement the same behaviour than the regular ones, with the same IPTestDefinition parameter.
+All those operations shall be called in the FitNesse sub scenario named !-PatternWithSupplementsAndTransitionsAndBehaviourIdTest-!.
+This sub scenario shall be executed several times for each tested behaviour, with additional environment parameters, in a sub scenario named !-TestInteractionPatternWithTransitionsAndBehaviourId-!.
+
+The possible values of the sub scenario parameters are:
+-!|Comment|
+|parameter|values|comment|
+|interaction pattern|Submit, Request, Invoke, Progress|name of the tested IP|
+|transitions|{_M_}*,{M}*,{_M_}*, with M in ACK, ACK_ERROR, UPDATE, UPDATE_ERROR, RESPONSE, RESPONSE_ERROR|list of expected messages to be sent by the provider according to the tested behaviour, starting with a list of initial faulty messages sent by the testbed, followed by the list of regular messages sent by the provider, followed by a list of final faulty messages sent by the testbed|
+|behaviour id|numeric|numeric identifier used by the consumer and the provider to perform actions specific to the tested behaviour|
+|supplements|Empty, Full|environment parameter used to set the supplements header field, either empty or with a full set of various attribute types|
+
+The 'transitions' parameter describes the MAL messages exchanged from the provider MAL to the consumer MAL. It is used by the consumer to check the correctness of the indications it receives according to the test procedure.
+The 'behaviour id' parameter is used by the provider to actually send the messages expected by the test procedure.
+
+The transitions are checked by ensuring that the consumer receives proper indications, even when the consumer MAL receives faulty messages.
+
 
-The transitions are checked by ensuring that the faulty messages are not received by the consumer.
 
 !*> Sub scenario definitions
 |scenario| pattern | interaction | with supplements| supplements | and transitions | trans | and behaviour id | id | test |

diff --git a/...FitNesseRoot/MalTests/TestScenarios/Section4Scenarios/AllExceptPubSubTestCase/content.txt b/...FitNesseRoot/MalTests/TestScenarios/Section4Scenarios/AllExceptPubSubTestCase/content.txt
@@ -23,27 +23,5 @@
 
 *******!
 !3 Test case: all patterns except Pub/Sub
-The consumer initiates the patterns by calling the following operations provided by the service IPTest:
- * send
- * submit
- * request
- * invoke
- * progress
-Those operations shall be called once for each !-QoS-! level and session type. It is not necessary to test each combination of !-QoS-! and session. One call for each !-QoS-! level and session type is enough.
-
-The following parameters are used to make the calls:
-|comment|
-|authenticationId|	{0x00, 0x01}|
-|qos|	Best Effort, Assured, Queued, Timely|
-|priority|	1|
-|domain|	{"Test", "Domain"}|
-|networkZone|	"!-TestNetwork-!"|
-|session|	Live, Simulation, Replay|
-|session name|!{Live:"LIVE", Replay:"R1", Simulation:"S1"}|
-
-The consumer has to execute the following test procedure for every possible header values (!-QoS-! and session fields) and every possible transitions. The name of the procedure is built from the parameters:
- * ip: name of the tested IP
- * qos: !-QoS-! level to be used
- * session: session type to be used
- * transition list id: identifier of the IPTestTransitionList that is expected by the consumer (see table above).
 
+A single test procedure is defined and executed multiple times to cover all the tested behaviours.
diff --git a/...cenarios/Section4Scenarios/DataTypeTestScenario/ElementParameterTestProcedure/content.txt b/...cenarios/Section4Scenarios/DataTypeTestScenario/ElementParameterTestProcedure/content.txt
@@ -0,0 +1,56 @@
+# ----------------------------------------------------------------------------
+# Copyright (C) 2013      European Space Agency
+#                         European Space Operations Centre
+#                         Darmstadt
+#                         Germany
+# ----------------------------------------------------------------------------
+# System                : CCSDS MO MAL Test bed
+# ----------------------------------------------------------------------------
+# Licensed under the European Space Agency Public License, Version 2.0
+# You may not use this file except in compliance with the License.
+#
+# Except as expressly set forth in this License, the Software is provided to
+# You on an "as is" basis and without warranties of any kind, including without
+# limitation merchantability, fitness for a particular purpose, absence of
+# defects or errors, accuracy or non-infringement of intellectual property rights.
+# 
+# See the License for the specific language governing permissions and
+# limitations under the License. 
+# ----------------------------------------------------------------------------
+
+!*< Hidden
+!define TEST_SYSTEM {slim}
+
+|Import|
+|org.ccsds.moims.mo.mal.test.datatype|
+
+*!
+!4 Element parameter test procedure
+
+The test procedure does the following actions:
+ 1 Call the generic testData operation provided by the !-DataTest-! service provider with a statically defined value.
+ 2 Check that no error is raised by the provider, especially DATA_ERROR and BAD_ENCODING, meaning that the provider received the right value.
+ 3 Check the return value which must equal to the original value.
+
+The generic testData operation of the !-DataTest-! service is defined as: Element testData(Element).
+
+A list of MAL data structure instances is statically defined according to the following constraints:
+ * All the data types shall be instantiated at least once.
+ * Enumerations shall be instantiated once for each enumerated value.
+ * Abstract types need to be extended by a concrete type for the test
+ * The value Null shall belong to the list
+ * The value Null shall be inserted into a Composite structure
+
+This data list is statically known by the !-DataTest-! service provider and consumer.
+The consumer takes the data from the list one by one, in the same order, and applies the test procedure.
+
+|script| data type scenario|
+|note| Element parameter test|
+|check| attribute types work| OK|
+|check| enumerations work| OK|
+|check| complete composites work | OK|
+|check| abstract composites work | OK|
+|check| lists work | OK|
+|check| nulls work|OK|
+|check| composites with null work| OK|
+|check| polymorphic types work | OK|
diff --git a/...arios/Section4Scenarios/DataTypeTestScenario/ElementParameterTestProcedure/properties.xml b/...arios/Section4Scenarios/DataTypeTestScenario/ElementParameterTestProcedure/properties.xml
@@ -0,0 +1,11 @@
+<?xml version="1.0"?>
+<properties>
+	<Edit>true</Edit>
+	<Files>true</Files>
+	<Properties>true</Properties>
+	<RecentChanges>true</RecentChanges>
+	<Refactor>true</Refactor>
+	<Search>true</Search>
+	<Versions>true</Versions>
+	<WhereUsed>true</WhereUsed>
+</properties>
diff --git a/...stScenarios/Section4Scenarios/DataTypeTestScenario/ExplicitTypesTestProcedure/content.txt b/...stScenarios/Section4Scenarios/DataTypeTestScenario/ExplicitTypesTestProcedure/content.txt
@@ -0,0 +1,60 @@
+# ----------------------------------------------------------------------------
+# Copyright (C) 2013      European Space Agency
+#                         European Space Operations Centre
+#                         Darmstadt
+#                         Germany
+# ----------------------------------------------------------------------------
+# System                : CCSDS MO MAL Test bed
+# ----------------------------------------------------------------------------
+# Licensed under the European Space Agency Public License, Version 2.0
+# You may not use this file except in compliance with the License.
+#
+# Except as expressly set forth in this License, the Software is provided to
+# You on an "as is" basis and without warranties of any kind, including without
+# limitation merchantability, fitness for a particular purpose, absence of
+# defects or errors, accuracy or non-infringement of intellectual property rights.
+# 
+# See the License for the specific language governing permissions and
+# limitations under the License. 
+# ----------------------------------------------------------------------------
+
+!*< Hidden
+!define TEST_SYSTEM {slim}
+
+|Import|
+|org.ccsds.moims.mo.mal.test.datatype|
+
+*!
+!4 Explicit types test procedure
+
+The test procedure does the following actions:
+ 1 Call the proper data type operation provided by the !-DataTest-! service provider with a statically defined value.
+ 2 Check that no error is raised by the provider, especially DATA_ERROR and BAD_ENCODING, meaning that the provider received the right value.
+ 3 Check the return value which must equal to the original value.
+
+The test procedure is applied to all the MAL Attribute data types.
+
+The data values are statically known by the !-DataTest-! service provider and consumer. This allows the provider to check that the value he receives equals to the value sent by the consumer.
+The test operation of the !-DataTest-! service is specific to the tested type T. It is defined as: <T> testData<T>(<T>).
+
+|script| data type scenario|
+|note| Explicit types test|
+|check| explicit duration type works| OK|
+|check| explicit Fine Time type works| OK|
+|check| explicit Identifier type works| OK|
+|check| explicit Time type works| OK|
+|check| explicit URI type works| OK|
+|check| explicit Blob type works| OK|
+|check| explicit Boolean type works| OK|
+|check| explicit Octet type works| OK|
+|check| explicit Double type works| OK|
+|check| explicit Float type works| OK|
+|check| explicit Integer type works| OK|
+|check| explicit Long type works| OK|
+|check| explicit Short type works| OK|
+|check| explicit String type works| OK|
+|check| explicit UOctet type works| OK|
+|check| explicit UInteger type works| OK|
+|check| explicit ULong type works| OK|
+|check| explicit UShort type works| OK|
+|check| explicit Object Ref type works| OK|
diff --git a/...cenarios/Section4Scenarios/DataTypeTestScenario/ExplicitTypesTestProcedure/properties.xml b/...cenarios/Section4Scenarios/DataTypeTestScenario/ExplicitTypesTestProcedure/properties.xml
@@ -0,0 +1,11 @@
+<?xml version="1.0"?>
+<properties>
+	<Edit>true</Edit>
+	<Files>true</Files>
+	<Properties>true</Properties>
+	<RecentChanges>true</RecentChanges>
+	<Refactor>true</Refactor>
+	<Search>true</Search>
+	<Versions>true</Versions>
+	<WhereUsed>true</WhereUsed>
+</properties>
diff --git a/...ts/TestScenarios/Section4Scenarios/DataTypeTestScenario/MoObjectTestProcedure/content.txt b/...ts/TestScenarios/Section4Scenarios/DataTypeTestScenario/MoObjectTestProcedure/content.txt
@@ -0,0 +1,38 @@
+# ----------------------------------------------------------------------------
+# Copyright (C) 2013      European Space Agency
+#                         European Space Operations Centre
+#                         Darmstadt
+#                         Germany
+# ----------------------------------------------------------------------------
+# System                : CCSDS MO MAL Test bed
+# ----------------------------------------------------------------------------
+# Licensed under the European Space Agency Public License, Version 2.0
+# You may not use this file except in compliance with the License.
+#
+# Except as expressly set forth in this License, the Software is provided to
+# You on an "as is" basis and without warranties of any kind, including without
+# limitation merchantability, fitness for a particular purpose, absence of
+# defects or errors, accuracy or non-infringement of intellectual property rights.
+# 
+# See the License for the specific language governing permissions and
+# limitations under the License. 
+# ----------------------------------------------------------------------------
+
+!*< Hidden
+!define TEST_SYSTEM {slim}
+
+|Import|
+|org.ccsds.moims.mo.mal.test.datatype|
+
+*!
+!4 MO Object test procedure
+
+The test procedure does the following actions:
+ 1 Call the objectAssertionsAreChecked operation provided by the !-DataTest-! service provider.
+ 2 Let the provider check a list of assertions related to the MO Object concept.
+ 3 Check the return value which confirms the validation of those assertions.
+
+
+|script| data type scenario|
+|note| MO Object test|
+|check| object assertions are checked | OK|
diff --git a/...TestScenarios/Section4Scenarios/DataTypeTestScenario/MoObjectTestProcedure/properties.xml b/...TestScenarios/Section4Scenarios/DataTypeTestScenario/MoObjectTestProcedure/properties.xml
@@ -0,0 +1,11 @@
+<?xml version="1.0"?>
+<properties>
+	<Edit>true</Edit>
+	<Files>true</Files>
+	<Properties>true</Properties>
+	<RecentChanges>true</RecentChanges>
+	<Refactor>true</Refactor>
+	<Search>true</Search>
+	<Versions>true</Versions>
+	<WhereUsed>true</WhereUsed>
+</properties>
diff --git a/...TestScenarios/Section4Scenarios/DataTypeTestScenario/MultiReturnTestProcedure/content.txt b/...TestScenarios/Section4Scenarios/DataTypeTestScenario/MultiReturnTestProcedure/content.txt
@@ -0,0 +1,47 @@
+# ----------------------------------------------------------------------------
+# Copyright (C) 2013      European Space Agency
+#                         European Space Operations Centre
+#                         Darmstadt
+#                         Germany
+# ----------------------------------------------------------------------------
+# System                : CCSDS MO MAL Test bed
+# ----------------------------------------------------------------------------
+# Licensed under the European Space Agency Public License, Version 2.0
+# You may not use this file except in compliance with the License.
+#
+# Except as expressly set forth in this License, the Software is provided to
+# You on an "as is" basis and without warranties of any kind, including without
+# limitation merchantability, fitness for a particular purpose, absence of
+# defects or errors, accuracy or non-infringement of intellectual property rights.
+# 
+# See the License for the specific language governing permissions and
+# limitations under the License. 
+# ----------------------------------------------------------------------------
+
+!*< Hidden
+!define TEST_SYSTEM {slim}
+
+|Import|
+|org.ccsds.moims.mo.mal.test.datatype|
+
+*!
+!4 Multi return test procedure
+
+The test procedure does the following actions:
+ 1 Call a specific operation provided by the !-DataTest-! service provider with a set of statically defined values.
+ 2 Check that no error is raised by the provider, especially DATA_ERROR and BAD_ENCODING, meaning that the provider received the right values.
+ 3 Check the return values which must equal to the original values.
+
+The test operation of the !-DataTest-! service declares 4 input parameters and 4 return parameters. The objective is not to test all data types, as this has already been done with the ExplicitTypesTestProcedure. The objective is to check various use cases when the data is not the single return parameter. A specific operation is defined for each use case as the type of the declared parameter is important.
+The tested use cases include:
+ 1 a set of explicitely defined types
+ 2 a set of parameters with the last one declared as Element
+ 3 a set of parameters with the 2nd and 3rd ones declared as Element
+ 4 a set of parameters including a NULL value
+
+|script| data type scenario|
+|note| Multi return test|
+|check| explicit multi return works| OK|
+|check| abstract multi return works | OK|
+|check| inner abstract multi return works | OK|
+|check| multi return with nulls work| OK|
diff --git a/...tScenarios/Section4Scenarios/DataTypeTestScenario/MultiReturnTestProcedure/properties.xml b/...tScenarios/Section4Scenarios/DataTypeTestScenario/MultiReturnTestProcedure/properties.xml
@@ -0,0 +1,11 @@
+<?xml version="1.0"?>
+<properties>
+	<Edit>true</Edit>
+	<Files>true</Files>
+	<Properties>true</Properties>
+	<RecentChanges>true</RecentChanges>
+	<Refactor>true</Refactor>
+	<Search>true</Search>
+	<Versions>true</Versions>
+	<WhereUsed>true</WhereUsed>
+</properties>