Pipeline Validation Service

[akchinSTC]

Validation service is being served under /elyra/pipeline/validate

Validates basic pipeline payload JSON structure

• contains valid json
• contains a `primary pipeline’ and is a string
• contains at least one pipeline element in the json that is of type list
• checks pipeline schema version matches or works with the current version being run
• primary_pipeline is located in the pipelines array
• primary_pipeline has an app_data field that contains a version field that is a int
• primary_pipeline has a nodes field that is an array

Validate compatibility

• checks pipeline specific pipelines can only be run in their respective runtimes (metadata endpoints)
• checks that the runtime specified in the payload is a supported runtime
• based on the version of Elyra, determines what operations are supported
• support validating of pre 3.0 version pipelines e.g. generic components only

Validate node properties

• check filename for validity
• check filename for existence (file in filepath exists on FS)
• check resource values
• check image registry name
• check dependencies for validity
• check dependencies for existence (file in filepath exists on FS)
• check env vars are properly formatted
• check ui data label validity against acceptable runtime naming standards (only kfp for now)
• check runtime specific components properties in pipeline payload compared to current component registry
  • all required fields specified by the component properties exist in the node payload
  • all required fields are type checked

Validate the graph

• check for graph cycles (circular reference)
• check for singleton nodes (nodes not connected to any part of the pipeline) (only warning)
• check supernodes for the above conditions

Responses will have the following format (example shows error - a non-numeric string was used for the CPU value when validating the resources) and will differ based on the type of error discovered:

{
    "title": "Elyra Pipeline Diagnostics",
    "description": "Issues discovered when parsing the pipeline",
    "issues": [
        {
            "severity": "1",
            "source": "Elyra Pipeline Validation Service",
            "type": "invalidNodeProperty",
            "message": "Property has a non-parsable value",
            "data": {
                "nodeID": "4b1a73b3-3b64-4818-ab98-73965410ea10",
                "propertyName": "cpu"
            }
        },
     ]
}

Severity Levels:
Currently support 4 severity levels on a scale of 1 to 4: Error, Warning, Info, Hint
Only Errors will cause a fatal exception during pipeline submission and halt the run.

Error Types:

invalidJSON - Invalid JSON detected
data: None

invalidPipeline - Pipeline level payload structure is incomplete, missing or malformed
data: None

invalidRuntime - Incompatible/Unsupported pipeline runtime associated with the pipeline file
data: pipelineRuntime, pipelineID

invalidNodeType - Unsupported node type in the pipeline for the runtime
data: NodeID, PipelineID

invalidNodeProperty - Node level payload structure is incomplete, missing or malformed
data: NodeID, Node Property Name

invalidFilePath - Property has an invalid path to a file/dir or the file/dir does not exist
data: nodeID, Node Property Name, filepath

invalidEnvPair - Invalid env pair in node property
data: nodeID, Node Property Name, env pair

invalidNodeLabel: Label associated with node is invalid for the runtime
data: nodeID, Node Property Name, label name

circularReference: a cycle was found in the graph
data: cycle number, linkIDList (list of linkIDs associated with the cycle)

Warning Types:

singletonReference: node is not connected to any part of the pipeline
data: nodeID, pipelineID

Additional things we should check for

• Check image name registry coordinate syntax - Move to metadata service?

Additional topics of discussion:

• Making the validation service more performant
  • The currently implementation is a bit slow when used for real time pipeline checks and could bog the backend down when queries have a short interval