getCharacterizationResults - error when pulling back a run with a generation on a source that was deleted

[jhardin29]

For characterization 240 - the request is finding all generations across all sources, and there is a record of a generation on a source that was deleted.

So when ROhdsiWebApi tries to look up the source for that generation, it doesn't find it, which results in an error

##Request:
Employ data cleanup such that a deleted run won't raise an error when attempting to pull back results.

Tagging @chrisknoll because we chatted about this issue.

[chrisknoll]

right. this is compounded by marking a source 'deleted' doesn't actually delete the record, so we can't depend on FK cascade deletes to clean up generations on sources that are deleted. But, there should probably be some cleanup that when a source gets deleted (or a check that only returns non-deleted sources in results).

Tagging @anthonysena : I think you thought about soft-deletes on sources, and wanted your thoughts on handling this.

[anthonysena]

I'm not sure of the query/endpoint in ROhdsiWebApi that is driving this behavior but it is most likely we need to add a check on the source table to exclude those generations where "deleted_date IS NOT NULL"

[chrisknoll]

The error on the call comes from the following steps:

1. Get the list of generations for an entity (example: CohortCharacterizations). This will contain a set of generationIDs and what source_id they were on.
2. Query the results by looking up the Source by source_id to get the connection details
3. Error: source not found (Hibernate error).

Probably the logic should be: 1) get the list of available sources, 2) get the list of generations, 3) filter generations to where the source_id exists in the available sources, 4) return the generation_ids.