Multiple models outputs in one catalog?

[apreucil]

I am confused if the catalogs are meant to be unique per model run or if they can and should be combined. This would greatly change the way analysis scripts are created if multiple catalogs are needed to perform an analysis between several model runs. My question is how to combine catalogs if that is indeed an intended use of this tool?

[ceblanton]

Thank you, great question! I've thought of combining catalogs too a couple times too but the reasons for it and use cases haven't really been discussed. (Until now :).

In principle, multiple catalogs that use the same schema can be trivially combined (concatenating the csv's). Catalogs that use different schemas probably cannot be easily combined.

Why would you need to combine a catalog though? Here might be a common scenario:

1. PP single experiment, which creates a catalog of the output
2. Analysis script refers to 2 additional catalogs: a standard CMIP experiment say and observations
3. Analysis script uses the 3 catalogs to create difference plots.

In that case, would combining catalogs be useful? What case might it be useful for?

[aradhakrishnanGFDL]

So, we started off with a combined catalog (manually combined) use-case to do a multicase example for MDTF. I think it's a great use-case and with enough validation tools and best practices, the combining could be done outside of the catalog builder [or within, incrementally]

Added some examples and notes to the issue @ceblanton opened.

#56

[aradhakrishnanGFDL]

#58

I have added a preliminary cut of a simple script one can use to combine catalogs. I welcome contributions to make the script more robust.

*NOT tested the combined catalog with intake-esm.