Data science and kart: incorporating non-geospatial data and long-term archiving for repositories

[tcwilkinson]

I am currently in process of learning kart, as have been searching for a good spatial and database version control system for a while.

My use cases are in archaeological data science. I already use git to track spatial files mostly in [original format] GeoJSON and tabular data in CSV, with accompanying explanatory info in plain text or markdown.

It's important that the working archives are in plain text as much as possible to facilitate long-term archiving of data, but also because it reduces the chances of format orphaning. (Archaeological projects often take years if not decades to go from field to publication, yet there are few data standards, so making things easy to navigate is important).

Git helps, but tracking changes with spatial and tabular data is still clunky (not surprising, it was designed for line-based code). So far, I like what I see in kart for spatial data... and would like to adopt it for some projects. But have some questions about the direction of travel and how it would fit with data science workflows, so wanted to open a discussion here for both kart developers and wider users.

1. How easy is it to incorporate non-spatial data into the same repository? For example, text files which should be in version-sync with the data, to help readers to interpret and re-use the data schemas in use, or the history of the data production. (I see the attachment feature, Attachment support for Kart #583 , but it currently seems unfinished).
2. How human-readable are kart repositories long-term? If kart was no longer developed, would the next generation of data scientists be able to decode easily and re-use the kart-managed part of the repository (either the history or the latest version)?

[rcoup]

Hi Toby,

Thanks for reaching out

How easy is it to incorporate non-spatial data into the same repository? For example, text files which should be in version-sync with the data, to help readers to interpret and re-use the data schemas in use, or the history of the data production. (I see the attachment feature, #583 , but it currently seems unfinished).

It'll definitely happen, but it's not on the do-next list at the moment.

How human-readable are kart repositories long-term? If kart was no longer developed, would the next generation of data scientists be able to decode easily and re-use the kart-managed part of the repository (either the history or the latest version)?

Kart is built on Git which has a strong history of backwards compatibility and legacy platform support, and we document the format and internals of how we store features/etc within Git objects. So while Kart repositories aren't human readable, doing a one-off conversion/extract (eg: to GeoJSON) without any Kart code running would not be a particularly complex task for a developer.