Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[c++] Add an abstraction layer between SOMA columns and TileDB dimensions and attributes #3337

Open
XanthosXanthopoulos opened this issue Nov 19, 2024 · 0 comments · May be fixed by #3407, #3405, #3424, #3426 or #3427
Assignees
Labels
enhancement New feature or request

Comments

@XanthosXanthopoulos
Copy link
Collaborator

Currently there is a 1-1 mapping between TileDB dimension as index columns in SOMA objects and TileDB attributes and non-index columns. The datatype for index columns should match the supported types for TileDB dimensions and the same holds true for attributes as well.

With the introduction of the GeometryDataFrame the two previous condition are not true anymore. soma_geometry is an index column with binary datatype holding WKB encoded geometries. TileDB doesn't support binary blobs as dimensions out of the box and even if it did, spatial indexing would require decoding the underlying WKB blob. To provide spatial indexing GeometryDataFrame utilizes addition dimensions and makes the actual soma_geometry an attribute of type WKB.

Dimensions and attributes generated internally for indexing should be hidden to the end user and any kind of interaction with that column should be through the original index column name.

This approach requires special handling of that column in multiple places throughout the codebase. Any column in the future that would require special indexing will also face the same issues and would increase the complexity of the codebase.

(UPDATE: SOMA specification removed index columns from GeometryDataFrame with the possibility to add it back at a later stage)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment