Field Selection: Reconsider support for field index,* for all other fields

[jerstlouis]

Currently, the properties= parameter requires support for numeric indices and for * with special meaning of "all other fields".

Perhaps we should reconsider this to make things simpler to implement, and to better align with other OGC APIs.
(e.g., Features - Part 6: property selection).

opengeospatial/ogcapi-features#931

Consider a negative exclude-properties as for Features (see opengeospatial/ogcapi-features#16 (comment) opengeospatial/ogcapi-features#738 (comment)), which would be mutually exclusive with properties=.

Also clarify the ability to not return all fields by default.

[fmigneault]

+1
Especially regarding the * and numeric indices.

When properties are defined in an object (dictionary/map), there is no guaranteed ordering.
Definitions like properties=B07,* (https://docs.ogc.org/DRAFTS/19-087.html#field-selection-examples) seem to imply B07 up to B12 was intended, but this is only because people tend to order them alpha-numerically. The use of * generally means "any", and B07,* == * (and also equal to omitting properties entirely) could be a correct interpretation.

If properties are used, they should not imply any form of ordering. Each item to include in the result should be listed (or excluded) explicitly.

[jerstlouis]

When properties are defined in an object (dictionary/map), there is no guaranteed ordering.

This is what the x-ogc-propertySeq is for in the schema, which defines the property order, which is important for example in a GeoTIFF output which doesn't itself have any semantic or even field name information about individual bands.

equal to omitting properties entirely

There's also the permission to not return all bands by default, so * in that case would not have been the same as omitting properties entirely. For example, for our sentinel-2 datacube we only return B04,B03,B02 by default if not explicitly asking for all bands (because it's very costly, especially at low resolutions without small enough overviews in the COGs, to gather all 15 bands instead of just 3 bands)

One potential way to indicate this would be by only specifying an x-ogc-propertySeq for the bands that are returned by default. So we would change our schema to say "x-ogc-propertySeq": 1 for B04, 2 for B03, 3 for B02, and the others would not specify that property.

[fmigneault]

This is what the x-ogc-propertySeq is for in the schema

Good to know. I was not aware of this property.

My personal take on this is that, ideally, properties= should be as agnostic as possible of this, the reason being that interpreting a URI that includes properties= becomes much more convoluted than if the properties were listed explicitly. One always needs to double-check the presence x-ogc-propertySeq, and that they match expectations.

permission to not return all bands by default

Which parameter (if any) controls/exposes this behavior? Can it be set for distinct collections?
If only the default bands included x-ogc-propertySeq, that would make any other sequence combinations unpredictable, such as properties=B04,B03,B9,*. Seems to defeat the purpose of x-ogc-propertySeq.

[jerstlouis]

If only the default bands included x-ogc-propertySeq, that would make any other sequence combinations unpredictable, such as properties=B04,B03,B9,*

The x-ogc-propertySeq number is only for when NOT using properties=. If using properties=, it's the sequential order of the properties specified in that parameter that defines the sequence order.
It does bring unpredictability with this ,*, which is one of the reason for proposing to drop this * altogether.

Which parameter (if any) controls/exposes this behavior? Can it be set for distinct collections?

It would be a server-side collection-specific decision for when properties= is not used by the client.

[fmigneault]

Hold on, I'm confused.
I thought the reason x-ogc-propertySeq was brought up was to address the case of properties=B07,* where * cannot be assumed to be B08, B8A, etc. because of a mapping representation does not guarantee the order, and that x-ogc-propertySeq must be used to resolve it, as per https://docs.ogc.org/DRAFTS/19-087.html#_parameter_properties requirements?

I mean, if the proposal is to drop * entirely, forcing to list all desired properties in their specific order, then you got my +1
That is way better.

[jerstlouis]

if the proposal is to drop * entirely, forcing to list all desired properties in their specific order

That is the proposal.

The x-ogc-propertySeq order would only be relevant when not using properties=.

Note that we're also proposing exclude-properties so specify it the other way around, also to align with features (useful when there are fewer properties you don't want than those you want). In this case, the x-ogc-propertySeq order would still be relevant for non-excluded properties.

[m-mohr]

+1 on removing the wildcard * - see also opengeospatial/ogcapi-features#944

By the way, STAC API has something similar which we could align to: https://github.com/stac-api-extensions/fields

[jerstlouis]

SWG 2024-12-04:

• Discussed with @fmigneault and agreed to remove the *, except for the case where we explicitly want to request all fields (properties=*) when the default is to return only a subset of the field
• Regarding properties returned by default, we will separate the notion of returning a field or not by default in a property x-ogc-returnedByDefault with a default value is true. This way, the x-ogc-propertySeq can be set for all properties/fields, even those not returned by default, so that a client requesting all properties with * will have a deterministic order for formats such as GeoTIFF with no band information
• exclude-properties -- clarify with Features regarding the use case where a subset of properties is returned by default, to check whether the mutual exclusivity between properties and exclude-properties is still justified in this case (note the STAC use case of includew and exclude where both are also often used). Query parameter properties: align with OGC API Coverages ogcapi-features#931

[jerstlouis]

@fmigneault Regarding exclude-properties, I believe in Features the default set of properties when using exclude-properties would be all properties whether returnedByDefault is true or false. So you could actually do exclude-properties=<empty> as an alternative to using * -- perhaps we could drop properties=* as a requirement then? This also aligns with the text we have in OGC API - DGGS which is in the final stages. Implementations could still implement properties=* as an equivalent to exclude-properties= (we could make it an explicit permission).

[fmigneault]

I get the idea, but it sounds like a bad interface design.

For example, if you have B04,B03,B02 with the returnedByDefault: true, using exclude-properties= would actually cause an include of more properties than what is specified. If I was a user not experienced with this behavior, my first assumption would be that the server has a bug or doesn't support the query parameter.

An exclude query should really just remove properties without side effects. If the listed properties are already omitted, then it's a no-op.

I would expect exclude-properties=B02 using the same previous defaults to return only B04,B03. If everything else gets added as a result, that just seems wrong.