Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor clean-up and more intro #58

Merged
merged 12 commits into from
Nov 18, 2020
Merged

Minor clean-up and more intro #58

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
13381aa
update stability note
cholmes Nov 13, 2020
a4cc645
added intro and section on compliance
cholmes Nov 13, 2020
ae4b0f1
added info on compliance addition
cholmes Nov 13, 2020
d223e44
better headings
cholmes Nov 13, 2020
19f0627
Merge branch 'master' into clean-up
matthewhanson Nov 18, 2020
1dbdece
fix invalid markdown link due to split across lines
matthewhanson Nov 18, 2020
fe80830
fix markdown warnings
matthewhanson Nov 18, 2020
cf46f0f
Merge branch 'master' into clean-up
matthewhanson Nov 18, 2020
12f67c4
Update api-spec.md
cholmes Nov 18, 2020
858caeb
took out compliance stuff, to put it in conformance PR
cholmes Nov 18, 2020
ab3cc73
updated changelog
cholmes Nov 18, 2020
c6b586b
took out changelog entry, as this is all minor stuff now
cholmes Nov 18, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Added
- Clearer language on compliance requirements

### Changed
- Context Extension OpenAPI spec was updated to remove the no longer used `next` attribute
Expand Down
17 changes: 8 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,14 @@ compatible and usable with any OGC API - Features clients. The STAC API can be t
to search STAC Catalogs, where the features returned are STAC [Items](https://github.com/radiantearth/stac-spec/item-spec/item-spec.md),
that have common properties, links to their assets and geometries that represent the footprints of the geospatial assets.

## WARNING

This specification is approaching maturity, but will likely have some minor changes as we approach the 1.0-beta release and
fully align with [OGC API - Features](http://docs.opengeospatial.org/is/17-069r3/17-069r3.html). This repo was also recently
split off from the main [stac-spec repository](https://github.com/radiantearth/stac-spec). Not all links and references have
been updated yet so should be considered a work in progress. To implement against a stable STAC API we recommend using
[STAC API Version 0.9.0](https://github.com/radiantearth/stac-spec/tree/v0.9.0/api-spec). This master branch will be
considered to be under active development until we release version 1.0-beta, at which time it will be stable, and a `dev`
branch will be created for active development.
## Stability Note

This specification has evolved over the past couple years, and is used in production in a variety of deployments. It is
currently in a 'beta' state, with no major changes anticipated. For 1.0-beta we remain fully aligned with [OGC API -
Features](http://docs.opengeospatial.org/is/17-069r3/17-069r3.html) Version 1.0. We have further aligned STAC extensions
with OGC API - Features extensions, particularly [CQL](https://github.com/opengeospatial/ogcapi-features/tree/master/extensions/cql/)
and [Transactions](https://github.com/opengeospatial/ogcapi-features/tree/master/extensions/transactions). These are not
yet entirely stable, so if they change then STAC will update to remain in line.

## Communication

Expand Down
51 changes: 41 additions & 10 deletions api-spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,27 @@
# STAC API Specification

The STAC API is intended to be a superset of the *OGC API - Features - Part 1: Core* (OAFeat) standard. STAC API
currently is based on [OAFeat version 1.0](http://docs.opengeospatial.org/is/17-069r3/17-069r3.html), previously known as
OGC Web Feature Service (WFS). Future STAC API releases will align with
[upcoming versions](https://github.com/opengeospatial/ogcapi-features).
## About

The STAC API defines a RESTful JSON-based server to query [SpatioTemporal Asset Catalogs](https://github.com/radiantearth/stac-spec/)
(STAC). While the core STAC specification provides a structure and language to describe assets, users usually want to access
a subset of the entire catalog, such as for a certain date range, in a particular area of interest, or matching properties
they care about. STAC API specifies those query parameters, and compliant servers return collections of STAC Items that
match the user's preferences.

## STAC API and OGC API - Features

The vast majority of what STAC API does is specified in the *[OGC API - Features](https://ogcapi.ogc.org/features/)*
standard ('OAFeat' for our shorthand). STAC API is fully compliant with [OAFeat - Part 1: Core, version
1.0](http://docs.opengeospatial.org/is/17-069r3/17-069r3.html), and will be aligned with their additional extensions
when they are finalized. Future STAC API releases will align with [upcoming versions](https://github.com/opengeospatial/ogcapi-features).
STAC API adds certain requirements and extensions on top of the OAFeat core, to meet its community's use cases, detailed below.

The OGC API - Features is a standard API that represents collections of geospatial data. It defines the RESTful interface
to query geospatial data, with GeoJSON as a main return type. With OAFeat you can return any `Feature`, which is a geometry
plus any number of properties. The core [STAC Item spec](https://github.com/radiantearth/stac-spec/item-spec/README.md)
enhances the core `Feature` with additional requirements and options to enable cataloging of spatiotemporal 'assets' like
satellite imagery. This STAC `Item` always links to an asset, and these assets always have a capture time, so it requires
fields for `datetime` and `assets`. The STAC API extends the OAFeat core with some key functionality to enable search of
geospatial assets, detailed below.
fields for `datetime` and `assets`.

OAFeat also defines the concept of a Collection, which contains Features. In OAFeat Collections are the sets of data that can
be queried ([7.11](http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#_collections_)), and each describes basic
Expand All @@ -21,15 +31,19 @@ information, along with other STAC specific fields to provide additional metadat
thus are compliant with both OAFeat Collections and STAC Collections and are returned from the `/collections/{collection_id}`
endpoint.

In OAFeat Features are the individual records within a Collection and are provided in GeoJSON format.
[STAC Items](https://github.com/radiantearth/stac-spec/item-spec/README.md) are analogous to OAFeat Features,
are in GeoJSON, and are returned from the `/collections/{collection_id}/items/{item_id}` endpoint.
In OAFeat Features are the individual records within a Collection and are usually provided in GeoJSON format.
[STAC Items](https://github.com/radiantearth/stac-spec/item-spec/README.md) are compliant with the OAFeat Features [GeoJSON
requirements class](http://docs.ogc.org/is/17-069r3/17-069r3.html#_requirements_class_geojson), and are returned from the
`/collections/{collection_id}/items/{item_id}` endpoint. The return of other encodings
matthewhanson marked this conversation as resolved.
Show resolved Hide resolved
([html](http://docs.ogc.org/is/17-069r3/17-069r3.html#rc_html), [gml](http://docs.ogc.org/is/17-069r3/17-069r3.html#rc_gmlsf0))
is outside the scope of STAC API, as the [STAC Item](https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md) is
specified in GeoJSON.

A typical OAFeat will have multiple collections, and each will just offer simple search for its particular collection at
`GET /collections/{collectionId}/items`.
The main STAC endpoint specified beyond what OAFeat offers is `/search`, which offers cross-collection search. A primary
use case of STAC is to search diverse imagery collections (like Landsat, Sentinel, MODIS) by location and cloud cover for any
relevant image. So the ability to do searches across Collections is required, and is not yet specified by OAFeat. Due to the
relevant image. So the ability to do searches across Collections is very important, and is not yet specified by OAFeat. Due to the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"for any relevant image" -> "for any matching Items."

"not yet specified by OAFeat" suggests this is an upcoming capability, we should avoid making assumptions about the future.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They have identified it as future work (in opengeospatial/ogcapi-features#451), and reference it in Filter/CQL - http://docs.opengeospatial.org/DRAFTS/19-079.html#filter-param-multiple-collections

cholmes marked this conversation as resolved.
Show resolved Hide resolved
limited parameter support in OGC API - Features, it is recommended to use the STAC API endpoint
`POST /search` for advanced queries, as it supports richer options.
matthewhanson marked this conversation as resolved.
Show resolved Hide resolved
The filtering is made to be compatible between STAC API and OGC API - Features whenever feasible, and the two specs seek
Expand All @@ -40,6 +54,23 @@ Implementations may **optionally** provide support for the full superset of STAC
where the collection ID in the path is equivalent to providing that single value in the `collections` query parameter in
STAC API.

### STAC API Compliance
Copy link
Collaborator

@m-mohr m-mohr Nov 13, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to expose only STAC Collections in an API? This would make 10+ implementations in openEO not STAC API compliant any more!

In general, I changed my mind on this since commenting in #42. I think we should only require the landing page and from there just look what has been provided in the links. The issue is that clients will now expect that all those endpoints are accessible, but for (partly) private APIs this is not the case. For example, Radiant Earth has only /, /collections and /collections/:id publicly available and then the others need authentication. So by just pointing a client without credentials to this API, it will fail on all requests that require authentication, which is the same as just not having them implemented (like in openEO). If clients follow the RESTful approach, they just follow links and don't necessarily need any pre-defined endpoint to exist (except for a landing page).

Also, I think there's a mismatch in this PR: Either its RESTful as specified in the introduction (and thus only links are required to navigate the API, no specific endpoints required other than a landing page) OR we require certain endpoints, but then it's not strictly RESTful any longer IMHO.

If we stick with this requirement, it must be mentioned in the Changelog as this is a major change compared to 0.9.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open on what exactly we do here, was just trying to capture what was in the PR.

My thinking was that a Collection only endpoint is actually not a STAC API. In my mind a STAC API is when you provide 'search'. OpenEO is a compliant STAC catalog, as is https://storage.googleapis.com/pdd-stac/disasters-0.9.0/catalog.json And I'd see implementations that are servers (RESTful API's) that only implement STAC core, and don't offer search/ or collections/items/ endpoints.

I think I agree with your point about RESTful, with no specific endpoints required, but we should be able to still specify functionality required in some way? Is OGC Features API RESTful?

Copy link
Collaborator

@m-mohr m-mohr Nov 13, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenEO is a compliant STAC catalog, as is https://storage.googleapis.com/pdd-stac/disasters-0.9.0/catalog.json

Unfortunately, it is not. We use the data rel type instead of child as it is specified in STAC API. So it's actually not a valid STAC catalog... And then /collections is also not valid STA, it's only valid in STAC API.

I think I agree with your point about RESTful, with no specific endpoints required, but we should be able to still specify functionality required in some way?

Why would that be required? If we have conformance classes anyway, couldn't we just specify what is implemented there or just follow links? I'm not sure myself yet, but I don't like that it's obviously getting towards just STAC API for Items and it seems I need to come up with my own STAC API spec for Collections. I mean it's a bit an issue of course that OGC API Features requires the item endpoints, too, and as such I'm basically only compliant to OGC API Commons, but then have STAC Collections of course. Anyway, we can't represent all static STACs in the API it seems...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, that's interesting on using data rel type instead of child, I'd thought they were child. I do think that's a confusing little area.

I'm not set on not having STAC API just for items, just saying that was more of my mental model. That everyone should implement STAC Core, and that you layer on Features API / Search endpoint for filtering items. And the latter is 'STAC API'.

But for beta.1 I prefer to not rock the boat - mostly just want to align with CQL / Transactions, clean things up, and get something for people to work with. I definitely see a beta.2, and I think the recommendations for how to implement STAC catalogs / collections is core to figure out, and this to me fits in. I think the conformance classes are probably the way to go. And I'm not set on my mental model, I can see value in having 'STAC API' specify how to implement Collections / Catalogs without search. But I also like that STAC Core can be implemented by a dynamic server, but that we don't need to specify that.

Anyway, we can't represent all static STACs in the API it seems...

What does this mean?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, that's interesting on using data rel type instead of child, I'd thought they were child. I do think that's a confusing little area.

It makes sense, because it returns you a different JSON. Not a collection/catalog, but the response of /collections, which contains collections and links. /collections then lists the "children" instead of adding (in our case) 450+ links to the links in the landing page. It's just that this is not specified in STAC itself, but only in API (originating from Features).

I think the conformance classes are probably the way to go

Agreed, see #27.

I can see value in having 'STAC API' specify how to implement Collections / Catalogs without search.

Actually, I'd rather say we should have collection search at some point and then there might be no "without search" anylonger.

Anyway, we can't represent all static STACs in the API it seems...

What does this mean?

If we merge this PR, you need Items in collections/catalogs, otherwise the API is not compliant although it's a valid STAC.

Copy link
Collaborator

@m-mohr m-mohr Nov 18, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, my statement about adding child links to root is a bit biased by openEO and is also weakened by the discussion on Monday. I don't think there's a major issue with your implementation. child links at the landing page are fine, BUT I'd recommend to put them in /collections and link to it with rel type data if OAFeat is implemented anyway. Also, in an API that is not necessarily only STAC (as openEO is), it would bloat the initial request with data that is not required at that stage and thus connection times can slow down. In an API just covering STAC, I'd argue it's not a major problem as it's the primary goal/content of the API.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd just remove the whole new chapter for now, merge this and then write-up the conformance classes. Those will basically change the whole chapter anyway.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main interest is in having the API browsable by STAC browser, so I think that will need an update to respect the data rel type link.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that was my plan - remove the whole chapter on compliance, and then redo it with the conformance PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewhanson I implemented the data rel type some days ago into STAC Browser: radiantearth/stac-browser#59 Already live on stacindex.org...


To be a compliant STAC API an implementation must either:

1) Fully implement [OGC API - Features](http://docs.opengeospatial.org/is/17-069r3/17-069r3.html) core, and have one or more
[Feature Collections](http://docs.ogc.org/is/17-069r3/17-069r3.html#_collections_) implement STAC
[Collections](https://github.com/radiantearth/stac-spec/blob/master/collection-spec/collection-spec.md), with its returned
[Features](http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#_items_) as valid STAC
[Items](https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md).
2) Implement the [STAC API Endpoints](#stac-api-endpoints), with `/` returning valid STAC
[Catalogs](https://github.com/radiantearth/stac-spec/blob/master/catalog-spec/catalog-spec.md) and `/search` returning valid
[Items](https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md).

The recommendation is to implement both, with `/search` providing cross-collection search and then having each
`/collections/{collectionId}` describe a homogeneous set of STAC Items, but only one of the two is required to be a
valid STAC API implementation.

## HTTP Request Methods and Content Types

The OAFeat and STAC APIs follow a RESTful model. A core principal of this is the use of HTTP Request Methods ("verbs") and
Expand Down