From 99aea7b4a2930141934fa9242f36bb200ac089b4 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Fri, 30 Apr 2021 16:53:39 -0700 Subject: [PATCH 01/46] Checkpoint work on eventing specs --- specs/eventing/data-plane.md | 112 +++++++++++++++++++++++++++++++++-- specs/eventing/motivation.md | 56 ++++++------------ specs/eventing/overview.md | 93 ++++++++++++++++++++++++++++- specs/serving/README.md | 4 +- 4 files changed, 219 insertions(+), 46 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 81e1aa3fc..f37b615e7 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -1,17 +1,117 @@ -# Knative Eventing Data Plane Contracts +# Knative Eventing Data Plane Contract ## Introduction -Developers using Knative Eventing need to know what is supported for delivery to -user provided components that receive events. Knative Eventing defines contract -for data plane components and we have listed them here. - -## Conformance +Late-binding event senders and receivers (composing applications using +configuration) only works when all event senders and recipients speak a common +protocol. In order to enable wide support for senders and receivers, Knative +Eventing extends the [CloudEvents HTTP +bindings](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) +with additional semantics for the following reasons: + +- Knative Eventing aims to enable highly-reliable event processing workflows. As + such, it prefers duplicate delivery to discarded events. The CloudEvents spec + does not take a stance here. + +- The CloudEvents HTTP bindings provide a relatively simple and efficient + network protocol which can easily be supported in a wide variety of + programming languages leveraging existing library investments in HTTP. + +- Knative Eventing assumes a sender-driven (push) event delivery system. That + is, each event processor is actively responsible for an event until it is + handled (or affirmatively delivered to all following recipients). + +- Knative Eventing aims to make writing [event + sources](./overview.md#event-source) and event-processing software easier to + write; as such, it imposes higher standards on system components like + [brokers](./overview.md#broker) and [channels](./overview.md#channel) than on + edge components. + +This contract defines a mechanism for a single event sender to reliably deliver +a single event to a single recipient. Building from this primitive, chains of +reliable event delivery and event-driven applications can be built. + +## Background The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119. +When not specified in this document, the [CloudEvents HTTP bindings, version +1](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) and +[HTTP 1.1 protocol](https://tools.ietf.org/html/rfc7230) standards should be +followed (with the CloudEvents bindings preferred in the case of conflict). + +The current version of this document does not describe protocol negotiation or +the ability to upgrade an HTTP 1.1 event delivery into a more efficient protocol +such as GRPC, AMQP or the like. It also aims not to preclude such a protocol +negotiation in future versions of the specification. + + +## Event Delivery + +To provide simpler support for event sources which may be translating events +from existing systems, some data plane requirements for senders are relaxed in +the general case. In the case of Knative Eventing provided resources (Channels +and Brokers) which implement these roles, requirements may be increased from +SHOULD to MUST. These cases are called out as they occur. + +### Minimum supported protocol + +All senders and recipients MUST support the CloudEvents 1.0 protocol and the +[binary](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#31-binary-content-mode) +and +[structured](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#32-structured-content-mode) +content modes of the CloudEvetns HTTP binding. Senders MUST support both +cleartext (`http`) and TLS (`https`) URLs as event delivery destinations. + +### HTTP Verbs + +In the absence of specific delivery preferences, the sender MUST initiate +delivery of the event to the recipient using the HTTP POST verb, using either +the structured or binary encoding of the event (sender's choice). This delivery +SHOULD be performed using the CloudEvents HTTP Binding, version 1. + +Senders MAY probe the recipient with an [HTTP OPTIONS +request](https://tools.ietf.org/html/rfc7231#section-4.3.7); if implemented, the +recipent MUST indicate support the POST verb using the [`Allow` +header](https://tools.ietf.org/html/rfc7231#section-7.4.1). Senders which +receive an error SHOULD proceed using the HTTP POST mechanism. + +### Event Acknowledgement and Repeat Delivery + +Event recipients MUST use the HTTP response code to indicate acceptance of an +event. The recipient MUST NOT return a response accepting the event until it has +handled event (processed the event or stored in stable storage). The following +response codes are explicitly defined; event recipients MAY also respond with +other response codes. A response code not in this table SHOULD be treated as an +error. + +| Response code | Meaning | Retry | +| ------------- | --------------------------- | ----- | +| 200 | [Event Reply](#event-reply) | No | +| 202 | Event accepted | No | +| 400 | Unparsable event | No | +| 404 | Endpoint does not exist | No | + +Recipients MUST be able to handle duplicate delivery of events and MUST accept +delivery of duplicate events, as the event acknowledgement could have been lost in +transit to the sender. + + +### Observability + + +### Derived (Reply) Events + + + + +================================================================ + CUT BELOW +================================================================ + + ## Data plane contract for Sinks A **Sink** MUST be able to handle duplicate events. diff --git a/specs/eventing/motivation.md b/specs/eventing/motivation.md index 9d2b48653..e807397be 100644 --- a/specs/eventing/motivation.md +++ b/specs/eventing/motivation.md @@ -1,49 +1,31 @@ # Motivation -The goal of Knative Eventing is to define common, composable primitives to -enable late-binding event sources and event consumers. +The goal of the Knative Eventing project is to define common primitives to +enable composing event-processing applications through configuration, rather +than application code. - +Building by combining independent components provides a number of benefits for +application designers: -Knative Eventing has following principles: - -1. Services are loosely coupled during development and deployed independently on - a variety of platforms (Kubernetes, VMs, SaaS or FaaS). +1. Services are loosely coupled during development and may be deployed + independently on a variety of platforms (Kubernetes, VMs, SaaS or FaaS). This + composability allows re-use of common patterns and building blocks, even + across programming language and tooling boundaries. 1. A producer can generate events before a consumer is listening, and a consumer can express an interest in an event or class of events that is not yet being - produced. + produced. This allows event-driven applications to be evolve over time + without needing to closely coordinate changes. -1. Services can be connected to create new applications - - without modifying producer or consumer. +1. Services can be connected to create new applications: + - without modifying producer or consumer - with the ability to select a specific subset of events from a particular producer -These primitives enable producing and consuming events adhering to the -[CloudEvents Specification](https://github.com/cloudevents/spec), in a decoupled -way. - -Kubernetes has no primitives related to event processing, yet this is an -essential component in serverless workloads. Eventing introduces high-level -primitives for event production and delivery with an initial focus on push over -HTTP. If a new event source or type is required of your application, the effort -required to plumb them into the existing eventing framework will be minimal and -will integrate with CloudEvents middleware and message consumers. - -Knative eventing implements common components of an event delivery ecosystem: -enumeration and discovery of event sources, configuration and management of -event transport, and declarative binding of events (generated either by storage -services or earlier computation) to further event processing and persistence. - -The Knative Eventing API is intended to operate independently, and interoperate -well with the [Serving API](https://github.com/knative/serving) and -[Build API](https://github.com/knative/build). - ---- - -_Navigation_: +In order to enable loose coupling and late-binding of event producers and +consumers, Knative Eventing utilizes and extends the [CloudEvents +specification](https:?/github.com/cloudevents/spec) as the data plane protocol +between components. -- **Motivation and goals** -- [Resource type overview](overview.md) -- [Interface contracts](interfaces.md) -- [Object model specification](spec.md) +Knative Eventing also defines patterns to simplify the construction and usage of +event senders and recipients. diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index 3fafca4d4..720da6d4e 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -1,12 +1,103 @@ # Resource Types +The Knative Eventing API provides primitives for two common event-processing +patterns (credit to James Urquhart for the formulation): + +* Point-to-point asynchronous communication ([`messaging.knative.dev`](#messaging)) + +* Content-based event routing ([`eventing.knative.dev`](#eventing)) + +The other two patterns James identifies are log-stream processing and complex +workflows; these are not currently addressed by Knative Eventing. + +In addition to the primitives needed to express the above patterns, Knative +Eventing defines two [_interface contracts_](#interface-contracts) to allow +connecting multiple types of Kubernetes objects as event sernders and recipients +to the core primitives. + + + +## Eventing + +### Broker + +**Broker** provides a central event-routing hub which exposes a URL address +which event senders may use to submit events to the router. A Broker may be +implemented using many different underlying event-forwarding mechanisms; the +broker provides a small set of common event-delivery configuration options and +may reference additional implementation-specific configuration options via a +reference to an external object; the format of the external objects is not +standardized. + +### Trigger + +**Trigger** defines a filtered delivery option to extract events delivered to a +Broker** and route them to an **Addressable** destination. Trigger implements +uniform event filtering based on the CloudEvents attributes associated with the +event, ignoring the payload (which might be large and/or binary and need not be +parsed during event routing). The addressable interface contract allows Triggers +to deliver events to a variety of different destinations, including external +resources such as a virtual machine or SaaS service. + +## Messaging + +### Channel + +**Channel** provides an abstract interface which may be fulfiled by several +concrete implementations of a backing asynchronous fan-out queue. The common +abstraction provided by channel allows both the composition of higher-level +constructed for chained or parallel processing of events, and the replacement of +particular messaging technologies (for example, allowing a development +environment to use a lower-reliability channel compared with the production +environment). + +### Subscription + +**Subscription** defines a delivery destination for all events sent to a +**Channel**. Events sent to a channel are delivered to _each_ subscription +_independently_ -- a subscription maintains its own list of undelivered events +and will manage retry indpendently of any other subscriptions to the same +channel. Like **Trigger**, subscriptions use the **Addressable** interface +contract to support event delivery to many different destination types. + +## Interface Contracts + +In addition to the concrete types described above in the `messaging.knative.dev` +and `eventing.knative.dev` API groups, Knative Eventing supports referencing +objects in other API groups as destinations for event delivery. This is done by +defining partial schemas which the other resources must support. The following +interface contracts define a set of expected resource fields on an referenced +resource. + +### Addressable + +**Addressable** resources expose a resource address (HTTP URL) in their `status` +object. The URL is used as a destination for delivery of events to the resource; +the exposed URL must implement the [data plane contract](data-plane.md) for +receiving events. + +**Broker** and **Channel** both implement **Addressable**. + +### Event Source + +**Event Sources** are resources which generate events and may be configured to +deliver the events to an **Addressable** resource designated by a `sink` object +in the resource's `spec`. The Knative Eventing spec does not define any specific +event sources, but does define common interfaces for discovering and managing +event sources. + +================================================================ +CUT HERE +================================================================ + + The API defines and provides a complete implementation for [Trigger](spec.md#kind-trigger), [Broker](spec.md#kind-broker), [Subscription](spec.md#kind-subscription) and abstract resource definitions for [Channels](spec.md#kind-channel). With extensibility and composability as a goal of Knative Eventing, the eventing -API defines several resources that can be reduced down to well understood +API defines several resources that can be reduced down to well understood contracts. These eventing resource interfaces may be fulfilled by other Kubernetes objects and then composed in the same way as the concrete objects. The interfaces are ([Addressable](interfaces.md#addressable), diff --git a/specs/serving/README.md b/specs/serving/README.md index f37f2e68a..d1ab6e0c1 100644 --- a/specs/serving/README.md +++ b/specs/serving/README.md @@ -7,7 +7,7 @@ document locations: Docs in this directory: -- [Motivation and goals](motivation.md) -- [Resource type overview](overview.md) +- [Motivation and goals](motivation.md) - non-normative +- [Resource type overview](overview.md) - non-normative - [Knative Serving API Specification](knative-api-specification-1.0.md) - [Knative Serving Runtime Specification](runtime-contract.md) From e07658dd753e7b645c15e59e25354848b3eb91fb Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Fri, 30 Apr 2021 23:11:32 -0700 Subject: [PATCH 02/46] More work on data plane & control plane. Mostly need to finish control plane now. --- specs/eventing/control-plane.md | 299 ++++++++++++++++++++++++++++++++ specs/eventing/data-plane.md | 91 ++++++++-- 2 files changed, 375 insertions(+), 15 deletions(-) create mode 100644 specs/eventing/control-plane.md diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md new file mode 100644 index 000000000..f6dd6692e --- /dev/null +++ b/specs/eventing/control-plane.md @@ -0,0 +1,299 @@ +# Abstract + +The Knative Eventing platform provides common primitives for routing CloudEvents +between cooperating HTTP clients. This document describes the structure, +lifecycle, and management of Knative Eventing resources in the context of the +[Kubernetes Resource +Model](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/resource-management.md). An +understanding of the Kubernetes API interface and the capabilities of +[Kubernetes Custom +Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) is assumed. + +This document does not define the [data plane event delivery +contract](./data-plane.md) (though it does describe how event delivery is +configured). This document also does not prescribe specific implementations of +supporting services such as access control, observability, or resource +management. + +# Background + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" are to be +interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119). + +There is no formal specification of the Kubernetes API and Resource Model. This +document assumes Kubernetes 1.21 behavior; this behavior will typically be +supported by many future Kubernetes versions. Additionally, this document may +reference specific core Kubernetes resources; these references may be +illustrative (i.e. _an implementation on Kubernetes_) or descriptive (i.e. _this +Kubernetes resource MUST be exposed_). References to these core Kubernetes +resources will be annotated as either illustrative or descriptive. + +This document considers two users of a given Knative Eventing environment, and is +particularly concerned with the expectations of developers (and language and +tooling developers, by extension) deploying applications to the environment. + +- **Developers** configure Knative resources to implement an event-routing + architecture. +- **Operators** (also known as **platform providers**) provision the underlying + event routing resources and manage the software configuration of Knative + Eventing and the underlying abstractions. + +# RBAC Profile + +In order to validate the controls described in [Resource +Overview](#resource-overview), the following Kubernetes RBAC profile may be +applied in a Kubernetes cluster. This Kubernetes RBAC is an illustrative example +of the minimal profile rather than a requirement. This Role should be sufficient +to develop, deploy, and manage event routing for an application within a single +namespace. Knative Conformance tests against "MUST", "MUST NOT", and "REQUIRED" +conditions are expected to pass when using this profile: + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: knative-developer +rules: + - apiGroups: ["eventing.knative.dev"] + resources: ["broker", "trigger"] + verbs: ["get", "list", "create", "update", "delete"] + - apiGroups: ["messaging.knative.dev"] + resources: ["channel", "subscription"] + verbs: ["get", "list", "create", "update", "delete"] +``` + + + +# Resource Overview + +The Knative Eventing API provides a set of primitives to support both +point-to-point communication channels (`messaging.knative.dev`) and +content-based event routing (`eventing.knative.dev`). This specification +describes API interfaces of Knative Eventing resources as well as the supported +[event routing](./data-plane.md) logic and configuration settings. + +At the moment, the Knative Eventing specification does not contemplate any +non-Kubernetes-backed implementations, and therefore does not specifically +define the mapping of kubernetes verbs (read, watch, patch, etc) to developer +roles. See the [Overview documentation](./overview.md) for general definitions +of the different API objects. + +# Error Signalling + + +The Knative API uses the +[Kubernetes Conditions convention](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) +to communicate errors and problems to the user. Each user-visible resource +described in Resource Overview MUST have a `conditions` field in `status`, which +must be a list of `Condition` objects of the following form (note that the +actual API object types may be named `FooCondition` to allow better code +generation and disambiguation between similar fields in the same `apiGroup`): + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field + Type + Description + Default Value +
type + string + The category of the condition, as a short, CamelCase word or phrase. +

+This is the primary key of the Conditions list when viewed as a map. +

REQUIRED – No default +
status + Enum:
    + +
  • "True" +
  • "False" +
  • "Unknown"
+ + 		The last measured status of this condition. + "Unknown" +
reason + string + One-word CamelCase reason for the condition's last transition. + "" +
message + string + Human-readable sentence describing the last transition. + "" +
severity + Enum:
    + +
  • "" +
  • "Warning" +
  • "Info"
+ + 		If present, represents the severity of the condition. An empty severity represents a severity level of "Error". + "" +
lastTransitionTime + Timestamp + Last update time for this condition. + "" – may be unset +
+ +Additionally, the resource's `status.conditions` field MUST be managed as +follows to enable clients (particularly user interfaces) to present useful +diagnostic and error message to the user. In the following section, conditions +are referred to by their `type` (aka the string value of the `type` field on the +Condition). + +1. Each resource MUST have either a `Ready` condition (for ongoing systems) or + `Succeeded` condition (for resources that run to completion) with + `severity=""`, which MUST use the `True`, `False`, and `Unknown` status + values as follows: + + 1. `False` MUST indicate a failure condition. + 1. `Unknown` SHOULD indicate that reconciliation is not yet complete and + success or failure is not yet determined. + 1. `True` SHOULD indicate that the application is fully reconciled and + operating correctly. + + `Unknown` and `True` are specified as SHOULD rather than MUST requirements + because there may be errors which prevent serving which cannot be determined + by the API stack (e.g. DNS record configuration in certain environments). + Implementations are expected to treat these as "MUST" for factors within the + control of the implementation. + +1. For non-`Ready` conditions, any conditions with `severity=""` (aka "Error + conditions") must be aggregated into the "Ready" condition as follows: + + 1. If the condition is `False`, `Ready` MUST be `False`. + 1. If the condition is `Unknown`, `Ready` MUST be `False` or `Unknown`. + 1. If the condition is `True`, `Ready` may be any of `True`, `False`, or + `Unknown`. + + Implementations MAY choose to report that `Ready` is `False` or `Unknown` + even if all Error conditions report a status of `True` (i.e. there may be + additional hidden implementation conditions which feed into the `Ready` + condition which are not reported.) + +1. Non-`Ready` conditions with non-error severity MAY be surfaced by the + implementation. Examples of `Warning` or `Info` conditions could include: + missing health check definitions, scale-to-zero status, or non-fatal + capacity limits. + +Conditions type names should be chosen to describe positive conditions where +`True` means that the condition has been satisfied. Some conditions may be +transient (for example, `ResourcesAllocated` might change between `True` and +`False` as an application scales to and from zero). It is RECOMMENDED that +transient conditions be indicated with a `severity="Info"`. + +# Resource Lifecycle + +## Broker + +TODO: lifecycle; are triggers deleted when the broker is deleted? Are trigger conditions reflected in the broker? What are the required conditions (Ready, ???)? `spec.class` SHOULD be immutable. + +## Trigger + +TODO: lifecycle; what happens if a trigger is created before a broker? What about events received by a broker before the trigger is created? Are broker conditions reflected in the trigger? What are the required conditions (Ready, ???)? `spec.broker` SHOULD be immutable. + +## Channel + +TODO: lifecycle; are subscriptions deleted when the channel is deleted? Are subscription conditions reflected in the channel? What are the required conditions (Ready, ???)? + +TODO: channel-compatible CRDs (Channelable) + +## Subscription + +TODO: lifecycle; what happens if a subscription is created before a channel? What about events received by a channel before the subscription is created? Are channel conditions reflected in the subscription? What are the required conditions (Ready, ???)? `spec.channel` SHOULD be immutable. + +## Addressable resolution + +TODO: What is it? How does it apply to Trigger / Subscription? Indicate that Broker & Channel MUST implement Addressable. + +# Event Routing + +## Content Based Routing + +TODO: How do Broker & Trigger handle routing. +- When must events arriving at a Broker be routed by a Trigger (when the Trigger is Ready?) + - How do retries and replies interact with configuration changes in the Trigger / Broker? +- Retry parameters +- Reply routing +- Dead-letter routing + +## Topology Based Routing + +TODO: How do Channel & Subscription handle routing +- When must events arriving at a Channel be routed to a Subscription (when the Subscription is Ready?) + - How do retries and replies interact with configuration changes in the Subscription / Channel? +- Retry parameters +- Reply routing +- Dead-letter routing + +## Event Sources + +TODO: What's required of an event source with respect to routing? Retries? Dead-letter? Status? + +# Detailed Resources + +## Broker v1 + +## Trigger v1 + +## Channel v1 + +## Subscription v1 + +## Addressable v1 + +================================================================ + CUT HERE +================================================================ + +## Broker + +The Knative Broker represents a single instance of an event router which accepts +events from one or more sources and routes them to selected destinations based +on rules matching the attributes of the received event. In order to do this, the Broker defines diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index f37b615e7..63a1f2c42 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -44,9 +44,8 @@ followed (with the CloudEvents bindings preferred in the case of conflict). The current version of this document does not describe protocol negotiation or the ability to upgrade an HTTP 1.1 event delivery into a more efficient protocol -such as GRPC, AMQP or the like. It also aims not to preclude such a protocol -negotiation in future versions of the specification. - +such as GRPC, AMQP or the like. It is expected that a future compatible version +of this specification may describe a protocol negotiation mechanism. ## Event Delivery @@ -84,26 +83,88 @@ Event recipients MUST use the HTTP response code to indicate acceptance of an event. The recipient MUST NOT return a response accepting the event until it has handled event (processed the event or stored in stable storage). The following response codes are explicitly defined; event recipients MAY also respond with -other response codes. A response code not in this table SHOULD be treated as an -error. - -| Response code | Meaning | Retry | -| ------------- | --------------------------- | ----- | -| 200 | [Event Reply](#event-reply) | No | -| 202 | Event accepted | No | -| 400 | Unparsable event | No | -| 404 | Endpoint does not exist | No | +other response codes. A response code not in this table SHOULD be treated as a +retriable error. + +| Response code | Meaning | Retry | Delivery completed | Error | +| ------------- | --------------------------- | ----- | ------------------ | ----- | +| `1xx` | (Unspecified) | Yes\* | No\* | Yes\* | +| `200` | [Event reply](#event-reply) | No | Yes | No | +| `202` | Event accepted | No | Yes | No | +| other `2xx` | (Unspecified) | Yes\* | No\* | Yes\* | +| other `3xx` | (Unspecified) | Yes\* | No\* | Yes\* | +| `400` | Unparsable event | No | No | Yes | +| `404` | Endpoint does not exist | No | No | Yes | +| other `4xx` | Error | Yes | No | Yes | +| other `5xx` | Error | Yes | No | Yes | + +\* `1xx`, `2xx`, and `3xx` response codes are **reserved for future +extension**. Event recipients SHOULD NOT send these response codes in this spec +version, but event senders MUST handle these response codes as errors and +implement appropriate failure behavior. + + Recipients MUST be able to handle duplicate delivery of events and MUST accept -delivery of duplicate events, as the event acknowledgement could have been lost in -transit to the sender. - +delivery of duplicate events, as the event acknowledgement could have been lost +in transit to the sender. Event recipients MUST use the [`source` and `id` +attributes](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#required-attributes) +to detect duplicated events (see [observability](#observability) for an example +case where other event attributes may vary from one delivery attempt to +another). + +Where possible, event senders SHOULD re-attempt delivery of events where the +HTTP request failed or returned a retriable status code. It is RECOMMENDED that +event senders implement some form of congestion control (such as exponential +backoff) when managing retry timing. This specification does not document any +specific congestion control algorithm or +parameters. [Brokers](./overview.md#broker) and +[Channels](./overview.md#channel) MUST implement congestion control and MUST +implement retries. ### Observability +Event senders MAY add or update CloudEvents attributes before sending to +implement observability features such as tracing; in particular, the +[`traceparent` and `tracestate` distributed tracing +attributes](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md) +may be modified in this way for each delivery attempt of the same event. + +This specification does not mandate any particular logging or metrics +aggregtion, nor a method of exposing observability information to users +configuring the resources. Platform administrators SHOULD expose event-delivery +telemetry to users through platform-specific interfaces, but such interfaces are +beyond the scope of this document. + + ### Derived (Reply) Events +In some applications, an event receiver might emit an event in reaction to a +received event. An event sender MAY document support for this pattern by +including a `Prefer: reply` header in the HTTP POST request. This header +indicates to the event receiver that the caller will accept a [`200` +response](#event-acknowledgement-and-repeat-delivery) which includes a +CloudEvent encoded using the binary or structured formats. + +The sender SHOULD NOT assume that a received reply event is directly related to +the event sent in the HTTP request. + +A recipient may reply to any HTTP POST with a `200` response to indicate that +the event was processed successfully, with or without a response payload. If the +recipient will _never_ provide a response payload, the `202` response code is +likely a better choice. + +If a recipient chooses to reply to a sender with a `200` response code and a +reply event in the absence of a `Prefer: reply` header, the sender SHOULD treat +the event as accepted, and MAY log an error about the unexpected payload. The +sender MUST NOT process the reply event if it did not advertise the `Prefer: +reply` capability. From 70d7e8b510ff75a8c033a27abdfc072341768a0e Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Mon, 3 May 2021 15:02:15 -0700 Subject: [PATCH 03/46] Add at-least-once detail on data plane. --- specs/eventing/motivation.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/specs/eventing/motivation.md b/specs/eventing/motivation.md index e807397be..42492f8b5 100644 --- a/specs/eventing/motivation.md +++ b/specs/eventing/motivation.md @@ -25,7 +25,9 @@ application designers: In order to enable loose coupling and late-binding of event producers and consumers, Knative Eventing utilizes and extends the [CloudEvents specification](https:?/github.com/cloudevents/spec) as the data plane protocol -between components. +between components. Knative Eventing prioritizes at-least-once delivery +semantics, using the CloudEvents HTTP POST (push) transport as a minimum common +transport between components. Knative Eventing also defines patterns to simplify the construction and usage of event senders and recipients. From dc743ca32a6c3e98f86b5ad2124cee2b2563ed20 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Tue, 4 May 2021 22:29:19 -0700 Subject: [PATCH 04/46] Finish off most of lifecycle --- specs/eventing/control-plane.md | 149 ++++++++++++++++++++++++++++---- 1 file changed, 133 insertions(+), 16 deletions(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index f6dd6692e..8ac15b9e4 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -230,32 +230,145 @@ transient conditions be indicated with a `severity="Info"`. # Resource Lifecycle -## Broker - -TODO: lifecycle; are triggers deleted when the broker is deleted? Are trigger conditions reflected in the broker? What are the required conditions (Ready, ???)? `spec.class` SHOULD be immutable. - -## Trigger - -TODO: lifecycle; what happens if a trigger is created before a broker? What about events received by a broker before the trigger is created? Are broker conditions reflected in the trigger? What are the required conditions (Ready, ???)? `spec.broker` SHOULD be immutable. - -## Channel - -TODO: lifecycle; are subscriptions deleted when the channel is deleted? Are subscription conditions reflected in the channel? What are the required conditions (Ready, ???)? - +## Trigger Lifecycle + +The lifecycle of a Trigger is independent of that of the Broker it refers to in +its `spec.broker` field; if the Broker does not currently exist or its `Ready` +condition is not `true`, then the Trigger's `Ready` condition MUST NOT be +`true`, and the reason SHOULD indicate that the corresponding Broker is missing +or not ready. Similarly, if the Trigger's `spec.subscriber` field contains a +`ref` to another resource, the Trigger's `Ready` condition MUST NOT be `true` if +the `spec.subscriber.ref` field points to a resource which does not exist or +which does not have a `status.address.url` field. + +Once created, the Trigger's `spec.broker` SHOULD NOT permit updates; to change +the `spec.broker`, the Trigger can be deleted and re-created. This pattern is +chosen to make it clear that changing the `spec.broker` is not an atomic +operation, as it may span multiple storage systems. Changes to +`spec.subscriber`, `spec.filter` and other fields SHOULD be permitted, as these +could occur within a single storage system. + +When a Trigger becomes associated with a Broker (either due to creating the +Trigger or the Broker), the Trigger MUST only set the `Ready` condition to `true` +after the Broker has been configured to send all future events matching the +`spec.filter` to the Trigger's `spec.subscriber`. The Broker MAY send some +events to the Trigger`s `spec.subscriber` prior to the Trigger's `Ready` condition +being set to `true`. When a Trigger is deleted, the Broker MAY send some +additional events to the Trigger`s `spec.subscriber`. + +## Broker Lifecycle + +A Broker represents an Addressable endpoint (i.e. it has a `status.address.url` +field) which can receive, store, and forward events to multiple recipients based +on a set of attribute filters (Triggers). Triggers are associated with a Broker +based on the `spec.broker` field on the Trigger; it is expected that the +controller for a Broker will also control the associated Triggers. When the +Broker's `Ready` condition is `true`, the Broker MUST provide a +`status.address.url` which accepts all CloudEvents and MUST forward the received +events to each associated Trigger whose `Ready` condition is `true`. As +described in the [Trigger Lifecycle](#trigger-lifecycle) section, a Broker MAY +forward events to an associated Trigger which which does not currently have a +`true` `Ready` condition, including events received by the Broker before the +Trigger was created. + +When a Broker is created, its `spec.class` field MUST be populated to indicate +which of several possible Broker implementations to use. It is RECOMMENDED to +default the `spec.class` field on creation if it is unpopulated. Once created, +the `spec.class` field MUST be immutable; the Broker must be deleted and +re-created to change the `spec.class`. This pattern is chosen to make it clear +that changing `spec.class` is not an atomic operation and that any +implementation would be likely to result in message loss during the transition. + +## Subscription Lifecycle + +The lifecycle of a Subscription is independent of that of the Channel it refers +to in its `spec.channel` field. The `spec.channel` object reference may refer to +either an `eventing.knative.dev/v1` Channel resource, or another resource which +meets the `spec.subscribers` and `spec.delivery` required elements in the +Channellable duck type. If the referenced `spec.channel` does not currently +exist or its `Ready` condition is not `true`, then the Subscription's `Ready` +condition MUST NOT be `true`, and the reason SHOULD indicate that the +corresponding Channel is missing or not ready. Similarly, if the Subscriptions +`spec.subscriber`, `spec.reply` or `spec.delivery.deadLetterSink` fields contain +a `ref` to another Resource, the Subscriptions `Ready` condition MUST NOT be +`true` unless the referred-to resources exist and contain a `status.address.url` +field. (It is acceptable for none of the `spec.subscriber`, `spec.reply`, and +`spec.delivery.deadLetterSink` fields to contain a `ref` field.) + +Once created, the Subscription's `spec.channel` SHOULD NOT permit updates; to +change the `spec.channel`, the Subscription can be deleted and re-created. This +pattern is chosen to make it clear that changing the `spec.channel` is not an +atomic operation, as it may span multiple storage systems. Changes to +`spec.subscriber`, `spec.reply`, `spec.delivery` and other fileds SHOULD be +permitted, as these could occur within a single storage system. + +When a Subscription becomes associated with a channel (either due to creating +the Subscription or the channel), the Subscription MUST only set the `Ready` +condition to `true` after the channel has been configured to send all future +events to the Subscriptions `spec.subscriber`. The Channel MAY send some events +to the Subscription before prior to the Subscription's `Ready` condition being +set to `true`. When a Subscription is deleted, the Channel MAY send some +additional events to the Subscription's `spec.subscriber`. + +## Channel Lifecycle + +A Channel represents an Addressable endpoint (i.e. it has as +`status.address.url` field) which can receive, store, and forward events to +multiple recipients (Subscriptions). Subscriptions are associated with a Channel +based on the `spec.channel` field on the Subscription; it is expected that the +controller for a Channel will also control the associated Subscriptions. When +the Channel's `Ready` condition is `true`, the Channel MUST provide a +`status.address.url` which accepts all CloudEvents and MUST forward the received +events to each associated Subscription whose `Ready` condition is `true`. As +described in the [Subscription Lifecycle](#subscription-lifecycle) section, a +Channel MAY forward events to an associated Subscription which does not +currently have a `true` `Ready` condition, including events received by the +Channel before the `Subscription` was created. + +When a Channel is created, its `spec.channelTemplate` field MUST be populated to +indicate which of several possible Channel implementations to use. It is +RECOMMENDED to default the `spec.channelTemplate` field on creation if it is +unpopulated. Once created, the `spec.channelTemplate` field MUST be immutabel; +the Channel MUST be deleted and re-created to change the +`spec.channelTemplate`. This pattern is chosen to make it clear that changing +`spec.channelTemplate` is not an atomic operation and that any implementation +would be likely to result in message loss during the transition. + + -## Subscription +## Event Source Lifecycle -TODO: lifecycle; what happens if a subscription is created before a channel? What about events received by a channel before the subscription is created? Are channel conditions reflected in the subscription? What are the required conditions (Ready, ???)? `spec.channel` SHOULD be immutable. + + +## Addressable Resolution -## Addressable resolution +Both Trigger and Subscription have optional object references (`ref` in +`spec.subscriber`, `spec.delivery.deadLetterSink`, and `spec.reply` for +Subscription) which are expected to conform to the Addressable partial schema +("duck type"). An object conforms with the Addressable partial schema if it +contains a `status.address.url` field containing a URL which may be used to +deliver CloudEvents over HTTP. As a special case, Kubernetes `v1` `Service` +objects are considered to have a `status.address.url` of +`http:///`. If one of these object references points to an +object which does not currently satisfy this partial schema (either because the +`status.address.url` field is empty, or because the object does not have that +field), then the Trigger or Subscription MUST indicate an error by setting the +`Ready` condition to `false`, and SHOULD include an indication of the error in a +condition reason or type. -TODO: What is it? How does it apply to Trigger / Subscription? Indicate that Broker & Channel MUST implement Addressable. +Both Broker and Channel MUST conform to the Addressable partial schema. # Event Routing ## Content Based Routing +Each filter (Trigger) is evaluated independently for each received event, and a received event may be + + TODO: How do Broker & Trigger handle routing. - When must events arriving at a Broker be routed by a Trigger (when the Trigger is Ready?) - How do retries and replies interact with configuration changes in the Trigger / Broker? @@ -278,8 +391,12 @@ TODO: What's required of an event source with respect to routing? Retries? Dead- # Detailed Resources +TODO: copy over schemas + ## Broker v1 + + ## Trigger v1 ## Channel v1 From 8ef8f61d1f580302c0ca28b731dab1c232677429 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 6 May 2021 11:36:18 -0700 Subject: [PATCH 05/46] Update (rewrite) overview, motivation, and data plane --- specs/eventing/data-plane.md | 260 +++++++++++++++++++++-------------- specs/eventing/motivation.md | 58 +++----- specs/eventing/overview.md | 135 +++++++++--------- 3 files changed, 237 insertions(+), 216 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 81e1aa3fc..abb5e6069 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -1,125 +1,171 @@ -# Knative Eventing Data Plane Contracts +# Knative Eventing Data Plane Contract ## Introduction -Developers using Knative Eventing need to know what is supported for delivery to -user provided components that receive events. Knative Eventing defines contract -for data plane components and we have listed them here. - -## Conformance +Late-binding event senders and receivers (composing applications using +configuration) only works when all event senders and recipients speak a common +protocol. In order to enable wide support for senders and receivers, Knative +Eventing extends the [CloudEvents HTTP +bindings](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) +with additional semantics for the following reasons: + +- Knative Eventing aims to enable highly-reliable event processing workflows. As + such, it prefers duplicate delivery to discarded events. The CloudEvents spec + does not take a stance here. + +- The CloudEvents HTTP bindings provide a relatively simple and efficient + network protocol which can easily be supported in a wide variety of + programming languages leveraging existing library investments in HTTP. + +- Knative Eventing assumes a sender-driven (push) event delivery system. That + is, each event processor is actively responsible for an event until it is + handled (or affirmatively delivered to all following recipients). + +- Knative Eventing aims to make writing [event + sources](./overview.md#event-source) and event-processing software easier to + write; as such, it imposes higher standards on system components like + [brokers](./overview.md#broker) and [channels](./overview.md#channel) than on + edge components. + +This contract defines a mechanism for a single event sender to reliably deliver +a single event to a single recipient. Building from this primitive, chains of +reliable event delivery and event-driven applications can be built. + +## Background The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119. -## Data plane contract for Sinks +When not specified in this document, the [CloudEvents HTTP bindings, version +1](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) and +[HTTP 1.1 protocol](https://tools.ietf.org/html/rfc7230) standards should be +followed (with the CloudEvents bindings preferred in the case of conflict). -A **Sink** MUST be able to handle duplicate events. +The current version of this document does not describe protocol negotiation or +the ability to upgrade an HTTP 1.1 event delivery into a more efficient protocol +such as GRPC, AMQP, or the like. It is expected that a future compatible version +of this specification may describe a protocol negotiation mechanism. -A **Sink** is an [_addressable_](./interfaces.md#addressable) resource that -takes responsibility for the event. A Sink could be a consumer of events, or -middleware. A Sink MUST be able to receive CloudEvents over HTTP and HTTPS. +## Event Delivery -A **Sink** MAY be [_callable_](./interfaces.md#callable) resource that -represents an Addressable endpoint which receives an event as input and -optionally returns an event to forward downstream. +To provide simpler support for event sources which may be translating events +from existing systems, some data plane requirements for senders are relaxed in +the general case. In the case of Knative Eventing provided resources (Channels +and Brokers) which implement these roles, requirements may be increased from +SHOULD to MUST. These cases are called out as they occur. -Almost every component in Knative Eventing may be a Sink providing -composability. +### Minimum supported protocol -Every Sink MUST support HTTP Protocol Binding for CloudEvents -[version 1.0](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md) +All senders and recipients MUST support the CloudEvents 1.0 protocol and the +[binary](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#31-binary-content-mode) and -[version 0.3](https://github.com/cloudevents/spec/blob/v0.3/http-transport-binding.md) -with restrictions and extensions specified below. - -### HTTP Support - -This section adds restrictions on -[requirements in HTTP Protocol Binding for CloudEvents](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md#12-relation-to-http). - -Sinks MUST accept HTTP requests with POST method and MAY support other HTTP -methods. If a method is not supported Sink MUST respond with HTTP status code -`405 Method Not Supported`. Non-event requests (e.g. health checks) are not -constrained. - -The URL used by a Sink MUST correspond to a single, unique endpoint at any given -moment in time. This MAY be done via the host, path, query string, or any -combination of these. This mapping is handled exclusively by the -[Addressable control-plane](./interfaces.md#control-plane) exposed via the -`status.address.url`. - -If an HTTP request's URL does not correspond to an existing endpoint, then the -Sink MUST respond with `404 Not Found`. - -Every non-Callable Sink MUST respond with `202 Accepted` if the request is -accepted. - -If Sink is Callable it MAY respond with `200 OK` and a single event in the HTTP -response. A returned event is not required to be related to the received event. -The Callable should return a successful response if the event was processed -successfully. If there is no event to send back then Callable Sink MUST respond -with 2xx HTTP and with empty body. - -If a Sink receives a request and is unable to parse a valid CloudEvent, then it -MUST respond with `400 Bad Request`. - -### Content Modes Supported - -A Sink MUST support `Binary Content Mode` and `Structured Content Mode` as -described in -[HTTP Message Mapping section of HTTP Protocol Binding for CloudEvents](https://github.com/cloudevents/spec/blob/master/http-protocol-binding.md#3-http-message-mapping) - -A Sink MAY support `Batched Content Mode` but that mode is not used in Knative -Eventing currently (that may change in future). - -### Retries - -Sinks should expect that retries and accept possibility that duplicate events -may be delivered. - -### Error handling - -If Sink is not returning HTTP success header (200 or 202) then the event may be -sent again. If the event can not be delivered then some sources of events (such -as Knative sources, brokers or channels) MAY support -[dead letter sink or channel](https://github.com/knative/eventing/blob/main/docs/delivery/README.md) for events that can not be -delivered. +[structured](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#32-structured-content-mode) +content modes of the CloudEvetns HTTP binding. Senders MUST support both +cleartext (`http`) and TLS (`https`) URLs as event delivery destinations. + +### HTTP Verbs + +In the absence of specific delivery preferences, the sender MUST initiate +delivery of the event to the recipient using the HTTP POST verb, using either +the structured or binary encoding of the event (sender's choice). This delivery +SHOULD be performed using the CloudEvents HTTP Binding, version 1. + +Senders MAY probe the recipient with an [HTTP OPTIONS +request](https://tools.ietf.org/html/rfc7231#section-4.3.7); if implemented, the +recipent MUST indicate support for the POST verb using the [`Allow` +header](https://tools.ietf.org/html/rfc7231#section-7.4.1). Senders which +receive an error when probing with HTTP OPTIONS SHOULD proceed using the HTTP +POST mechanism. + +### Event Acknowledgement and Repeat Delivery + +Event recipients MUST use the HTTP response code to indicate acceptance of an +event. The recipient MUST NOT return a response accepting the event until it has +handled event (processed the event or stored it in stable storage). The +following response codes are explicitly defined; event recipients MAY also +respond with other response codes. A response code not in this table SHOULD be +treated as a retriable error. + +| Response code | Meaning | Retry | Delivery completed | Error | +| ------------- | --------------------------- | ----- | ------------------ | ----- | +| `1xx` | (Unspecified) | Yes\* | No\* | Yes\* | +| `200` | [Event reply](#event-reply) | No | Yes | No | +| `202` | Event accepted | No | Yes | No | +| other `2xx` | (Unspecified) | Yes\* | No\* | Yes\* | +| other `3xx` | (Unspecified) | Yes\* | No\* | Yes\* | +| `400` | Unparsable event | No | No | Yes | +| `404` | Endpoint does not exist | No | No | Yes | +| other `4xx` | Error | Yes | No | Yes | +| other `5xx` | Error | Yes | No | Yes | + +\* Unspecified `1xx`, `2xx`, and `3xx` response codes are **reserved for future +extension**. Event recipients SHOULD NOT send these response codes in this spec +version, but event senders MUST handle these response codes as errors and +implement appropriate failure behavior. + + + +Recipients MUST be able to handle duplicate delivery of events and MUST accept +delivery of duplicate events, as the event acknowledgement could have been lost +in return to the sender. Event recipients MUST use the [`source` and `id` +attributes](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#required-attributes) +to detect duplicated events (see [observability](#observability) for an example +case where other event attributes may vary from one delivery attempt to +another). + +Where possible, event senders SHOULD re-attempt delivery of events where the +HTTP request failed or returned a retriable status code. It is RECOMMENDED that +event senders implement some form of congestion control (such as exponential +backoff) when managing retry timing. This specification does not document any +specific congestion control algorithm or +parameters. [Brokers](./overview.md#broker) and +[Channels](./overview.md#channel) MUST implement congestion control and MUST +implement retries. ### Observability -CloudEvents received by Sink MAY have -[Distributed Tracing Extension Attribute](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md). - -### Event reply contract - -An event sender supporting event replies SHOULD include a `Prefer: reply` header -in delivery requests to indicate to the sink that event reply is supported. An -event sender MAY ignore an event reply in the delivery response if the -`Prefer: reply` header was not included in the delivery request. - -An example is that a Broker supporting event reply sends events with an -additional header `Prefer: reply` so that the sink connected to the Broker knows -event replies will be accepted. While a source sends events without the header, -in which case the sink may assume that any event reply will be dropped without -error or retry attempt. If a sink wishes to ensure the reply events will be -delivered, it can check for the existence of the `Prefer: reply` header in the -delivery request and respond with an error code if the header is not present. - -### Data plane contract for Sources - -See [Source Delivery specification](sources.md#source-event-delivery) -for details. - -### Data plane contract for Channels - -See [Channel Delivery specification](channel.md#data-plane) for details. - -### Data plane contract for Brokers - -See [Broker Delivery specification](broker.md) - -## Changelog +Event senders MAY add or update CloudEvents attributes before sending to +implement observability features such as tracing; in particular, the +[`traceparent` and `tracestate` distributed tracing +attributes](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md) +may be modified in this way for each delivery attempt of the same event. + +This specification does not mandate any particular logging or metrics +aggregtion, nor a method of exposing observability information to users +configuring the resources. Platform administrators SHOULD expose event-delivery +telemetry to users through platform-specific interfaces, but such interfaces are +beyond the scope of this document. + + + +### Derived (Reply) Events + +In some applications, an event receiver might emit an event in reaction to a +received event. An event sender MAY document support for this pattern by +including a `Prefer: reply` header in the HTTP POST request. This header +indicates to the event receiver that the caller will accept a [`200` +response](#event-acknowledgement-and-repeat-delivery) which includes a +CloudEvent encoded using the binary or structured formats. +[Brokers](./overview.md#broker) and [Channels](./overview.md#channel) MUST +indicate support for replies using the `Prefer: reply` header. + +The sender SHOULD NOT assume that a received reply event is directly related to +the event sent in the HTTP request. + +A recipient MAY reply to any HTTP POST with a `200` response to indicate that +the event was processed successfully, with or without a response payload. If the +recipient will _never_ provide a response payload, the `202` response code is +likely a better choice. + +If a recipient chooses to reply to a sender with a `200` response code and a +reply event in the absence of a `Prefer: reply` header, the sender SHOULD treat +the event as accepted, and MAY log an error about the unexpected payload. The +sender MUST NOT process the reply event if it did not advertise the `Prefer: +reply` capability. -- 2020-04-20: `0.13.x release`: initial version that documents common contract - for sinks, sources, channels and brokers. diff --git a/specs/eventing/motivation.md b/specs/eventing/motivation.md index 9d2b48653..42492f8b5 100644 --- a/specs/eventing/motivation.md +++ b/specs/eventing/motivation.md @@ -1,49 +1,33 @@ # Motivation -The goal of Knative Eventing is to define common, composable primitives to -enable late-binding event sources and event consumers. +The goal of the Knative Eventing project is to define common primitives to +enable composing event-processing applications through configuration, rather +than application code. - +Building by combining independent components provides a number of benefits for +application designers: -Knative Eventing has following principles: - -1. Services are loosely coupled during development and deployed independently on - a variety of platforms (Kubernetes, VMs, SaaS or FaaS). +1. Services are loosely coupled during development and may be deployed + independently on a variety of platforms (Kubernetes, VMs, SaaS or FaaS). This + composability allows re-use of common patterns and building blocks, even + across programming language and tooling boundaries. 1. A producer can generate events before a consumer is listening, and a consumer can express an interest in an event or class of events that is not yet being - produced. + produced. This allows event-driven applications to be evolve over time + without needing to closely coordinate changes. -1. Services can be connected to create new applications - - without modifying producer or consumer. +1. Services can be connected to create new applications: + - without modifying producer or consumer - with the ability to select a specific subset of events from a particular producer -These primitives enable producing and consuming events adhering to the -[CloudEvents Specification](https://github.com/cloudevents/spec), in a decoupled -way. - -Kubernetes has no primitives related to event processing, yet this is an -essential component in serverless workloads. Eventing introduces high-level -primitives for event production and delivery with an initial focus on push over -HTTP. If a new event source or type is required of your application, the effort -required to plumb them into the existing eventing framework will be minimal and -will integrate with CloudEvents middleware and message consumers. - -Knative eventing implements common components of an event delivery ecosystem: -enumeration and discovery of event sources, configuration and management of -event transport, and declarative binding of events (generated either by storage -services or earlier computation) to further event processing and persistence. - -The Knative Eventing API is intended to operate independently, and interoperate -well with the [Serving API](https://github.com/knative/serving) and -[Build API](https://github.com/knative/build). - ---- - -_Navigation_: +In order to enable loose coupling and late-binding of event producers and +consumers, Knative Eventing utilizes and extends the [CloudEvents +specification](https:?/github.com/cloudevents/spec) as the data plane protocol +between components. Knative Eventing prioritizes at-least-once delivery +semantics, using the CloudEvents HTTP POST (push) transport as a minimum common +transport between components. -- **Motivation and goals** -- [Resource type overview](overview.md) -- [Interface contracts](interfaces.md) -- [Object model specification](spec.md) +Knative Eventing also defines patterns to simplify the construction and usage of +event senders and recipients. diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index 3fafca4d4..e0b63de5f 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -1,97 +1,88 @@ # Resource Types -The API defines and provides a complete implementation for -[Trigger](spec.md#kind-trigger), [Broker](spec.md#kind-broker), -[Subscription](spec.md#kind-subscription) and abstract resource definitions for -[Channels](spec.md#kind-channel). +The Knative Eventing API provides primitives for two common event-processing +patterns (credit to James Urquhart for the formulation): -With extensibility and composability as a goal of Knative Eventing, the eventing -API defines several resources that can be reduced down to well understood -contracts. These eventing resource interfaces may be fulfilled by other -Kubernetes objects and then composed in the same way as the concrete objects. -The interfaces are ([Addressable](interfaces.md#addressable), -[Callable](interfaces.md#callable)). For more details, see -[Interface Contracts](interfaces.md). +* Point-to-point asynchronous communication ([`messaging.knative.dev`](#messaging)) -- A **Trigger** describes a filter on event attributes which should be delivered - to an _Addressable_. +* Content-based event routing ([`eventing.knative.dev`](#eventing)) -- A **Broker** provides a bucket of events which can be selected by attribute. +The other two patterns James identifies are log-stream processing and complex +workflows; these are not currently addressed by Knative Eventing. - +In addition to the primitives needed to express the above patterns, Knative +Eventing defines two [_interface contracts_](#interface-contracts) to allow +connecting multiple types of Kubernetes objects as event senders and recipients +to the core primitives. -![Broker Trigger Overview](images/broker-trigger-overview.svg) + -The above diagram shows a _Broker_ ingesting events and delivering to a -_Service_ only when the _Trigger_ filter matches. +## Eventing -- A **Subscription** describes the transformation of an event and optional - forwarding of a returned event. +### Broker -- A **Channel** provides event persistence and fanout of events from a - well-known input address to multiple outputs described by _Subscriptions_. +**Broker** provides a central event-routing hub which exposes a URL address +which event senders may use to submit events to the router. A Broker may be +implemented using many different underlying event-forwarding mechanisms; the +broker provides a small set of common event-delivery configuration options and +may reference additional implementation-specific configuration options via a +reference to an external object; the format of the external objects is not +standardized. - +### Trigger -![Resource Types Overview](images/resource-types-overview.svg) +**Trigger** defines a filtered delivery option to extract events delivered to a +**Broker** and route them to an **Addressable** destination. Trigger implements +uniform event filtering based on the CloudEvents attributes associated with the +event, ignoring the payload (which might be large and/or binary and need not be +parsed during event routing). The addressable interface contract allows Triggers +to deliver events to a variety of different destinations, including external +resources such as a virtual machine or SaaS service. -Channels as well as Sources are defined by independent CRDs that can be -installed into a cluster. Both Sources and Channels implementations can be -created directly. A Channel however offers also a way to create the backing -implementation as part of the generic Channel (using a _channelTemplate_). +## Messaging -## Trigger +### Channel -**Trigger** describes a registration of interest on a filter set of events -delivered to a _Broker_ which should be delivered to an _Addressable_. Events -selected by a _Trigger_ are buffered independently from other _Triggers_, even -if they deliver to the same _Addressable_. +**Channel** provides an abstract interface which may be fulfiled by several +concrete implementations of a backing asynchronous fan-out queue. The common +abstraction provided by channel allows both the composition of higher-level +constructs for chained or parallel processing of events, and the replacement of +particular messaging technologies (for example, allowing a development +environment to use a lower-reliability channel compared with the production +environment). -For more details, see [Kind: Trigger](spec.md#kind-trigger). +### Subscription -## Broker +**Subscription** defines a delivery destination for all events sent to a +**Channel**. Events sent to a channel are delivered to _each_ subscription +_independently_ -- a subscription maintains its own list of undelivered events +and will manage retry indpendently of any other subscriptions to the same +channel. Like **Trigger**, subscriptions use the **Addressable** interface +contract to support event delivery to many different destination types. -**Broker** provides an eventing mesh. This allows producers to deliver events to -a single endpoint and not need to worry about the routing details for individual -consumers. +## Interface Contracts -For more details, see [Kind: Broker](spec.md#kind-broker). +In addition to the concrete types described above in the `messaging.knative.dev` +and `eventing.knative.dev` API groups, Knative Eventing supports referencing +objects in other API groups as destinations for event delivery. This is done by +defining partial schemas which the other resources must support. The following +interface contracts define a set of expected resource fields on an referenced +resource. -## Subscription +### Addressable -**Subscriptions** describe a flow of events from one _Channel_ to the next -Channel\* through transformations (such as a Knative Service which processes -CloudEvents over HTTP). A _Subscription_ controller resolves the addresses of -transformations (`subscriber`) and destination storage (`reply`) through the -_Callable_ and _Addressable_ interface contracts, and writes the resolved -addresses to the _Channel_ in the `channel` reference. _Subscriptions_ do not -need to specify both a transformation and a storage destination, but at least -one must be provided. +**Addressable** resources expose a resource address (HTTP URL) in their `status` +object. The URL is used as a destination for delivery of events to the resource; +the exposed URL must implement the [data plane contract](data-plane.md) for +receiving events. -All event delivery linkage from a **Subscription** is 1:1 – only a single -`channel`, `subscriber`, and `reply` may be provided. +**Broker** and **Channel** both implement **Addressable**. -For more details, see [Kind: Subscription](spec.md#kind-subscription). +### Event Source -## Channel +**Event Sources** are resources which generate events and may be configured to +deliver the events to an **Addressable** resource designated by a `sink` object +in the resource's `spec`. The Knative Eventing spec does not define any specific +event sources, but does define common interfaces for discovering and managing +event sources. -**Channel** provides an event delivery mechanism which can fan out received -events to multiple destinations via _Subscriptions_. A _Channel_ has a single -inbound _Addressable_ interface which may accept events delivered directly or -forwarded from multiple _Subscriptions_. Different _Channels_ may implement -different degrees of persistence. Event delivery order is dependent on the -backing implementation of the _Channel_. - -Event selection on a _Channel_ is 1:N – a single _Channel_ may fan out to -multiple _Subscriptions_. - -See [Kind: Channel](spec.md#kind-channel). - ---- - -_Navigation_: - -- [Motivation and goals](motivation.md) -- **Resource type overview** -- [Interface contracts](interfaces.md) -- [Object model specification](spec.md) From 86f9c556f82cde1a030de6e3b540672aef0dae2a Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Fri, 7 May 2021 14:28:54 -0700 Subject: [PATCH 06/46] Update lifecycle to be a bit clearer and cover more fields --- specs/eventing/control-plane.md | 256 ++++++++++++++++++++------------ 1 file changed, 165 insertions(+), 91 deletions(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index 8ac15b9e4..d6032e834 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -3,17 +3,16 @@ The Knative Eventing platform provides common primitives for routing CloudEvents between cooperating HTTP clients. This document describes the structure, lifecycle, and management of Knative Eventing resources in the context of the -[Kubernetes Resource -Model](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/resource-management.md). An -understanding of the Kubernetes API interface and the capabilities of -[Kubernetes Custom -Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) is assumed. - -This document does not define the [data plane event delivery -contract](./data-plane.md) (though it does describe how event delivery is -configured). This document also does not prescribe specific implementations of -supporting services such as access control, observability, or resource -management. +[Kubernetes Resource Model](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/resource-management.md). +An understanding of the Kubernetes API interface and the capabilities of +[Kubernetes Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) +is assumed. + +This document does not define the +[data plane event delivery contract](./data-plane.md) (though it does describe +how event delivery is configured). This document also does not prescribe +specific implementations of supporting services such as access control, +observability, or resource management. # Background @@ -29,8 +28,8 @@ illustrative (i.e. _an implementation on Kubernetes_) or descriptive (i.e. _this Kubernetes resource MUST be exposed_). References to these core Kubernetes resources will be annotated as either illustrative or descriptive. -This document considers two users of a given Knative Eventing environment, and is -particularly concerned with the expectations of developers (and language and +This document considers two users of a given Knative Eventing environment, and +is particularly concerned with the expectations of developers (and language and tooling developers, by extension) deploying applications to the environment. - **Developers** configure Knative resources to implement an event-routing @@ -41,13 +40,13 @@ tooling developers, by extension) deploying applications to the environment. # RBAC Profile -In order to validate the controls described in [Resource -Overview](#resource-overview), the following Kubernetes RBAC profile may be -applied in a Kubernetes cluster. This Kubernetes RBAC is an illustrative example -of the minimal profile rather than a requirement. This Role should be sufficient -to develop, deploy, and manage event routing for an application within a single -namespace. Knative Conformance tests against "MUST", "MUST NOT", and "REQUIRED" -conditions are expected to pass when using this profile: +In order to validate the controls described in +[Resource Overview](#resource-overview), the following Kubernetes RBAC profile +may be applied in a Kubernetes cluster. This Kubernetes RBAC is an illustrative +example of the minimal profile rather than a requirement. This Role should be +sufficient to develop, deploy, and manage event routing for an application +within a single namespace. Knative Conformance tests against "MUST", "MUST NOT", +and "REQUIRED" conditions are expected to pass when using this profile: ```yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -63,12 +62,36 @@ rules: verbs: ["get", "list", "create", "update", "delete"] ``` +In order to support resolving resources which meet the +[Addressable](./overview.md#addressable) contract, the system controlling the +[Trigger](#trigger-lifecycle) or [Subscription](#subscription-lifecycle) will +need _read_ access to these resources. On Kubernetes, this is most easily +achieved using role aggregation; on systems using Kubernetes RBAC, resources +which wish to participate in Addressable resolution should provide the following +`ClusterRole`: + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: -addressable-resolver + labels: + duck.knative.dev/addressable: "true" +rules: + - apiGroups: [] + resources: [, /status] + verbs: ["get", "list", "watch"] +``` + +This configuration advice SHALL NOT indicate a requirement for Kubernetes RBAC +support. + # Resource Overview @@ -86,6 +109,7 @@ roles. See the [Overview documentation](./overview.md) for general definitions of the different API objects. # Error Signalling + The Knative API uses the @@ -230,32 +254,6 @@ transient conditions be indicated with a `severity="Info"`. # Resource Lifecycle -## Trigger Lifecycle - -The lifecycle of a Trigger is independent of that of the Broker it refers to in -its `spec.broker` field; if the Broker does not currently exist or its `Ready` -condition is not `true`, then the Trigger's `Ready` condition MUST NOT be -`true`, and the reason SHOULD indicate that the corresponding Broker is missing -or not ready. Similarly, if the Trigger's `spec.subscriber` field contains a -`ref` to another resource, the Trigger's `Ready` condition MUST NOT be `true` if -the `spec.subscriber.ref` field points to a resource which does not exist or -which does not have a `status.address.url` field. - -Once created, the Trigger's `spec.broker` SHOULD NOT permit updates; to change -the `spec.broker`, the Trigger can be deleted and re-created. This pattern is -chosen to make it clear that changing the `spec.broker` is not an atomic -operation, as it may span multiple storage systems. Changes to -`spec.subscriber`, `spec.filter` and other fields SHOULD be permitted, as these -could occur within a single storage system. - -When a Trigger becomes associated with a Broker (either due to creating the -Trigger or the Broker), the Trigger MUST only set the `Ready` condition to `true` -after the Broker has been configured to send all future events matching the -`spec.filter` to the Trigger's `spec.subscriber`. The Broker MAY send some -events to the Trigger`s `spec.subscriber` prior to the Trigger's `Ready` condition -being set to `true`. When a Trigger is deleted, the Broker MAY send some -additional events to the Trigger`s `spec.subscriber`. - ## Broker Lifecycle A Broker represents an Addressable endpoint (i.e. it has a `status.address.url` @@ -279,21 +277,87 @@ re-created to change the `spec.class`. This pattern is chosen to make it clear that changing `spec.class` is not an atomic operation and that any implementation would be likely to result in message loss during the transition. +## Trigger Lifecycle + +The lifecycle of a Trigger is independent of the Broker it refers to in its +`spec.broker` field; if the Broker does not currently exist or the Broker's +`Ready` condition is not `true`, then the Trigger's `Ready` condition MUST be +`false`, and the reason SHOULD indicate that the corresponding Broker is missing +or not ready. + +The Trigger MUST also set the `status.subscriberUri` field based on resolving +the `spec.subcriber` field before setting the `Ready` condition to `true`. If +the `spec.subscriber.ref` field points to a resource which does not exist or +cannot be resolved via [Addressable resolution](#addressable-resolution), the +Trigger MUST set the `Ready` condition to `false`, and at least one condition +should indicate the reason for the error. + +If the Trigger's `spec.delivery.deadLetterSink` field it set, it MUST be +resolved to a URL and reported in `status.deadLetterSinkUri` in the same manner +as the `spec.subscriber` field before setting the `Ready` condition to `true`. + +Once created, the Trigger's `spec.broker` SHOULD NOT permit updates; to change +the `spec.broker`, the Trigger can be deleted and re-created. This pattern is +chosen to make it clear that changing the `spec.broker` is not an atomic +operation, as it may span multiple storage systems. Changes to +`spec.subscriber`, `spec.filter` and other fields SHOULD be permitted, as these +could occur within a single storage system. + +When a Trigger becomes associated with a Broker (either due to creating the +Trigger or the Broker), the Trigger MUST only set the `Ready` condition to +`true` after the Broker has been configured to send all future events matching +the `spec.filter` to the Trigger's `spec.subscriber`. The Broker MAY send some +events to the Trigger's `spec.subscriber` prior to the Trigger's +`Ready`condition being set to `true`. When a Trigger is deleted, the Broker MAY +send some additional events to the Trigger's `spec.subscriber` ftor tho +deletion. + +## Channel Lifecycle + +A Channel represents an Addressable endpoint (i.e. it has as +`status.address.url` field) which can receive, store, and forward events to +multiple recipients (Subscriptions). Subscriptions are associated with a Channel +based on the `spec.channel` field on the Subscription; it is expected that the +controller for a Channel will also control the associated Subscriptions. When +the Channel's `Ready` condition is `true`, the Channel MUST provide a +`status.address.url` which accepts all CloudEvents and MUST forward the received +events to each associated Subscription whose `Ready` condition is `true`. As +described in the [Subscription Lifecycle](#subscription-lifecycle) section, a +Channel MAY forward events to an associated Subscription which does not +currently have a `true` `Ready` condition, including events received by the +Channel before the `Subscription` was created. + +When a Channel is created, its `spec.channelTemplate` field MUST be populated to +indicate which of several possible Channel implementations to use. It is +RECOMMENDED to default the `spec.channelTemplate` field on creation if it is +unpopulated. Once created, the `spec.channelTemplate` field MUST be immutable; +the Channel MUST be deleted and re-created to change the `spec.channelTemplate`. +This pattern is chosen to make it clear that changing `spec.channelTemplate` is +not an atomic operation and that any implementation would be likely to result in +message loss during the transition. + ## Subscription Lifecycle -The lifecycle of a Subscription is independent of that of the Channel it refers +The lifecycle of a Subscription is independent of that of the channel it refers to in its `spec.channel` field. The `spec.channel` object reference may refer to either an `eventing.knative.dev/v1` Channel resource, or another resource which meets the `spec.subscribers` and `spec.delivery` required elements in the Channellable duck type. If the referenced `spec.channel` does not currently exist or its `Ready` condition is not `true`, then the Subscription's `Ready` condition MUST NOT be `true`, and the reason SHOULD indicate that the -corresponding Channel is missing or not ready. Similarly, if the Subscriptions -`spec.subscriber`, `spec.reply` or `spec.delivery.deadLetterSink` fields contain -a `ref` to another Resource, the Subscriptions `Ready` condition MUST NOT be -`true` unless the referred-to resources exist and contain a `status.address.url` -field. (It is acceptable for none of the `spec.subscriber`, `spec.reply`, and -`spec.delivery.deadLetterSink` fields to contain a `ref` field.) +corresponding channel is missing or not ready. + +The Subscription MUST also set the `status.physicalSubscription` URIs by +resolving the `spec.subscriber`, `spec.reply`, and +`spec.delivery.deadLetterSink` as described in +[Addressable resolution](#addressable-resolution) before setting the `Ready` +condition to `true`. If any of the addressable fields fails resolution, the +Subscription MUST set the `Ready` condition to `false`, and at least one +condition should indicate the reason for the error. (It is acceptable for none +of the `spec.subscriber`, `spec.reply`, and `spec.delivery.deadLetterSink` +fields to contain a `ref` field.) + + Once created, the Subscription's `spec.channel` SHOULD NOT permit updates; to change the `spec.channel`, the Subscription can be deleted and re-created. This @@ -310,30 +374,6 @@ to the Subscription before prior to the Subscription's `Ready` condition being set to `true`. When a Subscription is deleted, the Channel MAY send some additional events to the Subscription's `spec.subscriber`. -## Channel Lifecycle - -A Channel represents an Addressable endpoint (i.e. it has as -`status.address.url` field) which can receive, store, and forward events to -multiple recipients (Subscriptions). Subscriptions are associated with a Channel -based on the `spec.channel` field on the Subscription; it is expected that the -controller for a Channel will also control the associated Subscriptions. When -the Channel's `Ready` condition is `true`, the Channel MUST provide a -`status.address.url` which accepts all CloudEvents and MUST forward the received -events to each associated Subscription whose `Ready` condition is `true`. As -described in the [Subscription Lifecycle](#subscription-lifecycle) section, a -Channel MAY forward events to an associated Subscription which does not -currently have a `true` `Ready` condition, including events received by the -Channel before the `Subscription` was created. - -When a Channel is created, its `spec.channelTemplate` field MUST be populated to -indicate which of several possible Channel implementations to use. It is -RECOMMENDED to default the `spec.channelTemplate` field on creation if it is -unpopulated. Once created, the `spec.channelTemplate` field MUST be immutabel; -the Channel MUST be deleted and re-created to change the -`spec.channelTemplate`. This pattern is chosen to make it clear that changing -`spec.channelTemplate` is not an atomic operation and that any implementation -would be likely to result in message loss during the transition. - @@ -364,14 +404,40 @@ Both Broker and Channel MUST conform to the Addressable partial schema. # Event Routing -## Content Based Routing +Note that the event routing description below does not cover the actual +mechanics of sending an event from one component to another; see +[the data plane](./data-plane.md) contracts for details of the event transfer +mechanisms. -Each filter (Trigger) is evaluated independently for each received event, and a received event may be +## Content Based Routing +A Broker MUST publish a URL at `status.address.uri` when it is able to receive +events. This URL MUST accept CloudEvents in both the +[Binary Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#31-binary-content-mode) +and +[Structured Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#32-structured-content-mode) +HTTP formats. Before sending an HTTP response, the Broker MUST durably enqueue +the event (be able to deliver with retry without receiving the event again). + +For each event received by the Broker, the Broker MUST evaluate each associated +Trigger **once** (where "associated" means Trigger with a `spec.broker` which +references the Broker). If the Trigger has a `Ready` condition of `true` when +the event is evaluated, the the Broker MUST evaluate the Trigger's `spec.filter` +and, if matched, proceed with event delivery as described below. The Broker MAY +also evaluate and forward events to associated Triggers for which the `Ready` +condition is not currently `true`. (One example: a Trigger which is in the +process of being programmed in the Broker data plane might receive _some_ events +before the data plane programming was complete and the Trigger was updated to +set the `Ready` condition to `true`.) TODO: How do Broker & Trigger handle routing. -- When must events arriving at a Broker be routed by a Trigger (when the Trigger is Ready?) - - How do retries and replies interact with configuration changes in the Trigger / Broker? + +- When must events arriving at a Broker be routed by a Trigger (when the Trigger + is Ready?) + - How do retries and replies interact with configuration changes in the + Trigger / Broker? +- Duplicate Trigger destinations +- Duplicate filter rules - Retry parameters - Reply routing - Dead-letter routing @@ -379,15 +445,26 @@ TODO: How do Broker & Trigger handle routing. ## Topology Based Routing TODO: How do Channel & Subscription handle routing -- When must events arriving at a Channel be routed to a Subscription (when the Subscription is Ready?) - - How do retries and replies interact with configuration changes in the Subscription / Channel? + +- When must events arriving at a Channel be routed to a Subscription (when the + Subscription is Ready?) + - How do retries and replies interact with configuration changes in the + Subscription / Channel? - Retry parameters - Reply routing - Dead-letter routing +## Event Delivery + +Once a Trigger or Subscription has decided to deliver an event, it MUST do the +following: + +1. Attempt delivery to the `status.subscriberUri` URL + ## Event Sources -TODO: What's required of an event source with respect to routing? Retries? Dead-letter? Status? +TODO: What's required of an event source with respect to routing? Retries? +Dead-letter? Status? # Detailed Resources @@ -395,8 +472,6 @@ TODO: copy over schemas ## Broker v1 - - ## Trigger v1 ## Channel v1 @@ -405,12 +480,11 @@ TODO: copy over schemas ## Addressable v1 -================================================================ - CUT HERE -================================================================ +# ================================================================ CUT HERE ## Broker The Knative Broker represents a single instance of an event router which accepts events from one or more sources and routes them to selected destinations based -on rules matching the attributes of the received event. In order to do this, the Broker defines +on rules matching the attributes of the received event. In order to do this, the +Broker defines From e90a3adeb479b6b7aae08862c0817415dfb24f1c Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Sat, 8 May 2021 10:14:37 -0700 Subject: [PATCH 07/46] Fill in detailed resources. --- specs/eventing/control-plane.md | 547 ++++++++++++++++++++++++++++++++ 1 file changed, 547 insertions(+) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index d6032e834..ea10941c6 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -472,14 +472,561 @@ TODO: copy over schemas ## Broker v1 +### Metadata: + +Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. + +### Spec: + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
configKReference
(Optional)		A reference to an object which describes the configuration options for the Broker (for example, a ConfigMap).RECOMMENDED
deliveryDeliverySpec
(Optional)		A default delivery options for Triggers which do not specify more-specific options.REQUIRED
+ +### Status + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
conditionsSee Error SignallingUsed for signalling errors, see link. Conditions of types Ready MUST be present.REQUIRED
observedGenerationint64The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction.REQUIRED
addressduckv1.AddressableAddress used to deliver events to the Broker.REQUIRED
deadLetterSinkUriURL (string)If spec.delivery.deadLetterSink is specified, the resolved URL of the dead letter address.REQUIRED
+ ## Trigger v1 +### Metadata: + +Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. + +### Spec: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
brokerstring
(Required, Immutable)		The Broker which this Trigger should filter events from.REQUIRED
filterTriggerFilter
(Optional)		Event filters which are used to select events to be delivered to the Trigger's destination.REQUIRED
subscriberduckv1.Destination
(Required)		The destination that filtered events should be sent to.REQUIRED
deliveryDeliverySpec
(Optional)		Delivery options for this Trigger.RECOMMENDED
+ +### Status + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
conditionsSee Error SignallingUsed for signalling errors, see link. Conditions of types Ready MUST be present.REQUIRED
observedGenerationint64The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction.REQUIRED
subscriberUriURL (string)The resolved address of the spec.subscriber.REQUIRED
deadLetterSinkUriURL (string)If spec.delivery.deadLetterSink is specified, the resolved URL of the dead letter address.REQUIRED
+ ## Channel v1 +### Metadata: + +Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. + +### Spec: + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
channelTemplateobject
(Optional)		Implementation-specific parameters to configure the channel.OPTIONAL
subscribers[]duckv1.SubscriberSpecAggregated subscription information; this array should be managed automatically by the controller.RECOMMENDED
deliveryDeliverySpec
(Optional)		A default delivery options for Subscriptions which do not specify more-specific options.REQUIRED
+ +### Status + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
conditionsSee Error SignallingUsed for signalling errors, see link. Conditions of types Ready MUST be present.REQUIRED
observedGenerationint64The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction.REQUIRED
addressduckv1.AddressableAddress used to deliver events to the Broker.REQUIRED
subscribers[]duckv1.SubscriberStatusResolved addresses for the spec.subscribers (subscriptinos to this Channel).RECOMMENDED
deadLetterSinkUriURL (string)If spec.delivery.deadLetterSink is specified, the resolved URL of the dead letter address.REQUIRED
+ ## Subscription v1 +### Metadata: + +Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. + +### Spec: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
channelKubernetes v1/ObjectReference
(Required, Immutable)		The channel this subscription receives events from. Immutable.REQUIRED
subscriberduckv1.Destination
(Required)		The destination that events should be sent to.REQUIRED
replyduckv1.Destination
(Required)		The destination that reply events from spec.subscriber should be sent to.REQUIRED
deliveryDeliverySpec
(Optional)		Delivery options for this Subscription.RECOMMENDED
+ +### Status + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
conditionsSee Error SignallingUsed for signalling errors, see link. Conditions of types Ready MUST be present.REQUIRED
observedGenerationint64The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction.REQUIRED
physicalSubscriptionPhysicalSubscriptionStatusThe fully resolved values for spec endpoint references.REQUIRED
+ ## Addressable v1 +Note that the Addressable interface is a partial schema -- any resource which includes these fields may be referenced using a `duckv1.Destination`. + +### Metadata: + +Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. + +### Spec: + +There are no `spec` requirements for Addressable. + +### Status + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
addressduckv1.AddressableAddress used to deliver events to the Broker.REQUIRED
+ +# Detailed SubResource Objects + +## duckv1.Addressable + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
urlURLAddress used to deliver events to the Addressable.REQUIRED
+ +## duckv1.Destination + +Destination is used to indicate the destination for event delivery. This is an exclusive union; excatly one field MUST be set. + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
refduckv1.KReference
(Optional)		An ObjectReference to an Addressable reference to deliver events to. This is mutually exclusive with uri.REQUIRED
uriURL (string)
(Optional)		A resolved URL to deliver events to. This is mutually exclusive with ref.REQUIRED
+ +## duckv1.SubscriberSpec + +SubscriberSpec represents an automatically-populated extraction of information from a [Subscription](#subscription). + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
uidUID (string)UID is used to disambiguate Subscriptions which might be recreated.RECOMMENDED
generationint64Generation of the copied Subscription.RECOMMENDED
subscriberUriURL (string)The resolved address of the Subscription's spec.subscriber.REQUIRED
replyUriURL (string)The resolved address of the Subscription's spec.reply.REQUIRED
deliveryDeliverySpecThe resolved Subscription delivery options. The deadLetterSink should use the uri form.REQUIRED
+ +## duckv1.SubscriberStatus + +SubscriberStatus indicates the status of programming a Subscription by a Channel. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
uidUID (string)UID is used to disambiguate Subscriptions which might be recreated.RECOMMENDED
generationint64Generation of the copied Subscription.RECOMMENDED
readykubernetes v1/ConditionStatusReady status of the Subscription's programming into the Channel data plane.REQUIRED
messageduckv1.AddressableA human readable message indicating details of ready status.REQUIRED
+ +## DeliverySpec + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
deadLetterSinkduckv1.DestinationFallback address used to deliver events which cannot be delivered during the flow.RECOMMENDED
retryintRetry is the minimum number of retries the sender should attempt when sending an event before moving it to the dead letter sink.RECOMMENDED
backoffDelaystringThe initial delay when retrying delivery.RECOMMENDED
backoffPolicyenum
("linear", "exponential")		Retry timing scaling policy. Linear policy uses the same backoffDelay for each attempt; Exponential policy uses 2^N multiples of backoffDelayRECOMMENDED
+ +## KReference + +KReference is a lightweight version of kubernetes [`v1/ObjectReference`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectreference-v1-core) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
apiVersionstringApiVersion of the target reference.REQUIRED
kindstringKind of the target reference.REQUIRED
namespacestringNamespace of the target resource.REQUIRED
addressstringAddress used to deliver events to the Broker.REQUIRED
+ +## PhysicalSubscriptionStatus + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
subscriberUriURL (string)Resolved address of the spec.subscriber.REQUIRED
replyUriURL (string)Resolved address of the spec.reply.REQUIRED
deadLetterSinkUriURL (string)Resolved address of the spec.delivery.deadLetterSink.REQUIRED
+ +## TriggerFilter + + + + + + + + + + + + + + +
Field NameField TypeDescriptionSchema Requirement
attributesmap[string]stringEvent filter using exact match on event context attributes. Each key in the map is compared with the equivalent key in the event context. An event passes the filter if the event attribute values are equal to the specified values.REQUIRED
+ # ================================================================ CUT HERE ## Broker From 0d7b24693ba53efe55927ff4c9fdb97f368f1ead Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Sun, 9 May 2021 08:50:35 -0700 Subject: [PATCH 08/46] Document event delivery (Broker & Channel) --- specs/eventing/control-plane.md | 150 ++++++++++++++++++++++---------- 1 file changed, 105 insertions(+), 45 deletions(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index ea10941c6..863d4f003 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -378,12 +378,6 @@ additional events to the Subscription's `spec.subscriber`. TODO: channel-compatible CRDs (Channelable) --> -## Event Source Lifecycle - - - ## Addressable Resolution Both Trigger and Subscription have optional object references (`ref` in @@ -420,61 +414,114 @@ HTTP formats. Before sending an HTTP response, the Broker MUST durably enqueue the event (be able to deliver with retry without receiving the event again). For each event received by the Broker, the Broker MUST evaluate each associated -Trigger **once** (where "associated" means Trigger with a `spec.broker` which +Trigger **once** (where "associated" means a Trigger with a `spec.broker` which references the Broker). If the Trigger has a `Ready` condition of `true` when the event is evaluated, the the Broker MUST evaluate the Trigger's `spec.filter` -and, if matched, proceed with event delivery as described below. The Broker MAY -also evaluate and forward events to associated Triggers for which the `Ready` +and, if matched, proceed with +[event delivery as described below](#event-delivery). The Broker MAY also +evaluate and forward events to associated Triggers for which the `Ready` condition is not currently `true`. (One example: a Trigger which is in the process of being programmed in the Broker data plane might receive _some_ events before the data plane programming was complete and the Trigger was updated to set the `Ready` condition to `true`.) -TODO: How do Broker & Trigger handle routing. - -- When must events arriving at a Broker be routed by a Trigger (when the Trigger - is Ready?) - - How do retries and replies interact with configuration changes in the - Trigger / Broker? -- Duplicate Trigger destinations -- Duplicate filter rules -- Retry parameters -- Reply routing -- Dead-letter routing +If multiple Triggers match an event, one event delivery MUST be generated for +each match; duplicate matches with the same destination MUST each generate a +separate event delivery attempts, one per Trigger match. The implementation MAY +attach additional event attributes or other metaata distinguishing between these +deliveries. The implementation MUST NOT modify the event payload in this +process. + +Reply events generated during event delivery MUST be re-enqueued by the Broker +in the same way as events delivered to the Broker's Addressable URL. Reply +events re-enqued in this maanner MUST be evaluated against all triggers +associated with the Broker, including the Trigger that generated the reply. +Implementations MAY implement event-loop detection; it is RECOMMENDED that any +such controls be documented to end-users. Implementations MAY avoid using HTTP +to deliver event replies to the Broker's event-delivery input and instead use an +internal queueing mechanism. ## Topology Based Routing -TODO: How do Channel & Subscription handle routing +A Channel MUST publish a URL at `status.address.url` when it is able to receive +events. This URL MUST accept CloudEvents in both the +[Binary Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#31-binary-content-mode) +and +[Structured Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#32-structured-content-mode) +HTTP formats. Before sending an HTTP response, the Channel MUST durably enqueue +the event (be able to deliver with retry without receiving the event again). -- When must events arriving at a Channel be routed to a Subscription (when the - Subscription is Ready?) - - How do retries and replies interact with configuration changes in the - Subscription / Channel? -- Retry parameters -- Reply routing -- Dead-letter routing +For each event received by the Channel, the Channel MUST deliver the event to +each associated Subscription **once** (where "associated" means a Subscription +with a `spec.channel` which references the Channel). If the Subscription has a +`Ready` condition of `true` when the event is evaluated, the Channel MUST +forward the event as described in +[event delivery as described below](#event-delivery). The Channel MAY also +forward events to associated Subscriptions for with the `Ready` condition is not +currently `true`. (One example: a Subscription which is in the process of being +programmed in the Channel data plane might receive _some_ events before the data +plane programming was complete and the Subscription was updated to set the +`Ready` condition to `true`.) + +If multiple Subscriptions with the same destination are associated with the same +Channel, each Subscription MUST generate one delivery attempt per Subscription. +The implementation MAY attach additional event attributes or other metadata +distinguishing between these deliveries. The implementation MUST NOT modify the +event payload in this process. ## Event Delivery Once a Trigger or Subscription has decided to deliver an event, it MUST do the following: -1. Attempt delivery to the `status.subscriberUri` URL +1. Resolve all URLs and delivery options. -## Event Sources +1. Attempt delivery to the `status.subscriberUri` URL following the + [data plane contract](./data-plane.md). -TODO: What's required of an event source with respect to routing? Retries? -Dead-letter? Status? + 1. If the event delivery fails with a retriable error, it SHOULD be retried + up to `retry` times, following the `backoffPolicy` and `backoffDelay` + parameters if specified. -# Detailed Resources +1. If the delivery attempt is successful (either the original request or a + retry) and no event is returned, the event delivery is complete. + +1. If the delivery attempt is successful (either the original request or a + retry) and an event is returned in the reply, the reply event will be + delivered to the `status.replyUri` destination (for Subscriptions) or added + to the Broker for processing (for Triggers). If `status.replyUri` is not + present in the Subscription, the reply event is be dropped. + + 1. If delivery of the reply event fails with a retriable error, the delivery + of the reply event to the reply destination SHOULD be retried up to + `retry` times, following the `backoffPolicy` and `backoffDelay` parameters + if specified. -TODO: copy over schemas +1. If all retries are exhausted for either the original delivery or the retry, + or if a non-retryable error is received, the event should be delivered to the + `deadLetterSink` in the delivery options. If no `deadLetterSink` is + specified, the event is dropped. + + The implementation MAY set additional attributes on the event or wrap the + failed event in a "failed delivery" event; this behavior is not (currently) + standardized. + + If delivery of the reply event fails with a retriable error, the delivery to + the `deadLetterSink` SHOULD be retried up to `retry` times, following the + `backoffPolicy` and `backoffDelay` parameters if specified. Alternatively, + implementations MAY use an equivalent internal mechanism for delivery (for + example, if the `ref` form of `deadLetterSink` points to a compatible + implementation). + +# Detailed Resources ## Broker v1 ### Metadata: -Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. +Standard Kubernetes +[metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) +resource. ### Spec: @@ -538,7 +585,9 @@ Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/gen ### Metadata: -Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. +Standard Kubernetes +[metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) +resource. ### Spec: @@ -614,7 +663,9 @@ Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/gen ### Metadata: -Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. +Standard Kubernetes +[metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) +resource. ### Spec: @@ -690,7 +741,9 @@ Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/gen ### Metadata: -Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. +Standard Kubernetes +[metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) +resource. ### Spec: @@ -758,11 +811,14 @@ Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/gen ## Addressable v1 -Note that the Addressable interface is a partial schema -- any resource which includes these fields may be referenced using a `duckv1.Destination`. +Note that the Addressable interface is a partial schema -- any resource which +includes these fields may be referenced using a `duckv1.Destination`. ### Metadata: -Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. +Standard Kubernetes +[metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) +resource. ### Spec: @@ -806,7 +862,8 @@ There are no `spec` requirements for Addressable. ## duckv1.Destination -Destination is used to indicate the destination for event delivery. This is an exclusive union; excatly one field MUST be set. +Destination is used to indicate the destination for event delivery. This is an +exclusive union; excatly one field MUST be set. @@ -831,7 +888,8 @@ Destination is used to indicate the destination for event delivery. This is an e ## duckv1.SubscriberSpec -SubscriberSpec represents an automatically-populated extraction of information from a [Subscription](#subscription). +SubscriberSpec represents an automatically-populated extraction of information +from a [Subscription](#subscription).
@@ -874,7 +932,8 @@ SubscriberSpec represents an automatically-populated extraction of information f ## duckv1.SubscriberStatus -SubscriberStatus indicates the status of programming a Subscription by a Channel. +SubscriberStatus indicates the status of programming a Subscription by a +Channel.
@@ -921,7 +980,7 @@ SubscriberStatus indicates the status of programming a Subscription by a Channel - + @@ -946,7 +1005,8 @@ SubscriberStatus indicates the status of programming a Subscription by a Channel ## KReference -KReference is a lightweight version of kubernetes [`v1/ObjectReference`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectreference-v1-core) +KReference is a lightweight version of kubernetes +[`v1/ObjectReference`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectreference-v1-core)
deadLetterSink duckv1.DestinationFallback address used to deliver events which cannot be delivered during the flow.Fallback address used to deliver events which cannot be delivered during the flow. An implementation MAY place limits on the allowed destinations for the deadLetterSink. RECOMMENDED
From 0a33307584c20c3e71969daccb0e35ff2c95e9b2 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Sun, 9 May 2021 14:46:02 -0700 Subject: [PATCH 09/46] Updates from n3wscott review --- specs/eventing/data-plane.md | 10 +++++----- specs/eventing/motivation.md | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index abb5e6069..dd16b5f83 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -45,14 +45,14 @@ followed (with the CloudEvents bindings preferred in the case of conflict). The current version of this document does not describe protocol negotiation or the ability to upgrade an HTTP 1.1 event delivery into a more efficient protocol such as GRPC, AMQP, or the like. It is expected that a future compatible version -of this specification may describe a protocol negotiation mechanism. +of this specification might describe a protocol negotiation mechanism. ## Event Delivery -To provide simpler support for event sources which may be translating events +To provide simpler support for event sources which might be translating events from existing systems, some data plane requirements for senders are relaxed in the general case. In the case of Knative Eventing provided resources (Channels -and Brokers) which implement these roles, requirements may be increased from +and Brokers) which implement these roles, requirements are increased from SHOULD to MUST. These cases are called out as they occur. ### Minimum supported protocol @@ -69,7 +69,7 @@ cleartext (`http`) and TLS (`https`) URLs as event delivery destinations. In the absence of specific delivery preferences, the sender MUST initiate delivery of the event to the recipient using the HTTP POST verb, using either the structured or binary encoding of the event (sender's choice). This delivery -SHOULD be performed using the CloudEvents HTTP Binding, version 1. +SHOULD be performed using the CloudEvents HTTP Binding, version 1.0. Senders MAY probe the recipient with an [HTTP OPTIONS request](https://tools.ietf.org/html/rfc7231#section-4.3.7); if implemented, the @@ -95,7 +95,7 @@ treated as a retriable error. | other `2xx` | (Unspecified) | Yes\* | No\* | Yes\* | | other `3xx` | (Unspecified) | Yes\* | No\* | Yes\* | | `400` | Unparsable event | No | No | Yes | -| `404` | Endpoint does not exist | No | No | Yes | +| `404` | Endpoint does not exist | Yes | No | Yes | | other `4xx` | Error | Yes | No | Yes | | other `5xx` | Error | Yes | No | Yes | diff --git a/specs/eventing/motivation.md b/specs/eventing/motivation.md index 42492f8b5..44439748c 100644 --- a/specs/eventing/motivation.md +++ b/specs/eventing/motivation.md @@ -24,7 +24,7 @@ application designers: In order to enable loose coupling and late-binding of event producers and consumers, Knative Eventing utilizes and extends the [CloudEvents -specification](https:?/github.com/cloudevents/spec) as the data plane protocol +specification](https://github.com/cloudevents/spec) as the data plane protocol between components. Knative Eventing prioritizes at-least-once delivery semantics, using the CloudEvents HTTP POST (push) transport as a minimum common transport between components. From 7e9b26bcd9d7274d28b2c66c7f5a37ad1ac8ce56 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Sun, 9 May 2021 14:47:35 -0700 Subject: [PATCH 10/46] Clean up no longer needed files. --- specs/eventing/README.md | 13 +- specs/eventing/broker.md | 229 --------------------- specs/eventing/channel.md | 385 ----------------------------------- specs/eventing/helper.md | 92 --------- specs/eventing/interfaces.md | 71 ------- specs/eventing/sources.md | 366 --------------------------------- specs/eventing/spec.md | 382 ---------------------------------- 7 files changed, 2 insertions(+), 1536 deletions(-) delete mode 100644 specs/eventing/broker.md delete mode 100644 specs/eventing/channel.md delete mode 100644 specs/eventing/helper.md delete mode 100644 specs/eventing/interfaces.md delete mode 100644 specs/eventing/sources.md delete mode 100644 specs/eventing/spec.md diff --git a/specs/eventing/README.md b/specs/eventing/README.md index 243e86a49..99690d798 100644 --- a/specs/eventing/README.md +++ b/specs/eventing/README.md @@ -12,14 +12,5 @@ Docs in this directory: - [Motivation and goals](motivation.md) - [Resource type overview](overview.md) -- [Interface contracts](interfaces.md) -- [Object model specification](spec.md) -- [Channel specification](channel.md) -- [Sources specification](sources.md) -- [Error conditions and reporting](https://knative.dev/docs/serving/spec/knative-api-specification-1.0/#error-signalling) - -See the -[Knative Eventing Docs Architecture](https://www.knative.dev/docs/eventing/#architecture) -for more high level details. - - +- [Control Plane specification](control-plane.md) +- [Data Plane specification](data-plane.md) diff --git a/specs/eventing/broker.md b/specs/eventing/broker.md deleted file mode 100644 index 9c6b5e0c6..000000000 --- a/specs/eventing/broker.md +++ /dev/null @@ -1,229 +0,0 @@ -# Knative Broker Specification - -## Background - -A _Broker_ specifies an _ingress_ to which _producers_ may produce events. A -_Trigger_ specifies a _subscriber_ that will receive events delivered to its -assigned Broker which match a specified _filter_. - -## Conformance - -The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", -"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be -interpreted as described in RFC2119. - -## Control Plane - -### Broker - -Broker objects SHOULD include a Ready condition in their status. - -The Broker SHOULD indicate Ready=True when its ingress is available to receive -events. - -While a Broker is Ready, it SHOULD be a valid Addressable and its -`status.address.url` field SHOULD indicate the address of its ingress. - -The class of a Broker object SHOULD be immutable. - -### Trigger - -Triggers SHOULD include a Ready condition in their status. - -The Trigger SHOULD indicate Ready=True when events can be delivered to its -subscriber. - -While a Trigger is Ready, it SHOULD indicate its subscriber's URI via the -`status.subscriberUri` field. - -Triggers MUST be assigned to exactly one Broker. The assigned Broker of a -trigger SHOULD be immutable. - -A Trigger MAY be created before its assigned Broker exists. A Trigger SHOULD -progress to Ready when its assigned Broker exists and is Ready. - -The attributes filter specifying a list of key-value pairs MUST be supported by -Trigger. Events that pass the attributes filter MUST include context or -extension attributes that match all key-value pairs exactly. - -### Delivery Spec - -Both BrokerSpec and TriggerSpec have a `Delivery` field of type -[`duck.DeliverySpec`](./spec.md#eventingduckv1deliveryspec). This field, among -the other features, allows the user to define the dead letter sink and retries. -The `BrokerSpec.Delivery` field is global across all the Triggers registered -with that particular Broker, while the `TriggerSpec.Delivery`, if configured, -fully overrides `BrokerSpec.Delivery` for that particular Trigger, hence: - -- When `BrokerSpec.Delivery` and `TriggerSpec.Delivery` are both not configured, - no delivery spec SHOULD be used. -- When `BrokerSpec.Delivery` is configured, but not the specific - `TriggerSpec.Delivery`, then the `BrokerSpec.Delivery` SHOULD be used. -- When `TriggerSpec.Delivery` is configured, then `TriggerSpec.Delivery` SHOULD - be used. - -## Data Plane - -### Ingress - -A Broker SHOULD expose either an HTTP or HTTPS endpoint as ingress. It MAY -expose both. - -The ingress endpoint(s) MUST conform to at least one of the following versions -of the specification: - -- [CloudEvents 0.3 specification](https://github.com/cloudevents/spec/blob/v0.3/http-transport-binding.md) -- [CloudEvents 1.0 specification](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md) - -Other versions MAY be rejected. The usage of CloudEvents version 1.0 is -RECOMMENDED. - -The Broker SHOULD NOT perform an upgrade of the produced event's CloudEvents -version. It SHOULD support both Binary Content Mode and Structured Content Mode -of the HTTP Protocol Binding for CloudEvents. - -The HTTP(S) endpoint MAY be on any port, not just the standard 80 and 443. -Channels MAY expose other, non-HTTP endpoints in addition to HTTP at their -discretion (e.g. expose a gRPC endpoint to accept events). - -Brokers MUST reject all HTTP produce requests with a method other than POST -responding with HTTP status code `405 Method Not Supported`. Non-event queueing -requests (e.g. health checks) are not constrained. - -The Broker MUST respond with a 200-level HTTP status code if a produce request -is accepted. - -If a Broker receives a produce request and is unable to parse a valid -CloudEvent, then it MUST reject the request with a 400 level HTTP status code, -for example: `400 Bad Request`, or `415 Unsupported Media Type`. - -### Delivery - -Delivered events must conform to the CloudEvents specification. All CloudEvent -attributes set by the producer, including the data and specversion attributes, -SHOULD be received at the subscriber identical to how they were received by the -Broker. - -The Broker SHOULD support delivering events via Binary Content Mode or -Structured Content Mode of the HTTP Protocol Binding for CloudEvents. - -Events accepted by the Broker SHOULD be delivered at least once to all -subscribers of all Triggers that: - -1. are Ready when the produce request was received, -1. specify filters that match the event, and -1. exist when the event is able to be delivered. - -Events MAY additionally be delivered to Triggers that become Ready after the -event was accepted. - -Events MAY be enqueued or delayed between acceptance from a producer and -delivery to a subscriber. - -The Broker MAY choose not to deliver an event due to persistent unavailability -of a subscriber or limitations such as storage capacity. The Broker SHOULD -attempt to notify the operator in this case. The Broker MAY forward these events -to an alternate endpoint or storage mechanism such as a dead letter queue. - -If no ready Trigger would match an accepted event, the Broker MAY drop that -event without notifying the producer. From the producer's perspective, the event -was delivered successfully to zero subscribers. - -If multiple Triggers reference the same subscriber, the subscriber MAY be -expected to acknowledge successful delivery of an event multiple times. - -Events contained in delivery responses SHOULD be published to the Broker ingress -and processed as if the event had been produced to the Broker's addressable -endpoint. - -Events contained in delivery responses that are malformed SHOULD be treated as -if the event delivery had failed. Reasoning being that if the event was being -transformed unsuccessfully (programming error for example) it should be treated -as a failure. - -The subscriber MAY receive a confirmation that a reply event was accepted by the -Broker. If the reply event was not accepted, the initial event SHOULD be -redelivered to the subscriber. - -### Observability - -The Broker SHOULD expose a variety of metrics, including, but not limited to: - -- Number of malformed produce requests (400-level responses) -- Number of accepted produce requests (200-level responses) -- Number of events delivered - -Metrics SHOULD be enabled by default, with a configuration parameter included to -disable them if desired. - -Upon receiving an event with context attributes defined in the -[CloudEvents Distributed Tracing extension](https://github.com/cloudevents/spec/blob/master/extensions/distributed-tracing.md), -the Broker SHOULD preserve that trace header on delivery to subscribers and on -reply events, unless the reply is sent with a different set of tracing -attributes. Forwarded trace headers SHOULD be updated with any intermediate -spans emitted by the broker. - -Spans emitted by the Broker SHOULD follow the -[OpenTelemetry Semantic Conventions for Messaging Systems](https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/semantic_conventions/messaging.md) -whenever possible. In particular, spans emitted by the Broker SHOULD set the -following attributes: - -- messaging.system: "knative" -- messaging.destination: broker:name.namespace or trigger:name.namespace with - the Broker or Trigger to which the event is being routed -- messaging.protocol: the name of the underlying transport protocol -- messaging.message_id: the event ID - -## Conformance Tests - -This test outline validates the conformance of a Broker implementation, with -some [exceptions](#exceptions). - -1. Control Plane Tests - 1. A Trigger can be created before its Broker exists. That Trigger specifies - an attributes filter. - 1. A Broker can be created (given valid configuration) and progresses to - Ready. - 1. The Broker, once Ready, is a valid Addressable. - 1. The Trigger, once its Broker is Ready, progresses to Ready. - 1. The Trigger, once Ready, includes a status.subscriberUri field. - 1. A second Trigger, with a different but overlapping filter,, can be created - and progresses to Ready. -1. Data Plane Tests - 1. Broker ingress can receive events in the following formats via HTTP: - 1. CloudEvents 0.3 - 1. CloudEvents 1.0 - 1. Structured mode - 1. Binary mode - 1. Broker ingress responds with: - 1. 2xx on valid event - 1. 400 on invalid event - 1. Broker ingress responds with 405 to any method except POST to publish URI - 1. Produce 2 events to ingress: - 1. Matching only first Trigger (event 1) - 1. Matching both first and second Trigger (event 2) - 1. With a known trace header - 1. First Trigger subscriber receives events 1 and 2: - 1. With original version - 1. With all originally specified attributes identical - 1. With the correct trace header - 1. Matching its filter - 1. And replies to event 1 with an event that matches only second trigger - (event 3) - 1. Second Trigger subscriber receives events 2 and 3: - 1. And rejects events on first delivery, verifying events are redelivered - 1. With the correct trace header - -### Exceptions - -These aspects of the spec are not tested in this outline: - -- **Replies that fail to be published cause initial message to be redelivered.** - Requires implementation-specific setup to induce a failure. -- **Metrics support.** Currently there is no shared format that could be used to - test support for metrics. - -## Changelog - -- `0.13.x release`: Initial version. -- Add conformance test outline. diff --git a/specs/eventing/channel.md b/specs/eventing/channel.md deleted file mode 100644 index 3bf8624c7..000000000 --- a/specs/eventing/channel.md +++ /dev/null @@ -1,385 +0,0 @@ -# Knative Channel Specification - -## Background - -The Knative Eventing project has one generic `Channel` CRD and might ship -different Channel CRDs implementations (e.g.`InMemoryChannel`) inside of in the -`messaging.knative.dev/v1` API Group. The generic `Channel` CRD points to the -chosen _default_ `Channel` implementation, like the `InMemoryChannel`. - -A _channel_ logically receives events on its input domain and forwards them to -its subscribers. Below is a specification for the generic parts of each -`Channel`. - -A typical channel consists of a _Controller_ and a _Dispatcher_ pod. - -## Conformance - -The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", -"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be -interpreted as described in [RFC2119](https://www.ietf.org/rfc/rfc2119.txt). - -## Channel Spec Parts - -### API Group - -The CRD's API group MAY be any valid API Group. If it is in `knative.dev`, then -it SHOULD be `messaging.knative.dev`. - -### Kind Naming - -The CRD's Kind SHOULD have the suffix `Channel`. The name MAY be just `Channel`. - -### Control Plane - -Each Channel implementation is backed by its own CRD, like the -`InMemoryChannel`. Below is an example for the `InMemoryChannel`: - -``` -apiVersion: messaging.knative.dev/v1 -kind: InMemoryChannel -metadata: - name: my-channel -``` - -Each _Channel Controller_ ensures the required tasks on the backing technology -are applied. - -> NOTE: For instance on a `KafkaChannel` this would mean taking care of creating -> an Apache Kafka topic and backing all messages from the _Knative Channel_. - -#### Aggregated Channelable Manipulator ClusterRole - -Every CRD MUST create a corresponding ClusterRole, that will be aggregated into -the `channelable-manipulator` ClusterRole. This ClusterRole MUST include -permissions to create, get, list, watch, patch, and update the CRD's custom -objects and their status. Below is an example for the `InMemoryChannel`: - -``` -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: imc-channelable-manipulator - labels: - duck.knative.dev/channelable: "true" -rules: - - apiGroups: - - messaging.knative.dev - resources: - - inmemorychannels - - inmemorychannels/status - verbs: - - create - - get - - list - - watch - - update - - patch -``` - -Each channel MUST have the `duck.knative.dev/channelable: "true"` label on its -`channelable-manipulator` ClusterRole. - -#### Aggregated Addressable Resolver ClusterRole - -Every CRD MUST create a corresponding ClusterRole, that will be aggregated into -the `addressable-resolver` ClusterRole. This ClusterRole MUST include -permissions to get, list, and watch the CRD's custom objects and their status. -Below is an example for the `InMemoryChannel`: - -``` -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: imc-addressable-resolver - labels: - duck.knative.dev/addressable: "true" -rules: - - apiGroups: - - messaging.knative.dev - resources: - - inmemorychannels - - inmemorychannels/status - verbs: - - get - - list - - watch -``` - -Each channel MUST have the `duck.knative.dev/addressable: "true"` label on its -`addressable-resolver` ClusterRole. - -#### CustomResourceDefinition per Channel - -For each channel implementation a `CustomResourceDefinition` is created, like: - -``` -apiVersion: apiextensions.k8s.io/v1beta1 -kind: CustomResourceDefinition -metadata: - name: inmemorychannels.messaging.knative.dev - labels: - knative.dev/crd-install: "true" - messaging.knative.dev/subscribable: "true" - duck.knative.dev/addressable: "true" -spec: - group: messaging.knative.dev - version: v1 - names: - kind: InMemoryChannel - plural: inmemorychannels - singular: inmemorychannel - categories: - - all - - knative - - messaging - - channel - shortNames: - - imc - scope: Namespaced -... -``` - -Each channel is _namespaced_ and MUST have the following: - -- label of `messaging.knative.dev/subscribable: "true"` -- label of `duck.knative.dev/addressable: "true"` -- The category `channel` - -#### Annotation Requirements - -Each instantiated Channel (ie, Custom Object) SHOULD have an annotation -indicating which version of the `Channelable` duck type it conforms to. We -currently have these versions: - -1. [v1beta1](https://github.com/knative/eventing/blob/main/pkg/apis/duck/v1beta1/channelable_types.go) -1. [v1](https://github.com/knative/eventing/blob/main/pkg/apis/duck/v1/channelable_types.go) - -So, for example to indicate that the Channel supports v1beta1 duck type, you -should annotate it like so (only showing the annotations): - -``` -- apiVersion: messaging.knative.dev/v1beta1 - kind: YourChannelType - metadata: - annotations: messaging.knative.dev/subscribable: v1beta1 -``` - -Unfortunately, we had to make breaking changes between v1alpha1 and v1beta1 -versions, and to ensure functionality, the channel implementer must indicate -which version they support. To ensure backwards compatibility with old channels, -if no annotation is given, we assume it's `v1alpha1`. - -#### Spec Requirements - -##### v1beta1 Spec - -Each channel CRD MUST contain an array of subscribers: -[`spec.subscribers`](https://github.com/knative/eventing/blob/main/pkg/apis/duck/v1beta1/subscribable_types.go) - -Note: The array of subscribers MUST NOT be set directly on the generic Channel -custom object, but rather appended to the backing channel by the subscription -itself. - -##### v1 Spec - -Each channel CRD MUST contain an array of subscribers: -[`spec.subscribers`](https://github.com/knative/eventing/blob/main/pkg/apis/duck/v1/subscribable_types.go) - -Note: The array of subscribers MUST NOT be set directly on the generic Channel -custom object, but rather appended to the backing channel by the subscription -itself. - -#### Channelable and Subscription Delivery Spec - -Both Channelable and Subscription have a -[`delivery`](https://github.com/knative/eventing/blob/main/pkg/apis/duck/v1/delivery_types.go) -field that allows the user to define the dead letter sink and retries. The -Channelable `spec.delivery` field is global across all the Subscriptions -registered with that particular Channelable, while the Subscription -`spec.delivery`, if configured, fully overrides Channeleable `spec.delivery` for -that particular Subscription. - -#### Status Requirements - -##### v1beta1 Status - -Each channel CRD MUST have a `status` subresource which contains - -- [`address`](https://github.com/knative/pkg/blob/main/apis/duck/v1/addressable_types.go) -- [`subscribers`](https://github.com/knative/eventing/blob/main/pkg/apis/duck/v1beta1/subscribable_types.go) - (as an array) - -Each channel CRD SHOULD have the following fields in `Status` - -- [`observedGeneration`](https://github.com/knative/pkg/blob/main/apis/duck/v1/status_types.go) - MUST be populated if present -- [`conditions`](https://github.com/knative/pkg/blob/main/apis/duck/v1/status_types.go) - (as an array) SHOULD indicate status transitions and error reasons if present - -##### v1 Status - -Each channel CRD MUST have a `status` subresource which contains - -- [`address`](https://github.com/knative/pkg/blob/main/apis/duck/v1/addressable_types.go) -- [`subscribers`](https://github.com/knative/eventing/blob/main/pkg/apis/duck/v1/subscribable_types.go) - (as an array) - -Each channel CRD SHOULD have the following fields in `Status` - -- [`observedGeneration`](https://github.com/knative/pkg/blob/main/apis/duck/v1/status_types.go) - MUST be populated if present -- [`conditions`](https://github.com/knative/pkg/blob/main/apis/duck/v1/status_types.go) - (as an array) SHOULD indicate status transitions and error reasons if present - -#### Channel Status - -##### v1beta1 - -When the channel instance is ready to receive events `status.address.url` MUST -be populated and `status.addressable` MUST be set to `True`. - -##### v1 - -When the channel instance is ready to receive events `status.address.url` MUST -be populated and `status.addressable` MUST be set to `True`. - -#### Channel Subscriber Status - -##### v1beta1 - -Each subscription to a channel is added to the channel `status.subscribers` -automatically. The `ready` field of the subscriber identified by its `uid` MUST -be set to `True` when the subscription is ready to be processed. - -##### v1 - -Each subscription to a channel is added to the channel `status.subscribers` -automatically. The `ready` field of the subscriber identified by its `uid` MUST -be set to `True` when the subscription is ready to be processed. - -### Data Plane - -The data plane describes the input and output flow of a _Channel_. All Channels -exclusively communicate using CloudEvents. - -#### Input - -Every Channel MUST expose either an HTTP or HTTPS endpoint. It MAY expose both. -The endpoint(s) MUST conform to one of the following versions of the -specification: - -- [CloudEvents 0.3 specification](https://github.com/cloudevents/spec/blob/v0.3/http-transport-binding.md) -- [CloudEvents 1.0 specification](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md) - -The usage of CloudEvents version `1.0` is RECOMMENDED. - -The Channel MUST NOT perform an upgrade of the passed in version. It MUST emit -the event with the same version. - -It MUST support both _Binary Content Mode_ and _Structured Content Mode_ of the -HTTP Protocol Binding for CloudEvents. When dispatching the event, the channel -MAY use a different -[HTTP Message mode](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md#3-http-message-mapping) -of the one used by the event. For example, It MAY receive an event in -_Structured Content Mode_ and dispatch in _Binary Content Mode_. - -The HTTP(S) endpoint MAY be on any port, not just the standard 80 and 443. -Channels MAY expose other, non-HTTP endpoints in addition to HTTP at their -discretion (e.g. expose a gRPC endpoint to accept events). - -##### Generic - -If a Channel receives an event queueing request and is unable to parse a valid -CloudEvent, then it MUST reject the request. - -##### HTTP - -Channels MUST reject all HTTP event queueing requests with a method other than -POST responding with HTTP status code `405 Method Not Supported`. Non-event -queueing requests (e.g. health checks) are not constrained. - -The HTTP event queueing request's URL MUST correspond to a single, unique -Channel at any given moment in time. This MAY be done via the host, path, query -string, or any combination of these. This mapping is handled exclusively by the -Channel implementation, exposed via the Channel's `status.address`. - -If a Channel receives a request that does not correspond to a known channel, -then it MUST respond with a `404 Not Found`. - -The Channel MUST respond with `202 Accepted` if the event queueing request is -accepted by the server. - -If a Channel receives an event queueing request and is unable to parse a valid -CloudEvent, then it MUST respond with `400 Bad Request`. - -#### Output - -Channels MUST output CloudEvents. The output MUST match the CloudEvent version -of the [Input](#input). Channels MUST NOT alter an event that goes through them. -All CloudEvent attributes, including the data attribute, MUST be received at the -subscriber identical to how they were received by the Channel, except: - -- The extension attribute `knativehistory`, which the channel MAY modify to - append its hostname - -Every Channel SHOULD support sending events via _Binary Content Mode_ or -_Structured Content Mode_ of the HTTP Protocol Binding for CloudEvents, although -dispatching events using _Binary Content Mode_ is RECOMMENDED. - -Channels MUST send events to all subscribers which are marked with a status of -`ready: "True"` in the channel's `status.subscribers` (v1beta1 / v1). The events -must be sent to the `subscriberURI` field of `spec.subscribers` (v1beta1 / v1). -Each channel implementation will have its own quality of service guarantees -(e.g. at least once, at most once, etc) which SHOULD be documented. - -##### Retries - -Channels SHOULD retry resending CloudEvents when they fail to either connect or -send CloudEvents to subscribers. - -Channels SHOULD support various retry configuration parameters, including, but -not limited to: - -- the maximum number of retries -- the time in-between retries -- the backoff rate - -#### Observability - -Channels SHOULD expose a variety of metrics, including, but not limited to: - -- Number of malformed incoming event queueing events (`400 Bad Request` - responses) -- Number of accepted incoming event queuing events (`202 Accepted` responses) -- Number of egress CloudEvents produced (with the former metric, used to derive - channel queue size) - -Metrics SHOULD be enabled by default, with a configuration parameter included to -disable them if desired. - -The Channel MUST recognize and pass through all tracing information from sender -to subscribers using [W3C Tracecontext](https://w3c.github.io/trace-context/), -although internally it MAY use another mechanism(s) to propagate the tracing -information. The Channel SHOULD sample and write traces to the location -specified in -[`config-tracing`](https://github.com/knative/eventing/blob/main/config/config-tracing.yaml). - -Spans emitted by the Channel SHOULD follow the -[OpenTelemetry Semantic Conventions for Messaging Systems](https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/semantic_conventions/messaging.md) -whenever possible. In particular, spans emitted by the Channel SHOULD set the -following attributes: - -- messaging.system: "knative" -- messaging.destination: url to which the event is being routed -- messaging.protocol: the name of the underlying transport protocol -- messaging.message_id: the event ID - -## Changelog - -- `0.11.x release`: CloudEvents in 0.3 and 1.0 are supported. -- `0.13.x release`: Types in the API group `messaging.knative.dev` will be - promoted from `v1alpha1`to `v1beta1`. Add requirement for labeling Custom - Objects to indicate which duck type they support as well as document - differences. -- `0.22.x release`: Drop support for v1alpha1 channelable. diff --git a/specs/eventing/helper.md b/specs/eventing/helper.md deleted file mode 100644 index dbe75e83f..000000000 --- a/specs/eventing/helper.md +++ /dev/null @@ -1,92 +0,0 @@ -# Implementation Helper - -This document supplements the official [spec](spec.md) with useful -implementation details for components implemented in this repo. Note that the -implementations for other components in other repos (Kafka, RabbitMQ, GCP, etc.) -may vary - -- [Trigger](#kind-trigger) -- [Broker](#kind-broker) -- [Channel](#kind-channel) -- [Subscription](#kind-subscription) - -## kind: Trigger - -The `Trigger` specification is described in -[Object Model: Trigger](spec.md#kind-trigger). - -### Readiness Sub-Conditions - -- **BrokerReady.** True when the broker exists and is ready. -- **SubscriptionReady.** True when the subscriber is subscribed to the broker. -- **DependencyReady.** True when the sources the trigger depends on are ready. -- **SubscriberResolved.** True when the subscriber is resolved, i.e. the trigger - can get its address. - -### Events - -- TriggerReconciled -- TriggerReconcileFailed -- TriggerUpdateStatusFailed - ---- - -## kind: Broker - -The `Broker` specification is described in -[Object Model: Broker](spec.md#kind-broker). - -## Readiness Sub-Conditions - -- **IngressReady.** True when the broker ingress is ready. -- **TriggerChannelReady.** True when the trigger channel is ready. -- **FilterReady.** True when the filter is ready. -- **Addressable.** True when the broker has a resolved address in its status. - -### Events - -- BrokerReconciled -- BrokerUpdateStatusFailed - ---- - -## kind: Channel - -The `Channel` specification is described in -[Object Model: Channel](spec.md#kind-channel). - -### Readiness Sub-Conditions - -- **BackingChannelReady.** True when the backing `Channel` CRD is ready. -- **Addressable.** True when the channel meets the - [`Addressable`](interfaces.md#addressable) contract and has a non-empty - hostname. - -### Events - -- ChannelReconcileError -- ChannelReconciled -- ChannelReadinessChanged -- ChannelUpdateStatusFailed - ---- - -## kind: Subscription - -The `Subscription` specification is described in -[Object Model: Subscription](spec.md#kind-subscription). - -### Readiness Sub-Conditions - -- **ReferencesResolved.** True when all the specified references (`channel`, - `subscriber`, and `reply` ) have been successfully resolved. -- **AddedToChannel.** True when the controller has successfully added a - subscription to the `spec.channel` resource. -- **ChannelReady.** True when the channel has marked the subscriber as ready. - -### Events - -- PublisherAcknowledged -- ActionFailed - ---- diff --git a/specs/eventing/interfaces.md b/specs/eventing/interfaces.md deleted file mode 100644 index 1af901549..000000000 --- a/specs/eventing/interfaces.md +++ /dev/null @@ -1,71 +0,0 @@ -# Interface Contracts - -## Addressable - -An **Addressable** resource receives events over a network transport (currently -only HTTP is supported). The _Addressable_ returns success when it has -successfully handled the event (for example, by committing it to stable -storage). When used as an _Addressable_, only the acknowledgement or return code -is used to determine whether the event was handled successfully. One example of -an _Addressable_ is a _Channel_. - -### Control Plane - -An **Addressable** resource MUST expose a `status.address.url` field. The -_hostname_ value is a cluster-resolvable DNS name which is capable of receiving -event deliveries. _Addressable_ resources may be referenced in the `reply` -section of a _Subscription_, and also by other custom resources acting as an -event Source. - -### Data Plane - -An **Addressable** resource will only respond to requests with success or -failure. Any payload (including a valid CloudEvent) returned to the sender will -be ignored. An _Addressable_ may receive the same event multiple times even if -it previously indicated success. - ---- - -## Callable - -A **Callable** resource represents an _Addressable_ endpoint which receives an -event as input and optionally returns an event to forward downstream. One -example of a _Callable_ is a function. Note that all _Callable_ resources are -_Addressable_ (they accept an event and return a status code when completed), -but not all _Addressable_ resources are _Callable_. - -### Control Plane - -A **Callable** resource MUST expose a `status.address.url` field (like -_Addressable_). The _hostname_ value is a cluster-resolvable DNS name which is -capable of receiving event deliveries and returning a resulting event in the -reply.. _Callable_ resources may be referenced in the `subscriber` section of a -_Subscription_. - - - -### Data Plane - -The **Callable** resource receives one event and returns no or a single event in -the response. A returned event is not required to be related to the received -event. The _Callable_ should return a successful response if the event was -processed successfully. - -The _Callable_ is not responsible for ensuring successful delivery of any -received or returned event. It may receive the same event multiple times even if -it previously indicated success. - ---- - -_Navigation_: - -- [Motivation and goals](motivation.md) -- [Resource type overview](overview.md) -- **Interface contracts** -- [Object model specification](spec.md) diff --git a/specs/eventing/sources.md b/specs/eventing/sources.md deleted file mode 100644 index da93d155d..000000000 --- a/specs/eventing/sources.md +++ /dev/null @@ -1,366 +0,0 @@ -# Sources - -A **Source** is any Kubernetes object that generates or imports an event and -relays that event to another endpoint on the cluster via -[CloudEvents](https://cloudevents.io). Sourcing events is critical to developing -a distributed system that reacts to events. - -A **Sink** is an [_addressable_](./interfaces.md#addressable) resource that -takes responsibility for the event. A **Sink** could be a consumer of events, or -middleware. A **Sink** will respond with 2xx when it has accepted and processed -the event. - -A Source: - -- Represents an off- or on-cluster system, service or application that produces - events to be consumed by a **Sink**. -- Produces or imports CloudEvents. -- Sends CloudEvents to the configured **Sink**. - -In practice, sources are an abstract concept that allow us to create declarative -configurations through the usage of Custom Resource Definitions (CRDs) extending -Kubernetes. Source systems are configured by creating an instance of the -resource described by the CRD. It is up to the implementation of the source -author to understand the best way to realize the source application. This could -be as 1:1 deployments inside of Kubernetes per resource, as a single -multi-tenant application, or even an off-cluster implementation; or all -combinations in-between. - -For operators of a Kubernetes cluster, there are two states to sources: - -1. A Source CRD and controller (if required) have been installed into the - cluster. - - A cluster with a source CRD installed allows the developers in the cluster to - find and discover what is possible to source events from. This topic is - expanded upon in the [Source CRDs](#source-crds) section. - -1. A Source Custom Object (CO) has been created in the cluster. - - Once a developer creates an instance of a source and provides the necessary - parameters, the source controller will realize this into whatever is required - for that source for the current situation. While this resource is running, a - cluster operator would like to inspect the resource without needing to be - fully aware of the implementation. This is done by conforming to the - [Source]() ducktype. This topic is expanded upon in the - [Source Custom Objects](#source-custom-objects) section. - -The goal of requiring CRD labels and running resource shapes is to enable -discovery and understanding of potential sources that could be leveraged in the -cluster. This structure also aids in understanding sources that exist and are -running in the cluster. - -## Source CRDs - -Sources are more useful if they are discoverable. Knative Sources MUST use a -standardized label to allow controllers and operators the ability to find which -CRDs are considered to be adhering to the -[Source](https://pkg.go.dev/github.com/knative/pkg/apis/duck/v1#Source) -ducktype. - -CRDs that are to be understood as a `source` MUST be labeled: - -```yaml -apiVersion: apiextensions.k8s.io/v1beta1 -kind: CustomResourceDefinition -metadata: - labels: - duck.knative.dev/source: "true" # <-- required to be a source. -``` - -Labeling sources in this way allows for developers to filter the list of CRDs: - -```shell -kubectl get crds -l duck.knative.dev/source=true -``` - -CRDs SHOULD be added to the `sources` category: - -```yaml -spec: - names: - categories: - - sources -``` - -By adding to the sources category, we give an easy way to list running sources -in the cluster with: - -```shell -kubectl get sources -``` - -Source CRDs SHOULD provide additional printer columns to provide useful feedback -to cluster operators. For example, if the resource is long-lived, it would be a -good choice to show the `Ready` status and `Reason`, as well as the `Age` of the -resource. - -```yaml -additionalPrinterColumns: - - name: Age - type: date - JSONPath: .metadata.creationTimestamp - - name: Ready - type: string - JSONPath: '.status.conditions[?(@.type=="Ready")].status' - - name: Reason - type: string - JSONPath: '.status.conditions[?(@.type=="Ready")].reason' -``` - -### Source Validation - -Sources MUST implement conditions with a `Ready` condition for long lived -sources, and `Succeeded` for batch style sources. - -Sources MUST propagate the `sinkUri` to their status to signal to the cluster -where their events are being sent. - -Knative has standardized on the following minimum OpenAPI definition of `status` -for Source CRDs: - -```yaml -validation: - openAPIV3Schema: - properties: - status: - type: object - properties: - conditions: - type: array - items: - type: object - properties: - lastTransitionTime: - type: string - message: - type: string - reason: - type: string - severity: - type: string - status: - type: string - type: - type: string - required: - - type - - status - sinkUri: - type: string -``` - -Please see -[SourceStatus](https://pkg.go.dev/github.com/knative/pkg/apis/duck/v1#SourceStatus) -and [Condition](https://pkg.go.dev/github.com/knative/pkg/apis/#Condition) for -more details. - -Sources SHOULD provide OpenAPI validation for the `spec` field. At minimum -sources SHOULD have the following, - -```yaml -validation: - openAPIV3Schema: - properties: - spec: - type: object - required: - - sink - properties: - sink: - type: object - description: - "Reference to an object that will resolve to a domain name to use - as the sink." - ceOverrides: - type: object - description: - "Defines overrides to control modifications of the event sent to - the sink." - properties: - extensions: - type: object - description: - "Extensions specify what attributes are added or overridden on - the outbound event. Each `Extensions` key-value pair are set - on the event as an extension attribute independently." -``` - -### Container Runtime Contract - -#### SinkBinding - -SinkBinding augments the runtime contract of the subject's containers in the -following ways: - -- `status.sinkUri` (resolved from the `duckv1.SourceSpec`’s Sink) is bound into - the subject’s containers as the environment variable `K_SINK`. -- `spec.ceOverrides` (`duckv1.CloudEventOverrides`) is converted into JSON and - is bound into the subject’s containers as the environment variable - `K_CE_OVERRIDES`. - -The URI that is provided by `K_SINK` is the intended target of CloudEvents -produced by the subject. - -`K_CE_OVERRIDES` augments the outbound CloudEvent sent by the subject. - -- `spec.ceOverrides.extensions` is a map of attribute name to value that should - be added or overridden on the outbound event. - -### Source Registry - -Source CRDs SHOULD use a standard annotation to expose the types of events they -can emit in order to ease discoverability. - -```yaml -apiVersion: apiextensions.k8s.io/v1beta1 -kind: CustomResourceDefinition -metadata: - annotations: - registry.knative.dev/eventTypes: | # <-- optional but encouraged. - [ - { "type": "foo", "schema": "foo-schema", "description" : "Foo event description" }, - { "type": "bar", "schema": "bar-schema", "description" : "Bar event description" }, - ... - ] -``` - -If specified, the annotation MUST be a valid JSON array so that it can be easily -unmarshalled by tooling (e.g., a CLI). In particular, each object in the array -MUST contain the following fields: - -- type: String. Refers to the - [CloudEvents type](https://github.com/cloudevents/spec/blob/v1.0-rc1/spec.md#type) - attribute. Mandatory. -- schema: String. Refers to the - [CloudEvents dataschema](https://github.com/cloudevents/spec/blob/v1.0-rc1/spec.md#dataschema) - attribute. Optional. -- description: String describing the event. Optional. - -### Source RBAC - -Sources are expected to be extensions onto Kubernetes. To prevent cluster -operators from duplicating RBAC for all accounts that will interact with -sources, cluster operators should leverage aggregated RBAC roles to dynamically -update the rights of controllers that are using common service accounts provided -by Knative. This allows Eventing controllers to check the status of source type -resources without being aware of the exact source type or implementation at -compile time. - -The -[`source-observer` ClusterRole](https://github.com/knative/eventing/blob/main/config/200-source-observer-clusterrole.yaml) -looks like: - -```yaml -# Use this aggregated ClusterRole when you need read "Sources". -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: source-observer -aggregationRule: - clusterRoleSelectors: - - matchLabels: - duck.knative.dev/source: "true" # Matched by source-observer ClusterRole -rules: [] # Rules are automatically filled in by the controller manager. -``` - -And new sources MUST include a ClusterRole as part of installing themselves into -a cluster: - -```yaml -kind: ClusterRole -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: foos-source-observer - labels: - duck.knative.dev/source: "true" -rules: - - apiGroups: - - example.com - resources: - - foos - verbs: - - get - - list - - watch -``` - -## Source Custom Objects - -All Source Custom Objects MUST implement the -[Source](https://pkg.go.dev/github.com/knative/pkg/apis/duck/v1#Source) -ducktype. Additional data in spec and status is explicitly permitted. - -### duck.Spec - -The `spec` field is expected to have the following minimum shape: - -```go -type SourceSpec struct { - // Sink is a reference to an object that will resolve to a uri to use as the sink. - Sink duckv1.Destination `json:"sink,omitempty"` - - // CloudEventOverrides defines overrides to control the output format and - // modifications of the event sent to the sink. - // +optional - CloudEventOverrides *CloudEventOverrides `json:"ceOverrides,omitempty"` -} -``` - -For a golang structure definition of `Sink` and `CloudEventsOverrides`, please -see -[Destination](https://pkg.go.dev/github.com/knative/pkg/apis/duck/v1?tab=doc#Destination), -and -[CloudEventOverrides](https://pkg.go.dev/github.com/knative/pkg/apis/duck/v1#CloudEventOverrides). - -### duck.Status - -The `status` field is expected to have the following minimum shape: - -```go -type SourceStatus struct { - // inherits duck/v1 Status, which currently provides: - // * ObservedGeneration - the 'Generation' of the Service that was last - // processed by the controller. - // * Conditions - the latest available observations of a resource's current - // state. - Status `json:",inline"` - - // SinkURI is the current active sink URI that has been configured for the - // Source. - // +optional - SinkURI *apis.URL `json:"sinkUri,omitempty"` -} -``` - -For a full definition of `Status` and `SinkURI`, please see -[Status](https://pkg.go.dev/github.com/knative/pkg/apis/duck/v1#Status), and -[URL](https://pkg.go.dev/knative.dev/pkg/apis#URL). - -### EventType Registry - -Upon instantiation of a Source Custom Object, a controller (potentially the -source controller) SHOULD realize the -[EventType(s)](https://pkg.go.dev/knative.dev/eventing/pkg/apis/eventing/v1beta1#EventType) -this instantiation brings onto the eventing mesh. For a more detailed -description, please refer to the -[Event Registry](https://knative.dev/docs/eventing/event-registry/) -documentation. - -## Source Event delivery - -Sources SHOULD produce CloudEvents. The output SHOULD be via the HTTP binding -specified in one of the following versions of the specification: - -- [CloudEvents 0.3 specification](https://github.com/cloudevents/spec/blob/v0.3/http-transport-binding.md) -- [CloudEvents 1.0 specification](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md) - -The usage of CloudEvents version `1.0` is RECOMMENDED. - -Every Source SHOULD support sending events via _Binary Content Mode_ or -_Structured Content Mode_ of the HTTP Protocol Binding for CloudEvents. Sources -SHOULD send events to its -[Destination](https://pkg.go.dev/github.com/knative/pkg/apis/duck/v1?tab=doc#Destination). - -For more details of the Knative Event delivery, take a look at its -[specification](https://github.com/knative/eventing/blob/main/docs/delivery/README.md). diff --git a/specs/eventing/spec.md b/specs/eventing/spec.md deleted file mode 100644 index ad5f74866..000000000 --- a/specs/eventing/spec.md +++ /dev/null @@ -1,382 +0,0 @@ -# Object Model - -Knative ships with several objects which implement useful sets of -[interfaces](interfaces.md). It is expected that additional objects which -implement these interfaces will be added over time. For context, see the -[overview](overview.md) and [motivations](motivation.md) sections. - -These are Kubernetes resources that been introduced using Custom Resource -Definitions. They will have the expected _ObjectMeta_, _Spec_, _Status_ fields. -This document details our _Spec_ and _Status_ customizations. - -- [Trigger](#kind-trigger) -- [Broker](#kind-broker) -- [Channel](#kind-channel) -- [Subscription](#kind-subscription) - -## kind: Trigger - -### group: eventing.knative.dev/v1 - -_A `Trigger` represents a request to have events delivered to a subscriber from -a broker's event pool._ - -### Object Schema - -#### Metadata - -Standard Kubernetes -[meta.v1/ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#objectmeta-v1-meta) -resource. - -#### Spec - -The `TriggerSpec` defines the desired state for the `Trigger`. - -| Field Name | Field Type | Requirement | Description | Default Value | -| ------------ | ------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------- | ----------------------- | -| `broker` | `string` | Required | The broker that this trigger receives events from. | 'default' | -| `filter` | [`TriggerFilter`](#triggerfilter) | Optional | The filter to apply against all events from the broker. Only events that pass this filter will be sent to the subscriber. | subscribe to all events | -| `subscriber` | [`duckv1.Destination`](#duckv1.destination) | Required | The addressable that receives events from the broker that pass the filter. | | - -#### Status - -The `TriggerStatus` represents the current state of the `Trigger`. - -| Field Name | Field Type | Requirement | Description | Constraints | -| -------------------- | ------------------------------------- | ----------- | -------------------------------------------------------------------------------------- | ----------- | -| `observedGeneration` | `int` | Optional | The 'Generation' of the `Trigger` that was last processed by the controller. | | -| `conditions` | [`[]apis.Condition`](#apis.condition) | Optional | Trigger conditions. The latest available observations of the resource's current state. | | -| `annotations` | `map[string]string` | Optional | Fields to save additional state as well as convey more information to the user. | | -| `subscriberURI` | [`apis.URL`](#apis.url) | Required | The resolved address of the receiver for this trigger. | | - -##### Conditions - -- **Ready.** True when the trigger is provisioned and ready to deliver events to - the subscriber. - ---- - -## kind: Broker - -### group: eventing.knative.dev/v1 - -_A `Broker` collects a pool of events that are consumable using triggers. -Brokers provide a discoverable endpoint (`status.address`) for event delivery -that senders can use with minimal knowledge of the event routing strategy. -Subscribers use triggers to request delivery of events from a broker's pool to a -specific URL or `Addressable` endpoint._ - -### Object Schema - -#### Metadata - -Standard Kubernetes -[meta.v1/ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#objectmeta-v1-meta) -resource. - -#### Spec - -The `BrokerSpec` defines the desired state for the `Broker`. - -| Field Name | Field Type | Requirement | Description | Constraints | -| ---------- | ------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------ | ----------- | -| `config` | [`duckv1.KReference`](#duckv1.kreference) | Optional | Reference to the configuration options for this broker. For example, this could be a pointer to a ConfigMap. | | -| `delivery` | [`eventingduckv1.DeliverySpec`](#eventingduckv1.deliveryspec) | Optional | The delivery specification for events within the broker. This includes things like retries, DLQ, etc. | | - -#### Status - -The `BrokerStatus` represents the current state of the `Broker`. - -| Field Name | Field Type | Requirement | Description | Constraints | -| -------------------- | ------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------- | ----------- | -| `observedGeneration` | `int` | Optional | The 'Generation' of the `Broker` that was last processed by the controller. | | -| `conditions` | [`[]apis.Condition`](#apis.condition) | Optional | Broker conditions. The latest available observations of the resource's current state. | | -| `annotations` | `map[string]string` | Optional | Fields to save additional state as well as convey more information to the user. | | -| `address` | [`duckv1.Addressable`](#duckv1.addressable) | Required | The exposed endpoint URI for getting events delivered into the broker. The broker is [`Addressable`](#duckv1.addressable). | | - -##### Conditions - -- **Ready.** True when the broker is provisioned and ready to accept events. - ---- - -## kind: Channel - -### group: messaging.knative.dev/v1 - -_A Channel logically receives events on its input domain and forwards them to -its subscribers. A custom channel implementation (other than the default -\_InMemoryChannel_) can be referenced via the _channelTemplate_. The concrete -channel CRD can also be instantiated directly.\_ - -### Object Schema - -#### Metadata - -Standard Kubernetes -[meta.v1/ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#objectmeta-v1-meta) -resource. - -#### Spec - -The `ChannelSpec` defines which subscribers have expressed interest in receiving -events from this `Channel`. It also defines the ChannelTemplate to use in order -to create the CRD Channel backing this Channel. - -| Field Name | Field Type | Description | Constraints | -| ----------------- | --------------------------------------------- | ----------------------------------- | ------------------------- | -| `channelTemplate` | [`ChannelTemplateSpec`](#ChannelTemplateSpec) | Specifies which channel CRD to use. | Immutable after creation. | - -#### Status - -The `ChannelStatus` represents the current state of the `Channel`. - -| Field Name | Field Type | Requirement | Description | Constraints | -| -------------------- | ------------------------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------- | ----------- | -| `observedGeneration` | `int` | Optional | The 'Generation' of the `Channel` that was last processed by the controller. | | -| `conditions` | [`[]apis.Condition`](#apis.condition) | Optional | Channel conditions. The latest available observations of the resource's current state. | | -| `annotations` | `map[string]string` | Optional | Fields to save additional state as well as convey more information to the user. | | -| `address` | [`duckv1.Addressable`](#duckv1.addressable) | Required | Address of the endpoint (as an URI) for getting events delivered into the channel. | | -| `subscribers` | [`[]SubscriberStatus`](#subscriberstatus) | Required | The list of statuses for each of the channel's subscribers. | | -| `deadLetterChannel` | [`duckv1.KReference`](#duckv1.kreference) | Optional | Reference set by the channel when it supports native error handling via a channel. Failed messages are delivered here. | | -| `channel` | [`duckv1.KReference`](#duckv1.kreference) | Required | Reference to the `Channel` CRD backing this channel. | | - -##### Conditions - -- **Ready.** True when the channel is ready to accept events. - -### Life Cycle - -| Action | Reactions | Constraints | -| ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | -| Create | The `Channel` referenced will take ownership of the concrete `Channel` and begin provisioning the backing resources required for the `Channel` depending on implementation. | Only one Channel is allowed to be the Owner for a given Channel implementation (own CRD). | -| Update | The `Channel` will synchronize the `Channel` backing resources to reflect the update. | | -| Delete | The `Channel` will deprovision the backing resources if no longer required depending on the backing Channel implementation. | | - ---- - -## kind: Subscription - -### group: messaging.knative.dev/v1 - -_Subscription routes events received on a Channel to a DNS name._ - -### Object Schema - -#### Metadata - -Standard Kubernetes -[meta.v1/ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#objectmeta-v1-meta) -resource. - -#### Spec - -The `SubscriptionSpec` specifies the `Channel` for incoming events, a subscriber -target for processing those events and where to put the result of this -processing. Only the channel where the events are coming from is always -required. You can optionally only process the events (results in no output -events) by leaving out the result. You can also perform an identity -transformation on the incoming events by leaving out the subscriber and only -specifying the result. - -| Field Name | Field Type | Requirement | Description | Constraints | -| ------------ | ------------------------------------------------------------- | ------------------------- | ---------------------------------------------------------------------------------------- | ----------------------------- | -| `channel` | [`corev1.ObjectReference` ](#corev1.ObjectReference) | Required | Reference to a channel that will be used to create the subscription. | Must be a Channel. Immutable. | -| `subscriber` | [`duckv1.Destination`](#duckv1.Destination) | Required if no reply | Optional function for processing events. The result of subscriber will be sent to reply. | | -| `reply` | [`duckv1.Destination`](#duckv1.Destination) | Required if no subscriber | Optionally specifies how to handle events returned from the subscriber target. | | -| `delivery` | [`eventingduckv1.DeliverySpec`](#eventingduckv1.DeliverySpec) | Optional | Delivery configuration. | | - -##### Owner References - -- If a resource controller created this Subscription: Owned by the originating - resource. - -#### Status - -The `SubscriptionStatus` represents the current state of the `Subscription`. - -| Field Name | Field Type | Requirement | Description | Constraints | -| ---------------------- | ----------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------- | ----------- | -| `observedGeneration` | `int` | Optional | The 'Generation' of the `Subscription` that was last processed by the controller. | | -| `conditions` | [`[]apis.Condition`](#apis.condition) | Optional | Subscription conditions. The latest available observations of the resource's current state. | | -| `annotations` | `map[string]string` | Optional | Fields to save additional state as well as convey more information to the user. | | -| `physicalSubscription` | [`SubscriptionStatusPhysicalSubscription`](#SubscriptionStatusPhysicalSubscription) | Required | The fully resolved values that this `Subscription` represents. | | - -##### Conditions - -- **Ready.** True when the subscription is ready to deliver events to the - subscriber. - -#### Events - -- PublisherAcknowledged -- ActionFailed - -### Life Cycle - -| Action | Reactions | Constraints | -| ------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -| Create | The subscription controller adds the resolved URIs of `subscriber` and `reply` to the `subscribers` field in the `channel` _Subscribable_ resource. | | -| Update | | | -| Delete | | | - ---- - -## Shared Object Schema - -### duckv1.Addressable - -`Addressable` provides a generic mechanism for a custom resource definition to -indicate a destination for message delivery. `Addressable` is the schema for the -destination information. This is typically stored in the object's `status`, as -this information may be generated by the controller. More details in -[`Addressable` interface](interfaces.md#addressable). - -| Field Name | Field Type | Description | Constraints | -| ---------- | ----------------------- | ------------------------------------------- | --------------- | -| `url` | [`apis.URL`](#apis.url) | The URI name of the endpoint for the reply. | Must be an URL. | - -### apis.URL - -`URL` is an alias of [`url.URL`](https://golang.org/pkg/net/url/#URL) . It has -custom JSON marshal methods that enable it to be used in Kubernetes CRDs such -that the CRD resource will have the URL, but operator code can work with the -`url.URL` struct. - -### duckv1.KReference - -`KReference` contains enough information to refer to another object. - -| Field Name | Field Type | Requirement | Description | Default Value | Constraints | -| ------------ | ---------- | ----------- | ------------------------------------------------------------------------------------------------------------------------ | ------------------------------------ | ----------- | -| `kind` | `string` | Required | [Kind of the referent.](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds) | | | -| `namespace` | `string` | Optional | [Namespace of the referent.](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) | defaulted to the object embedding it | | -| `name` | `string` | Required | [Name of the referent.](https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names) | | | -| `apiVersion` | `string` | Required | API version of the referent. | | | - -### duckv1.Destination - -`Destination` represents a target of an invocation over HTTP. - -| Field Name | Field Type | Description | Constraints | -| ----------------- | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- | -| `ref`1 | [`duckv1.KReference`](#duckv1.kreference) | Reference to an [`duckv1.Addressable`](#duckv1.addressable). | Must adhere to [`duckv1.Addressable`](#duckv1.addressable). | -| `uri`1 | [`apis.URL`](#apis.url) | Either an absolute URL (non-empty scheme and non-empty host) pointing to the target or a relative URI. The relative URIs will be resolved using the base URI retrieved from `ref`. | Must be an URL. | - -1: One or both (ref, uri), Required. If only uri is specified, it must be an -absolute URL. If both are specified, uri will be resolved using the base URI -retrieved from ref. - -### eventingduckv1.DeliverySpec - -`DeliverySpec` contains the delivery options for event senders. - -| Field Name | Field Type | Requirement | Description | Constraints | -| ---------------- | ------------------------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -| `deadLetterSink` | [`duckv1.Destination`](#duckv1.Destination) | Optional | The sink receiving event that could not be sent to a `Destination`. | | -| `retry` | `int` | Optional | The minimum number of retries the sender should attempt when sending an event before moving it to the dead letter sink (when specified) or discarded (otherwise). | | -| `backoffPolicy` | `string` | Optional | The retry backoff policy (`linear` or `exponential`). | | -| `backoffDelay` | `string` | Optional | For linear policy, backoff delay is backoffDelay\*\. For exponential policy, backoff delay is backoffDelay\*2^\. | | - -### SubscriberStatus - -`SubscriberStatus` defines the status of a single subscriber to a `Channel`. - -| Field Name | Field Type | Requirement | Description | Constraints | -| -------------------- | ------------------------ | ----------- | ------------------------------------------------------------ | ----------- | -| `uid` | `types.UID` | Optional | UID is used to understand the origin of the subscriber. | | -| `observedGeneration` | `int` | Optional | Generation of the origin of the subscriber with uid:UID. | | -| `ready` | `corev1.ConditionStatus` | Required | Status of the subscriber. | | -| `message` | `string` | Optional | A human readable message indicating details of Ready status. | | - -### SubscriptionStatusPhysicalSubscription - -`SubscriptionStatusPhysicalSubscription` represents the fully resolved values -for a `Subscription`. - -| Field Name | Field Type | Requirement | Description | Constraints | -| ------------------- | ----------------------- | ----------- | -------------------------------------------------------------- | ----------- | -| `subscriberURI` | [`apis.URL`](#apis.url) | Required | The fully resolved URI for `spec.subscriber`. | | -| `replyURI` | [`apis.URL`](#apis.url) | Required | The fully resolved URI for the `spec.reply`. | | -| `deadLetterSinkURI` | [`apis.URL`](#apis.url) | Required | The fully resolved URI for the `spec.delivery.deadLetterSink`. | | - -### ChannelTemplateSpec - -| Field Name | Field Type | Requirement | Description | Constraints | -| ------------ | ---------------------- | ----------- | ------------------------------------------------------------ | ----------- | -| `kind` | `string` | Optional | The backing channel CRD | | -| `apiVersion` | `string` | Optional | API version of backing channel CRD | | -| `spec` | `runtime.RawExtension` | Optional | Spec to be passed verbatim to backing channel implementation | | - -### TriggerFilter - -| Field Name | Field Type | Requirement | Description | Constraints | -| ------------ | ------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -| `attributes` | `map[string]string` | Optional | A map of context attribute names to values for filtering by equality. Each key in the map is compared with the equivalent key in the event context. An event passes the filter if all values are equal to the specified values. The value '' to indicate all strings match. | Nested context attributes are not supported as keys. Only string values are supported. Only exact matches will pass the filter. | - -### apis.Condition - -The Knative API uses the -[Kubernetes Conditions convention](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) -to communicate errors and problems to the user. Each user-visible resource -described in Resource Overview MUST have a `conditions` field in `status`, which -must be a list of `Condition` objects of the following form (note that the -actual API object types may be named `FooCondition` to allow better code -generation and disambiguation between similar fields in the same `apiGroup`): - -| Field Name | Field Type | Description | Default Value | -| -------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -| `type` | `string` | The category of the condition, as a short, CamelCase word or phrase.

This is the primary key of the Conditions list when viewed as a map. | REQUIRED – No default | -| `status` | Enum:

  • "True"
  • "False"
  • "Unknown"
| The last measured status of this condition. | "Unknown" | -| `reason` | `string` | One-word CamelCase reason for the condition's last transition. | "" | -| `message` | `string` | Human-readable sentence describing the last transition. | "" | -| `severity` | Enum:
  • ""
  • "Warning"
  • "Info"
| If present, represents the severity of the condition. An empty severity represents a severity level of "Error". | "" | -| `lastTransitionTime` | `Timestamp` | Last update time for this condition. | "" – may be unset | - -Additionally, the resource's `status.conditions` field MUST be managed as -follows to enable clients (particularly user interfaces) to present useful -diagnostic and error message to the user. In the following section, conditions -are referred to by their `type` (aka the string value of the `type` field on the -Condition). - -1. Each resource MUST have either a `Ready` condition (for ongoing systems) or - `Succeeded` condition (for resources that run to completion) with - `severity=""`, which MUST use the `True`, `False`, and `Unknown` status - values as follows: - - 1. `False` MUST indicate a failure condition. - 1. `Unknown` SHOULD indicate that reconciliation is not yet complete and - success or failure is not yet determined. - 1. `True` SHOULD indicate that the application is fully reconciled and - operating correctly. - - `Unknown` and `True` are specified as SHOULD rather than MUST requirements - because there may be errors which prevent serving which cannot be determined - by the API stack (e.g. DNS record configuration in certain environments). - Implementations are expected to treat these as "MUST" for factors within the - control of the implementation. - -1. For non-`Ready` conditions, any conditions with `severity=""` (aka "Error - conditions") must be aggregated into the "Ready" condition as follows: - - 1. If the condition is `False`, `Ready` MUST be `False`. - 1. If the condition is `Unknown`, `Ready` MUST be `False` or `Unknown`. - 1. If the condition is `True`, `Ready` may be any of `True`, `False`, or - `Unknown`. - - Implementations MAY choose to report that `Ready` is `False` or `Unknown` - even if all Error conditions report a status of `True` (i.e. there may be - additional hidden implementation conditions which feed into the `Ready` - condition which are not reported.) - -1. Non-`Ready` conditions with non-error severity MAY be surfaced by the - implementation. Examples of `Warning` or `Info` conditions could include: - missing health check definitions, scale-to-zero status, or non-fatal - capacity limits. - -Conditions type names should be chosen to describe positive conditions where -`True` means that the condition has been satisfied. Some conditions may be -transient (for example, `ResourcesAllocated` might change between `True` and -`False` as an application scales to and from zero). It is RECOMMENDED that -transient conditions be indicated with a `severity="Info"`. From 4964fd790485e3301f4e88f86283b7dd1d9b14d3 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Sun, 9 May 2021 14:46:02 -0700 Subject: [PATCH 11/46] Updates from n3wscott review --- specs/eventing/data-plane.md | 129 ++--------------------------------- specs/eventing/motivation.md | 2 +- 2 files changed, 6 insertions(+), 125 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 63a1f2c42..f54744ea0 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -45,14 +45,14 @@ followed (with the CloudEvents bindings preferred in the case of conflict). The current version of this document does not describe protocol negotiation or the ability to upgrade an HTTP 1.1 event delivery into a more efficient protocol such as GRPC, AMQP or the like. It is expected that a future compatible version -of this specification may describe a protocol negotiation mechanism. +of this specification might describe a protocol negotiation mechanism. ## Event Delivery -To provide simpler support for event sources which may be translating events +To provide simpler support for event sources which might be translating events from existing systems, some data plane requirements for senders are relaxed in the general case. In the case of Knative Eventing provided resources (Channels -and Brokers) which implement these roles, requirements may be increased from +and Brokers) which implement these roles, requirements are increased from SHOULD to MUST. These cases are called out as they occur. ### Minimum supported protocol @@ -69,7 +69,7 @@ cleartext (`http`) and TLS (`https`) URLs as event delivery destinations. In the absence of specific delivery preferences, the sender MUST initiate delivery of the event to the recipient using the HTTP POST verb, using either the structured or binary encoding of the event (sender's choice). This delivery -SHOULD be performed using the CloudEvents HTTP Binding, version 1. +SHOULD be performed using the CloudEvents HTTP Binding, version 1.0. Senders MAY probe the recipient with an [HTTP OPTIONS request](https://tools.ietf.org/html/rfc7231#section-4.3.7); if implemented, the @@ -94,7 +94,7 @@ retriable error. | other `2xx` | (Unspecified) | Yes\* | No\* | Yes\* | | other `3xx` | (Unspecified) | Yes\* | No\* | Yes\* | | `400` | Unparsable event | No | No | Yes | -| `404` | Endpoint does not exist | No | No | Yes | +| `404` | Endpoint does not exist | Yes | No | Yes | | other `4xx` | Error | Yes | No | Yes | | other `5xx` | Error | Yes | No | Yes | @@ -165,122 +165,3 @@ reply event in the absence of a `Prefer: reply` header, the sender SHOULD treat the event as accepted, and MAY log an error about the unexpected payload. The sender MUST NOT process the reply event if it did not advertise the `Prefer: reply` capability. - - - -================================================================ - CUT BELOW -================================================================ - - -## Data plane contract for Sinks - -A **Sink** MUST be able to handle duplicate events. - -A **Sink** is an [_addressable_](./interfaces.md#addressable) resource that -takes responsibility for the event. A Sink could be a consumer of events, or -middleware. A Sink MUST be able to receive CloudEvents over HTTP and HTTPS. - -A **Sink** MAY be [_callable_](./interfaces.md#callable) resource that -represents an Addressable endpoint which receives an event as input and -optionally returns an event to forward downstream. - -Almost every component in Knative Eventing may be a Sink providing -composability. - -Every Sink MUST support HTTP Protocol Binding for CloudEvents -[version 1.0](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md) -and -[version 0.3](https://github.com/cloudevents/spec/blob/v0.3/http-transport-binding.md) -with restrictions and extensions specified below. - -### HTTP Support - -This section adds restrictions on -[requirements in HTTP Protocol Binding for CloudEvents](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md#12-relation-to-http). - -Sinks MUST accept HTTP requests with POST method and MAY support other HTTP -methods. If a method is not supported Sink MUST respond with HTTP status code -`405 Method Not Supported`. Non-event requests (e.g. health checks) are not -constrained. - -The URL used by a Sink MUST correspond to a single, unique endpoint at any given -moment in time. This MAY be done via the host, path, query string, or any -combination of these. This mapping is handled exclusively by the -[Addressable control-plane](./interfaces.md#control-plane) exposed via the -`status.address.url`. - -If an HTTP request's URL does not correspond to an existing endpoint, then the -Sink MUST respond with `404 Not Found`. - -Every non-Callable Sink MUST respond with `202 Accepted` if the request is -accepted. - -If Sink is Callable it MAY respond with `200 OK` and a single event in the HTTP -response. A returned event is not required to be related to the received event. -The Callable should return a successful response if the event was processed -successfully. If there is no event to send back then Callable Sink MUST respond -with 2xx HTTP and with empty body. - -If a Sink receives a request and is unable to parse a valid CloudEvent, then it -MUST respond with `400 Bad Request`. - -### Content Modes Supported - -A Sink MUST support `Binary Content Mode` and `Structured Content Mode` as -described in -[HTTP Message Mapping section of HTTP Protocol Binding for CloudEvents](https://github.com/cloudevents/spec/blob/master/http-protocol-binding.md#3-http-message-mapping) - -A Sink MAY support `Batched Content Mode` but that mode is not used in Knative -Eventing currently (that may change in future). - -### Retries - -Sinks should expect that retries and accept possibility that duplicate events -may be delivered. - -### Error handling - -If Sink is not returning HTTP success header (200 or 202) then the event may be -sent again. If the event can not be delivered then some sources of events (such -as Knative sources, brokers or channels) MAY support -[dead letter sink or channel](https://github.com/knative/eventing/blob/main/docs/delivery/README.md) for events that can not be -delivered. - -### Observability - -CloudEvents received by Sink MAY have -[Distributed Tracing Extension Attribute](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md). - -### Event reply contract - -An event sender supporting event replies SHOULD include a `Prefer: reply` header -in delivery requests to indicate to the sink that event reply is supported. An -event sender MAY ignore an event reply in the delivery response if the -`Prefer: reply` header was not included in the delivery request. - -An example is that a Broker supporting event reply sends events with an -additional header `Prefer: reply` so that the sink connected to the Broker knows -event replies will be accepted. While a source sends events without the header, -in which case the sink may assume that any event reply will be dropped without -error or retry attempt. If a sink wishes to ensure the reply events will be -delivered, it can check for the existence of the `Prefer: reply` header in the -delivery request and respond with an error code if the header is not present. - -### Data plane contract for Sources - -See [Source Delivery specification](sources.md#source-event-delivery) -for details. - -### Data plane contract for Channels - -See [Channel Delivery specification](channel.md#data-plane) for details. - -### Data plane contract for Brokers - -See [Broker Delivery specification](broker.md) - -## Changelog - -- 2020-04-20: `0.13.x release`: initial version that documents common contract - for sinks, sources, channels and brokers. diff --git a/specs/eventing/motivation.md b/specs/eventing/motivation.md index 42492f8b5..44439748c 100644 --- a/specs/eventing/motivation.md +++ b/specs/eventing/motivation.md @@ -24,7 +24,7 @@ application designers: In order to enable loose coupling and late-binding of event producers and consumers, Knative Eventing utilizes and extends the [CloudEvents -specification](https:?/github.com/cloudevents/spec) as the data plane protocol +specification](https://github.com/cloudevents/spec) as the data plane protocol between components. Knative Eventing prioritizes at-least-once delivery semantics, using the CloudEvents HTTP POST (push) transport as a minimum common transport between components. From 62169690a04675569d8950dce292b42ff59342bf Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Mon, 10 May 2021 10:35:00 -0700 Subject: [PATCH 12/46] Update data-plane language per Ville's suggestions. --- specs/eventing/data-plane.md | 67 +++++++++++++++++------------------- 1 file changed, 32 insertions(+), 35 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index dd16b5f83..8c92bffa7 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -5,8 +5,8 @@ Late-binding event senders and receivers (composing applications using configuration) only works when all event senders and recipients speak a common protocol. In order to enable wide support for senders and receivers, Knative -Eventing extends the [CloudEvents HTTP -bindings](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) +Eventing extends the +[CloudEvents HTTP bindings](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) with additional semantics for the following reasons: - Knative Eventing aims to enable highly-reliable event processing workflows. As @@ -20,12 +20,11 @@ with additional semantics for the following reasons: - Knative Eventing assumes a sender-driven (push) event delivery system. That is, each event processor is actively responsible for an event until it is handled (or affirmatively delivered to all following recipients). - -- Knative Eventing aims to make writing [event - sources](./overview.md#event-source) and event-processing software easier to - write; as such, it imposes higher standards on system components like - [brokers](./overview.md#broker) and [channels](./overview.md#channel) than on - edge components. +- Knative Eventing aims to make writing + [event sources](./overview.md#event-source) and event-processing software + easier to write; as such, it imposes higher standards on system components + like [brokers](./overview.md#broker) and [channels](./overview.md#channel) + than on edge components. This contract defines a mechanism for a single event sender to reliably deliver a single event to a single recipient. Building from this primitive, chains of @@ -37,9 +36,9 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119. -When not specified in this document, the [CloudEvents HTTP bindings, version -1](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) and -[HTTP 1.1 protocol](https://tools.ietf.org/html/rfc7230) standards should be +When not specified in this document, the +[CloudEvents HTTP bindings, version 1](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) +and [HTTP 1.1 protocol](https://tools.ietf.org/html/rfc7230) standards should be followed (with the CloudEvents bindings preferred in the case of conflict). The current version of this document does not describe protocol negotiation or @@ -52,8 +51,9 @@ of this specification might describe a protocol negotiation mechanism. To provide simpler support for event sources which might be translating events from existing systems, some data plane requirements for senders are relaxed in the general case. In the case of Knative Eventing provided resources (Channels -and Brokers) which implement these roles, requirements are increased from -SHOULD to MUST. These cases are called out as they occur. +and Brokers) which implement these roles, requirements are increased from SHOULD +to MUST. Exceptions with stronger requirements for Channels and Brokers are +called out as they occur. ### Minimum supported protocol @@ -61,7 +61,7 @@ All senders and recipients MUST support the CloudEvents 1.0 protocol and the [binary](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#31-binary-content-mode) and [structured](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#32-structured-content-mode) -content modes of the CloudEvetns HTTP binding. Senders MUST support both +content modes of the CloudEvents HTTP binding. Senders MUST support both cleartext (`http`) and TLS (`https`) URLs as event delivery destinations. ### HTTP Verbs @@ -71,12 +71,12 @@ delivery of the event to the recipient using the HTTP POST verb, using either the structured or binary encoding of the event (sender's choice). This delivery SHOULD be performed using the CloudEvents HTTP Binding, version 1.0. -Senders MAY probe the recipient with an [HTTP OPTIONS -request](https://tools.ietf.org/html/rfc7231#section-4.3.7); if implemented, the -recipent MUST indicate support for the POST verb using the [`Allow` -header](https://tools.ietf.org/html/rfc7231#section-7.4.1). Senders which -receive an error when probing with HTTP OPTIONS SHOULD proceed using the HTTP -POST mechanism. +Senders MAY probe the recipient with an +[HTTP OPTIONS request](https://tools.ietf.org/html/rfc7231#section-4.3.7); if +implemented, the recipent MUST indicate support for the POST verb using the +[`Allow` header](https://tools.ietf.org/html/rfc7231#section-7.4.1). Senders +which receive an error when probing with HTTP OPTIONS SHOULD proceed using the +HTTP POST mechanism. ### Event Acknowledgement and Repeat Delivery @@ -111,9 +111,9 @@ be retried? What about `405` (Method Not Allowed), 413 (Payload Too Large), 414 Recipients MUST be able to handle duplicate delivery of events and MUST accept delivery of duplicate events, as the event acknowledgement could have been lost -in return to the sender. Event recipients MUST use the [`source` and `id` -attributes](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#required-attributes) -to detect duplicated events (see [observability](#observability) for an example +in return to the sender. Event recipients MUST use the +[`source` and `id` attributes](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#required-attributes) +to determine duplicate events (see [observability](#observability) for an example case where other event attributes may vary from one delivery attempt to another). @@ -121,21 +121,19 @@ Where possible, event senders SHOULD re-attempt delivery of events where the HTTP request failed or returned a retriable status code. It is RECOMMENDED that event senders implement some form of congestion control (such as exponential backoff) when managing retry timing. This specification does not document any -specific congestion control algorithm or -parameters. [Brokers](./overview.md#broker) and -[Channels](./overview.md#channel) MUST implement congestion control and MUST -implement retries. +specific congestion control algorithm or parameters. +[Brokers](./overview.md#broker) and [Channels](./overview.md#channel) MUST +implement congestion control and MUST implement retries. ### Observability Event senders MAY add or update CloudEvents attributes before sending to implement observability features such as tracing; in particular, the -[`traceparent` and `tracestate` distributed tracing -attributes](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md) +[`traceparent` and `tracestate` distributed tracing attributes](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md) may be modified in this way for each delivery attempt of the same event. This specification does not mandate any particular logging or metrics -aggregtion, nor a method of exposing observability information to users +aggregation, nor a method of exposing observability information to users configuring the resources. Platform administrators SHOULD expose event-delivery telemetry to users through platform-specific interfaces, but such interfaces are beyond the scope of this document. @@ -149,8 +147,8 @@ https://github.com/knative/specs/blob/main/specs/eventing/channel.md#observabili In some applications, an event receiver might emit an event in reaction to a received event. An event sender MAY document support for this pattern by including a `Prefer: reply` header in the HTTP POST request. This header -indicates to the event receiver that the caller will accept a [`200` -response](#event-acknowledgement-and-repeat-delivery) which includes a +indicates to the event receiver that the caller will accept a +[`200` response](#event-acknowledgement-and-repeat-delivery) which includes a CloudEvent encoded using the binary or structured formats. [Brokers](./overview.md#broker) and [Channels](./overview.md#channel) MUST indicate support for replies using the `Prefer: reply` header. @@ -166,6 +164,5 @@ likely a better choice. If a recipient chooses to reply to a sender with a `200` response code and a reply event in the absence of a `Prefer: reply` header, the sender SHOULD treat the event as accepted, and MAY log an error about the unexpected payload. The -sender MUST NOT process the reply event if it did not advertise the `Prefer: -reply` capability. - +sender MUST NOT process the reply event if it did not advertise the +`Prefer: reply` capability. From e776f367659a9b81d740a728042ebe4bd84793c4 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Mon, 10 May 2021 11:39:13 -0700 Subject: [PATCH 13/46] Overview revisions based on vaikas feedback. --- specs/eventing/overview.md | 95 ++++++++++++++++++++++---------------- 1 file changed, 54 insertions(+), 41 deletions(-) diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index e0b63de5f..3e159d326 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -1,14 +1,17 @@ # Resource Types The Knative Eventing API provides primitives for two common event-processing -patterns (credit to James Urquhart for the formulation): +patterns: -* Point-to-point asynchronous communication ([`messaging.knative.dev`](#messaging)) +- Topology-based event routing ([`messaging.knative.dev`](#messaging)) -* Content-based event routing ([`eventing.knative.dev`](#eventing)) +- Content-based event routing ([`eventing.knative.dev`](#eventing)) -The other two patterns James identifies are log-stream processing and complex -workflows; these are not currently addressed by Knative Eventing. +Knative Eventing does not directly specify mechanisms for other event-processing +models, including multi-stage workflows, corellated request-reply, and +sequential (windowed) event processing; these models could be built using the +primitives provided by Knative, or Knative could deliver events to an external +system that implements these models. In addition to the primitives needed to express the above patterns, Knative Eventing defines two [_interface contracts_](#interface-contracts) to allow @@ -17,6 +20,42 @@ to the core primitives. +## Interface Contracts + +In addition to the concrete types described below in the `messaging.knative.dev` +and `eventing.knative.dev` API groups, Knative Eventing supports referencing +objects in other API groups as destinations for event delivery. This is done by +defining partial schemas which the other resources must support. The following +interface contracts define object fragments and partial schemas (required fields +on an arbitrary API object) which form a basis for Knative Eventing. + +### Addressable + +**Addressable** resources expose a resource address (HTTP URL) in their `status` +object. The URL is used as a destination for delivery of events to the resource; +the exposed URL must implement the [data plane contract](data-plane.md) for +receiving events. + +[**Broker**](#broker) and [**Channel**](#channel) both implement **Addressable** +to receive events from other components. + +### Destination + +**Destination** is an interface (object fragment) which is used consistently +through Knative Eventing to reference a message delivery destination. A +destination is typically a +[tagged union](https://en.wikipedia.org/wiki/Tagged_union) of different +addressing models; at a minimum, it supports a direct URL, a reference to an +**Addressable** object, or a Kubernetes Service (as a special case). + +### Event Source + +**Event Sources** are resources which generate events and may be configured to +deliver the events to a **Destiantion** designated by a `sink` object in the +resource's `spec`. The Knative Eventing spec does not define any specific event +sources, but does define common interfaces for discovering and managing event +sources. + ## Eventing ### Broker @@ -26,17 +65,18 @@ which event senders may use to submit events to the router. A Broker may be implemented using many different underlying event-forwarding mechanisms; the broker provides a small set of common event-delivery configuration options and may reference additional implementation-specific configuration options via a -reference to an external object; the format of the external objects is not +reference to an external object (either a kubernetes built-in like ConfigMap or +a custom object); the format of the external objects is intentionally not standardized. ### Trigger -**Trigger** defines a filtered delivery option to extract events delivered to a -**Broker** and route them to an **Addressable** destination. Trigger implements -uniform event filtering based on the CloudEvents attributes associated with the -event, ignoring the payload (which might be large and/or binary and need not be -parsed during event routing). The addressable interface contract allows Triggers -to deliver events to a variety of different destinations, including external +**Trigger** defines a filtered delivery option to select events delivered to a +**Broker** and route them to a **Destination**. Trigger implements uniform event +filtering based on the CloudEvents attributes associated with the event, +ignoring the payload (which might be large and/or binary and need not be parsed +during event routing). The addressable interface contract allows Triggers to +deliver events to a variety of different destinations, including external resources such as a virtual machine or SaaS service. ## Messaging @@ -57,32 +97,5 @@ environment). **Channel**. Events sent to a channel are delivered to _each_ subscription _independently_ -- a subscription maintains its own list of undelivered events and will manage retry indpendently of any other subscriptions to the same -channel. Like **Trigger**, subscriptions use the **Addressable** interface -contract to support event delivery to many different destination types. - -## Interface Contracts - -In addition to the concrete types described above in the `messaging.knative.dev` -and `eventing.knative.dev` API groups, Knative Eventing supports referencing -objects in other API groups as destinations for event delivery. This is done by -defining partial schemas which the other resources must support. The following -interface contracts define a set of expected resource fields on an referenced -resource. - -### Addressable - -**Addressable** resources expose a resource address (HTTP URL) in their `status` -object. The URL is used as a destination for delivery of events to the resource; -the exposed URL must implement the [data plane contract](data-plane.md) for -receiving events. - -**Broker** and **Channel** both implement **Addressable**. - -### Event Source - -**Event Sources** are resources which generate events and may be configured to -deliver the events to an **Addressable** resource designated by a `sink` object -in the resource's `spec`. The Knative Eventing spec does not define any specific -event sources, but does define common interfaces for discovering and managing -event sources. - +channel. Like **Trigger**, subscriptions use the **Destination** interface to +support event delivery to many different destination types. From 5d0c5b4f476a179a1a6ede3abaffb34dd27a0fd1 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Mon, 10 May 2021 16:58:32 -0700 Subject: [PATCH 14/46] Updates from comments in #25 --- specs/eventing/data-plane.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 8c92bffa7..4eedd6c64 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -37,7 +37,7 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", interpreted as described in RFC2119. When not specified in this document, the -[CloudEvents HTTP bindings, version 1](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) +[CloudEvents HTTP bindings, version 1.0](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) and [HTTP 1.1 protocol](https://tools.ietf.org/html/rfc7230) standards should be followed (with the CloudEvents bindings preferred in the case of conflict). @@ -81,7 +81,7 @@ HTTP POST mechanism. ### Event Acknowledgement and Repeat Delivery Event recipients MUST use the HTTP response code to indicate acceptance of an -event. The recipient MUST NOT return a response accepting the event until it has +event. The recipient SHOULD NOT return a response accepting the event until it has handled event (processed the event or stored it in stable storage). The following response codes are explicitly defined; event recipients MAY also respond with other response codes. A response code not in this table SHOULD be From 17a9ae17d40b771a89ba36f820e9d364f2d5224a Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Tue, 11 May 2021 00:15:15 -0700 Subject: [PATCH 15/46] Address remaining Scott and Ville comments. --- specs/eventing/control-plane.md | 150 ++++++++++++++++++-------------- 1 file changed, 85 insertions(+), 65 deletions(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index 863d4f003..6f53041ea 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -56,10 +56,10 @@ metadata: rules: - apiGroups: ["eventing.knative.dev"] resources: ["broker", "trigger"] - verbs: ["get", "list", "create", "update", "delete"] + verbs: ["get", "list", "create", "update", "patch", "delete"] - apiGroups: ["messaging.knative.dev"] resources: ["channel", "subscription"] - verbs: ["get", "list", "create", "update", "delete"] + verbs: ["get", "list", "create", "update", "patch", "delete"] ``` In order to support resolving resources which meet the @@ -223,10 +223,10 @@ Condition). operating correctly. `Unknown` and `True` are specified as SHOULD rather than MUST requirements - because there may be errors which prevent serving which cannot be determined - by the API stack (e.g. DNS record configuration in certain environments). - Implementations are expected to treat these as "MUST" for factors within the - control of the implementation. + because there may be errors which prevent functioning which cannot be + determined by the API stack (e.g. DNS record configuration in certain + environments). Implementations are expected to treat these as "MUST" for + factors within the control of the implementation. 1. For non-`Ready` conditions, any conditions with `severity=""` (aka "Error conditions") must be aggregated into the "Ready" condition as follows: @@ -262,20 +262,24 @@ on a set of attribute filters (Triggers). Triggers are associated with a Broker based on the `spec.broker` field on the Trigger; it is expected that the controller for a Broker will also control the associated Triggers. When the Broker's `Ready` condition is `true`, the Broker MUST provide a -`status.address.url` which accepts all CloudEvents and MUST forward the received -events to each associated Trigger whose `Ready` condition is `true`. As -described in the [Trigger Lifecycle](#trigger-lifecycle) section, a Broker MAY -forward events to an associated Trigger which which does not currently have a -`true` `Ready` condition, including events received by the Broker before the -Trigger was created. - -When a Broker is created, its `spec.class` field MUST be populated to indicate -which of several possible Broker implementations to use. It is RECOMMENDED to -default the `spec.class` field on creation if it is unpopulated. Once created, -the `spec.class` field MUST be immutable; the Broker must be deleted and -re-created to change the `spec.class`. This pattern is chosen to make it clear -that changing `spec.class` is not an atomic operation and that any -implementation would be likely to result in message loss during the transition. +`status.address.url` which accepts all valid CloudEvents and MUST forward the +received events for filtering to each associated Trigger whose `Ready` condition +is `true`. As described in the [Trigger Lifecycle](#trigger-lifecycle) section, +a Broker MAY forward events to an associated Trigger which which does not +currently have a `true` `Ready` condition, including events received by the +Broker before the Trigger was created. + +The annotation `eventing.knative.dev/broker.class` may be used to select a +particular implementation of a Broker. When a Broker is created, its +implementation and the `spec.config` field MUST be populated (`spec.config` MAY +be an empty broker) to indicate which of several possible Broker implementations +to use. It is RECOMMENDED to default the `eventing.knative.dev/broker.class` +field on creation if it is unpopulated. Once created, both fields MUST be +immutable; the Broker must be deleted and re-created to change the +implementation class or `spec.config`. This pattern is chosen to make it clear +that changing the implementation class or `spec.config` is not an atomic +operation and that any implementation would be likely to result in event loss +during the transition. ## Trigger Lifecycle @@ -293,7 +297,7 @@ Trigger MUST set the `Ready` condition to `false`, and at least one condition should indicate the reason for the error. If the Trigger's `spec.delivery.deadLetterSink` field it set, it MUST be -resolved to a URL and reported in `status.deadLetterSinkUri` in the same manner +resolved to a URI and reported in `status.deadLetterSinkUri` in the same manner as the `spec.subscriber` field before setting the `Ready` condition to `true`. Once created, the Trigger's `spec.broker` SHOULD NOT permit updates; to change @@ -309,7 +313,7 @@ Trigger or the Broker), the Trigger MUST only set the `Ready` condition to the `spec.filter` to the Trigger's `spec.subscriber`. The Broker MAY send some events to the Trigger's `spec.subscriber` prior to the Trigger's `Ready`condition being set to `true`. When a Trigger is deleted, the Broker MAY -send some additional events to the Trigger's `spec.subscriber` ftor tho +send some additional events to the Trigger's `spec.subscriber` after the deletion. ## Channel Lifecycle @@ -320,11 +324,11 @@ multiple recipients (Subscriptions). Subscriptions are associated with a Channel based on the `spec.channel` field on the Subscription; it is expected that the controller for a Channel will also control the associated Subscriptions. When the Channel's `Ready` condition is `true`, the Channel MUST provide a -`status.address.url` which accepts all CloudEvents and MUST forward the received -events to each associated Subscription whose `Ready` condition is `true`. As -described in the [Subscription Lifecycle](#subscription-lifecycle) section, a -Channel MAY forward events to an associated Subscription which does not -currently have a `true` `Ready` condition, including events received by the +`status.address.url` which accepts all valid CloudEvents and MUST forward the +received events to each associated Subscription whose `Ready` condition is +`true`. As described in the [Subscription Lifecycle](#subscription-lifecycle) +section, a Channel MAY forward events to an associated Subscription which does +not currently have a `true` `Ready` condition, including events received by the Channel before the `Subscription` was created. When a Channel is created, its `spec.channelTemplate` field MUST be populated to @@ -334,16 +338,16 @@ unpopulated. Once created, the `spec.channelTemplate` field MUST be immutable; the Channel MUST be deleted and re-created to change the `spec.channelTemplate`. This pattern is chosen to make it clear that changing `spec.channelTemplate` is not an atomic operation and that any implementation would be likely to result in -message loss during the transition. +event loss during the transition. ## Subscription Lifecycle The lifecycle of a Subscription is independent of that of the channel it refers to in its `spec.channel` field. The `spec.channel` object reference may refer to -either an `eventing.knative.dev/v1` Channel resource, or another resource which +either an `messaging.knative.dev/v1` Channel resource, or another resource which meets the `spec.subscribers` and `spec.delivery` required elements in the -Channellable duck type. If the referenced `spec.channel` does not currently -exist or its `Ready` condition is not `true`, then the Subscription's `Ready` +Channelable duck type. If the referenced `spec.channel` does not currently exist +or its `Ready` condition is not `true`, then the Subscription's `Ready` condition MUST NOT be `true`, and the reason SHOULD indicate that the corresponding channel is missing or not ready. @@ -357,7 +361,10 @@ condition should indicate the reason for the error. (It is acceptable for none of the `spec.subscriber`, `spec.reply`, and `spec.delivery.deadLetterSink` fields to contain a `ref` field.) - +At least one of `spec.subscriber` and `spec.reply` MUST be set; if only +`spec.reply` is set, the behavior is equivalent to setting `spec.subscriber` +except that the Channel SHOULD NOT advertise the ability to process replies +during the delivery. Once created, the Subscription's `spec.channel` SHOULD NOT permit updates; to change the `spec.channel`, the Subscription can be deleted and re-created. This @@ -372,7 +379,7 @@ condition to `true` after the channel has been configured to send all future events to the Subscriptions `spec.subscriber`. The Channel MAY send some events to the Subscription before prior to the Subscription's `Ready` condition being set to `true`. When a Subscription is deleted, the Channel MAY send some -additional events to the Subscription's `spec.subscriber`. +additional events to the Subscription's `spec.subscriber` after the deletion. - Recipients MUST be able to handle duplicate delivery of events and MUST accept delivery of duplicate events, as the event acknowledgement could have been lost in return to the sender. Event recipients MUST use the @@ -148,17 +148,18 @@ https://github.com/knative/specs/blob/main/specs/eventing/channel.md#observabili ### Derived (Reply) Events In some applications, an event receiver might emit an event in reaction to a -received event. An event sender MAY document support for this pattern by -including a `Prefer: reply` header in the HTTP POST request. This header -indicates to the event receiver that the caller will accept a +received event. Components MAY to choose to support this pattern by accepting an +encoded CloudEvent in the HTTP response. The sender SHOULD NOT assume that a +received reply event is directly related to the event sent in the HTTP request. + +An event sender MAY document support for this pattern by including a +`Prefer: reply` header in the HTTP POST request. This header indicates to the +event receiver that the caller will accept a [`200` response](#event-acknowledgement-and-repeat-delivery) which includes a CloudEvent encoded using the binary or structured formats. [Brokers](./overview.md#broker) and [Channels](./overview.md#channel) MUST indicate support for replies using the `Prefer: reply` header. -The sender SHOULD NOT assume that a received reply event is directly related to -the event sent in the HTTP request. - A recipient MAY reply to any HTTP POST with a `200` response to indicate that the event was processed successfully, with or without a response payload. If the recipient will _never_ provide a response payload, the `202` response code is diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index 476cb5148..112d063d4 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -45,7 +45,9 @@ to receive events from other components. through Knative Eventing to reference a event delivery destination. A Destination eventually resolves the supplied information to a URL, and may be a simple URL or relative to an **Addressable** object reference; it also supports -a Kubernetes Service object reference (as a special case). +a Kubernetes Service object reference (as a special case). An absolute URL in a +Destination may be used to reference cluster-external resources such as a +virtual machine or SaaS service. ### Event Source @@ -74,9 +76,8 @@ standardized. **Broker** and route them to a **Destination**. Trigger implements uniform event filtering based on the CloudEvents attributes associated with the event, ignoring the payload (which might be large and/or binary and need not be parsed -during event routing). The addressable interface contract allows Triggers to -deliver events to a variety of different destinations, including external -resources such as a virtual machine or SaaS service. +during event routing). The destination interface contract allows Triggers to +deliver events both cluster-local objects or external resources. ## Messaging From 917eac3ae902c3702885a895ab57f6e8cfc3b71b Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 20 May 2021 12:27:07 -0700 Subject: [PATCH 20/46] Updates from slinky, travis, ville, matzew --- specs/eventing/data-plane.md | 33 ++++++++++++++++++++------------- specs/eventing/overview.md | 4 ++-- 2 files changed, 22 insertions(+), 15 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 95d36f973..b54d1105b 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -45,9 +45,10 @@ and [HTTP 1.1 protocol](https://tools.ietf.org/html/rfc7230) standards should be followed (with the CloudEvents bindings preferred in the case of conflict). The current version of this document does not describe protocol negotiation or -the ability to upgrade an HTTP 1.1 event delivery into a more efficient protocol -such as GRPC, AMQP, or the like. It is expected that a future compatible version -of this specification might describe a protocol negotiation mechanism. +any delivery mechanism other than HTTP 1.1. Future versions may define protocol +negotiation to optimize delivery; compliant implementations SHOULD aim to +interoperate by ignoring unrecognized negotiation options (such as HTTP Upgrade +headers). ## Event Delivery @@ -95,11 +96,11 @@ treated as a retriable error. | Response code | Meaning | Retry | Delivery completed | Error | | ------------- | --------------------------------- | ----- | ------------------ | ----- | -| `1xx` | (Unspecified) | Yes\* | No\* | Yes\* | +| `1xx` | (Unspecified) | No\* | No\* | Yes\* | | `200` | [Event reply](#event-reply) | No | Yes | No | | `202` | Event accepted | No | Yes | No | -| other `2xx` | (Unspecified) | Yes\* | No\* | Yes\* | -| other `3xx` | (Unspecified) | Yes\* | No\* | Yes\* | +| other `2xx` | (Unspecified) | No\* | No\* | Yes\* | +| other `3xx` | (Unspecified) | No\* | No\* | Yes\* | | `400` | Unparsable event | No | No | Yes | | `404` | Endpoint does not exist | Yes | No | Yes | | `409` | Conflict / Processing in progress | Yes | No | Yes | @@ -112,13 +113,17 @@ extension**. Event recipients SHOULD NOT send these response codes in this spec version, but event senders MUST handle these response codes as errors and implement appropriate failure behavior. -Recipients MUST be able to handle duplicate delivery of events and MUST accept -delivery of duplicate events, as the event acknowledgement could have been lost -in return to the sender. Event recipients MUST use the +Recipients MUST be able to handle duplicate delivery of events (for example, via +idempotency or tracking event delivery state) and MUST accept delivery of +duplicate events, as the event acknowledgement could have been lost in return to +the sender. As specified in the +[CloudEvents specification](https://github.com/cloudevents/spec/blob/v1.0.1/primer.md#id), +event recipients MUST use the [`source` and `id` attributes](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#required-attributes) -to determine duplicate events (see [observability](#observability) for an -example case where other event attributes may vary from one delivery attempt to -another). +to determine duplicate events if needed. This specification does not describe +state requirements for clients which need to detect duplicate events. (see +[observability](#observability) for an example case where other event attributes +may vary from one delivery attempt to another). Where possible, event senders SHOULD re-attempt delivery of events where the HTTP request failed or returned a retryable status code. It is RECOMMENDED that @@ -132,7 +137,9 @@ implement congestion control and MUST implement retries. Event senders MAY add or update CloudEvents attributes before sending to implement observability features such as tracing; in particular, the -[`traceparent` and `tracestate` distributed tracing attributes](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md) +`traceparent` and `tracestate` distributed tracing attributes defined by +[W3C](https://www.w3.org/TR/trace-context/) and +[CloudEvents](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md) may be modified in this way for each delivery attempt of the same event. This specification does not mandate any particular logging or metrics diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index 112d063d4..f3cc4b689 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -26,7 +26,7 @@ In addition to the concrete types described below in the `messaging.knative.dev` and `eventing.knative.dev` API groups, Knative Eventing supports referencing objects in other API groups as destinations for event delivery. This is done by defining partial schemas which the other resources must support. The following -interface contracts define object fragments and partial schemas (required fields +interface contracts define resource fragments and partial schemas (required fields on an arbitrary API object) which form a basis for Knative Eventing. ### Addressable @@ -41,7 +41,7 @@ to receive events from other components. ### Destination -**Destination** is an interface (object fragment) which is used consistently +**Destination** is an interface (resource fragment) which is used consistently through Knative Eventing to reference a event delivery destination. A Destination eventually resolves the supplied information to a URL, and may be a simple URL or relative to an **Addressable** object reference; it also supports From 7bd01619805c442665bd4e93825b1cca031317f2 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 20 May 2021 14:24:24 -0700 Subject: [PATCH 21/46] Link to upgrade header. --- specs/eventing/data-plane.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index b54d1105b..19dc4c999 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -47,8 +47,8 @@ followed (with the CloudEvents bindings preferred in the case of conflict). The current version of this document does not describe protocol negotiation or any delivery mechanism other than HTTP 1.1. Future versions may define protocol negotiation to optimize delivery; compliant implementations SHOULD aim to -interoperate by ignoring unrecognized negotiation options (such as HTTP Upgrade -headers). +interoperate by ignoring unrecognized negotiation options (such as +[HTTP `Upgrade` headers](https://datatracker.ietf.org/doc/html/rfc7230#section-6.7)). ## Event Delivery From fae7a876fb4f81993b5157ff9cd0d025b52c9805 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Tue, 25 May 2021 11:52:22 -0700 Subject: [PATCH 22/46] Finish up TODOs, fix some typos. --- specs/eventing/data-plane.md | 10 +-- specs/eventing/images/eventing-overview.svg | 87 +++++++++++++++++++++ specs/eventing/overview.md | 34 +++++++- 3 files changed, 123 insertions(+), 8 deletions(-) create mode 100644 specs/eventing/images/eventing-overview.svg diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 19dc4c999..6b906d217 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -121,7 +121,7 @@ the sender. As specified in the event recipients MUST use the [`source` and `id` attributes](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#required-attributes) to determine duplicate events if needed. This specification does not describe -state requirements for clients which need to detect duplicate events. (see +state requirements for clients which need to detect duplicate events. (See [observability](#observability) for an example case where other event attributes may vary from one delivery attempt to another). @@ -140,7 +140,7 @@ implement observability features such as tracing; in particular, the `traceparent` and `tracestate` distributed tracing attributes defined by [W3C](https://www.w3.org/TR/trace-context/) and [CloudEvents](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md) -may be modified in this way for each delivery attempt of the same event. +MAY be modified in this way for each delivery attempt of the same event. This specification does not mandate any particular logging or metrics aggregation, nor a method of exposing observability information to users @@ -148,14 +148,10 @@ configuring the resources. Platform administrators SHOULD expose event-delivery telemetry to users through platform-specific interfaces, but such interfaces are beyond the scope of this document. - - ### Derived (Reply) Events In some applications, an event receiver might emit an event in reaction to a -received event. Components MAY to choose to support this pattern by accepting an +received event. Components MAY choose to support this pattern by accepting an encoded CloudEvent in the HTTP response. The sender SHOULD NOT assume that a received reply event is directly related to the event sent in the HTTP request. diff --git a/specs/eventing/images/eventing-overview.svg b/specs/eventing/images/eventing-overview.svg new file mode 100644 index 000000000..4ba72413f --- /dev/null +++ b/specs/eventing/images/eventing-overview.svg @@ -0,0 +1,87 @@ +eventing.knative.devmessaging.knative.devBrokerTriggerTriggerChannelSubscriptionSubscription«Knative Service»Addressable«Kubernetes Service»Addressablebrokerbrokersubscriberchannelchannelsubscriberreply \ No newline at end of file diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index f3cc4b689..f21fbe83f 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -18,7 +18,39 @@ Eventing defines two [_interface contracts_](#interface-contracts) to allow connecting multiple types of Kubernetes objects as event senders and recipients to the core primitives. - +![Overview of objects](images/eventing-overview.svg) + ## Interface Contracts From 9e33332dd3117a79ceb24081829aaa509c602b49 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Tue, 25 May 2021 15:05:55 -0700 Subject: [PATCH 23/46] Address a number of duglin comments --- specs/eventing/data-plane.md | 76 ++++++++++++++++++++---------------- specs/eventing/motivation.md | 2 +- specs/eventing/overview.md | 2 +- 3 files changed, 44 insertions(+), 36 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 6b906d217..d69dcfb8d 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -1,10 +1,23 @@ # Knative Eventing Data Plane Contract +## Terminology + +This document discusses communication between two parties: + +- **Event Senders** initiate an HTTP POST to deliver a CloudEvent. +- **Event Recipients** receive an HTTP POST and accept (or reject) a CloudEvent. + +Additionally, these roles may be combined in different ways: + +- **Event Processors** may be event senders, event recipients, or both. +- **Event Sources** are exclusively event senders, and never act as recipients. +- **Event Sinks** are exclusively event recipients, and never act as senders. + ## Introduction -Late-binding event senders and receivers (composing applications using +Late-binding event senders and recipients (composing applications using configuration) only works when all event senders and recipients speak a common -protocol. In order to enable wide support for senders and receivers, Knative +protocol. In order to enable wide support for senders and recipients, Knative Eventing extends the [CloudEvents HTTP bindings](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) with additional semantics for the following reasons: @@ -20,14 +33,13 @@ with additional semantics for the following reasons: languages. - Knative Eventing assumes a sender-driven (push) event delivery system. That - is, each event processor is actively responsible for an event until it is - handled (or affirmatively delivered to all following recipients). + is, each recipient is actively responsible for an event until it is handled + (or affirmatively delivered to all following recipients). -- Knative Eventing aims to make writing - [event sources](./overview.md#event-source) and event-processing software - easier to write; as such, it imposes higher standards on system components - like [brokers](./overview.md#broker) and [channels](./overview.md#channel) - than on edge components. +- Knative Eventing aims to make [event sources](./overview.md#event-source) and + event-processing software easier to write; as such, it imposes higher + standards on system components like [brokers](./overview.md#broker) and + [channels](./overview.md#channel) than on edge components. This contract defines a mechanism for a single event sender to reliably deliver a single event to a single recipient. Building from this primitive, chains of @@ -42,23 +54,17 @@ interpreted as described in RFC2119. When not specified in this document, the [CloudEvents HTTP bindings, version 1.0](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) and [HTTP 1.1 protocol](https://tools.ietf.org/html/rfc7230) standards should be -followed (with the CloudEvents bindings preferred in the case of conflict). +followed (with the CloudEvents bindings taking precedence in the case of +conflict). The current version of this document does not describe protocol negotiation or -any delivery mechanism other than HTTP 1.1. Future versions may define protocol -negotiation to optimize delivery; compliant implementations SHOULD aim to -interoperate by ignoring unrecognized negotiation options (such as +any delivery mechanism other than HTTP 1.1. Future versions might define +protocol negotiation to optimize delivery; compliant implementations SHOULD aim +to interoperate by ignoring unrecognized negotiation options (such as [HTTP `Upgrade` headers](https://datatracker.ietf.org/doc/html/rfc7230#section-6.7)). ## Event Delivery -To provide simpler support for event sources which might be translating events -from existing systems, some data plane requirements for senders are relaxed in -the general case. In the case of Knative Eventing provided resources (Channels -and Brokers) which implement these roles, requirements are increased from SHOULD -to MUST. Exceptions with stronger requirements for Channels and Brokers are -called out as they occur. - ### Minimum supported protocol All senders and recipients MUST support the CloudEvents 1.0 protocol and the @@ -89,7 +95,7 @@ HTTP POST mechanism. Event recipients MUST use the HTTP response code to indicate acceptance of an event. The recipient SHOULD NOT return a response accepting the event until it -has handled event (processed the event or stored it in stable storage). The +has handled the event (processed the event or stored it in stable storage). The following response codes are explicitly defined; event recipients MAY also respond with other response codes. A response code not in this table SHOULD be treated as a retriable error. @@ -100,13 +106,13 @@ treated as a retriable error. | `200` | [Event reply](#event-reply) | No | Yes | No | | `202` | Event accepted | No | Yes | No | | other `2xx` | (Unspecified) | No\* | No\* | Yes\* | -| other `3xx` | (Unspecified) | No\* | No\* | Yes\* | +| `3xx` | (Unspecified) | No\* | No\* | Yes\* | | `400` | Unparsable event | No | No | Yes | | `404` | Endpoint does not exist | Yes | No | Yes | | `409` | Conflict / Processing in progress | Yes | No | Yes | | `429` | Too Many Requests / Overloaded | Yes | No | Yes | | other `4xx` | Error | No | No | Yes | -| other `5xx` | Error | Yes | No | Yes | +| `5xx` | Error | Yes | No | Yes | \* Unspecified `1xx`, `2xx`, and `3xx` response codes are **reserved for future extension**. Event recipients SHOULD NOT send these response codes in this spec @@ -121,17 +127,19 @@ the sender. As specified in the event recipients MUST use the [`source` and `id` attributes](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#required-attributes) to determine duplicate events if needed. This specification does not describe -state requirements for clients which need to detect duplicate events. (See +state requirements for recipients which need to detect duplicate events. (See [observability](#observability) for an example case where other event attributes may vary from one delivery attempt to another). Where possible, event senders SHOULD re-attempt delivery of events where the HTTP request failed or returned a retryable status code. It is RECOMMENDED that event senders implement some form of congestion control (such as exponential -backoff) when managing retry timing. This specification does not document any -specific congestion control algorithm or parameters. -[Brokers](./overview.md#broker) and [Channels](./overview.md#channel) MUST -implement congestion control and MUST implement retries. +backoff) and delivery throttling when managing retry timing. Congestion control +MAY cause event delivery to fail or retry attempts to be skipped. This +specification does not document any specific congestion control algorithm or +parameters. [Brokers](./overview.md#broker) and +[Channels](./overview.md#channel) MUST implement congestion control and MUST +implement retries. ### Observability @@ -150,14 +158,14 @@ beyond the scope of this document. ### Derived (Reply) Events -In some applications, an event receiver might emit an event in reaction to a +In some applications, an event recipient MAY emit an event in reaction to a received event. Components MAY choose to support this pattern by accepting an encoded CloudEvent in the HTTP response. The sender SHOULD NOT assume that a received reply event is directly related to the event sent in the HTTP request. An event sender MAY document support for this pattern by including a `Prefer: reply` header in the HTTP POST request. This header indicates to the -event receiver that the caller will accept a +event recipient that the caller will accept a [`200` response](#event-acknowledgement-and-repeat-delivery) which includes a CloudEvent encoded using the binary or structured formats. [Brokers](./overview.md#broker) and [Channels](./overview.md#channel) MUST @@ -169,7 +177,7 @@ recipient will _never_ provide a response payload, the `202` response code is likely a better choice. If a recipient chooses to reply to a sender with a `200` response code and a -reply event in the absence of a `Prefer: reply` header, the sender SHOULD treat -the event as accepted, and MAY log an error about the unexpected payload. The -sender MUST NOT process the reply event if it did not advertise the -`Prefer: reply` capability. +reply event in the absence of a `Prefer: reply` header from the sender, the +sender SHOULD treat the event as accepted, and MAY log an error about the +unexpected payload. The sender MUST NOT process the reply event if the sender +did not advertise the `Prefer: reply` capability. diff --git a/specs/eventing/motivation.md b/specs/eventing/motivation.md index 44439748c..961a6b57e 100644 --- a/specs/eventing/motivation.md +++ b/specs/eventing/motivation.md @@ -30,4 +30,4 @@ semantics, using the CloudEvents HTTP POST (push) transport as a minimum common transport between components. Knative Eventing also defines patterns to simplify the construction and usage of -event senders and recipients. +event producers and consumers. diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index f21fbe83f..1007d7637 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -63,7 +63,7 @@ on an arbitrary API object) which form a basis for Knative Eventing. ### Addressable -**Addressable** resources expose a resource address (HTTP URL) in their `status` +**Addressable** resources expose a resource `address` (HTTP URL) in their `status` object. The URL is used as a destination for delivery of events to the resource; the exposed URL must implement the [data plane contract](data-plane.md) for receiving events. From f69b8c8c20efd627d253f1047b85908285f2c60c Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Tue, 25 May 2021 15:26:37 -0700 Subject: [PATCH 24/46] Remove abandoned SVG images. --- specs/eventing/images/broker-trigger-overview.svg | 1 - specs/eventing/images/resource-types-overview.svg | 1 - 2 files changed, 2 deletions(-) delete mode 100644 specs/eventing/images/broker-trigger-overview.svg delete mode 100644 specs/eventing/images/resource-types-overview.svg diff --git a/specs/eventing/images/broker-trigger-overview.svg b/specs/eventing/images/broker-trigger-overview.svg deleted file mode 100644 index 035a4f2a0..000000000 --- a/specs/eventing/images/broker-trigger-overview.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/specs/eventing/images/resource-types-overview.svg b/specs/eventing/images/resource-types-overview.svg deleted file mode 100644 index c15f1755e..000000000 --- a/specs/eventing/images/resource-types-overview.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file From ea4f14c6f727b4870258f300480bf10dd6e71a43 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Wed, 26 May 2021 14:54:59 -0700 Subject: [PATCH 25/46] Update overview with Doug's suggestions. --- specs/eventing/overview.md | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index 1007d7637..7a8c7301f 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -5,8 +5,19 @@ patterns: - Topology-based event routing ([`messaging.knative.dev`](#messaging)) + Events are routed based on _connections between objects_ (in particular, + events flow along a [channel](#channel) to all + [subscriptions](#subscription)). This model can be thought of as "event + plumbing" in that events are managed like flows of water through pipes. + - Content-based event routing ([`eventing.knative.dev`](#eventing)) + Events are selected for routing based on the event _attributes_ rather than + primarily by object connections (a [broker](#broker) provides a stream of + events which can be selected by a [trigger](#trigger)). This model is more + akin to picking parts off a conveyor belt, where each event is considered + separately for processing. + Knative Eventing does not directly specify mechanisms for other event-processing models, including multi-stage workflows, correlated request-reply, and sequential (windowed) event processing; these models could be built using the @@ -19,6 +30,7 @@ connecting multiple types of Kubernetes objects as event senders and recipients to the core primitives. ![Overview of objects](images/eventing-overview.svg) + -# Resource Overview +## Resource Overview The Knative Eventing API provides a set of primitives to support both point-to-point communication channels (`messaging.knative.dev`) and @@ -108,7 +109,7 @@ define the mapping of kubernetes verbs (read, watch, patch, etc) to developer roles. See the [Overview documentation](./overview.md) for general definitions of the different API objects. -# Error Signalling +## Error Signalling @@ -252,9 +253,9 @@ transient (for example, `ResourcesAllocated` might change between `True` and `False` as an application scales to and from zero). It is RECOMMENDED that transient conditions be indicated with a `severity="Info"`. -# Resource Lifecycle +## Resource Lifecycle -## Broker Lifecycle +### Broker Lifecycle A Broker represents an Addressable endpoint (i.e. it has a `status.address.url` field) which can receive, store, and forward events to multiple recipients based @@ -281,7 +282,7 @@ that changing the implementation class or `spec.config` is not an atomic operation and that any implementation would be likely to result in event loss during the transition. -## Trigger Lifecycle +### Trigger Lifecycle The lifecycle of a Trigger is independent of the Broker it refers to in its `spec.broker` field; if the Broker does not currently exist or the Broker's @@ -316,7 +317,7 @@ events to the Trigger's `spec.subscriber` prior to the Trigger's send some additional events to the Trigger's `spec.subscriber` after the deletion. -## Channel Lifecycle +### Channel Lifecycle A Channel represents an Addressable endpoint (i.e. it has as `status.address.url` field) which can receive, store, and forward events to @@ -340,7 +341,7 @@ This pattern is chosen to make it clear that changing `spec.channelTemplate` is not an atomic operation and that any implementation would be likely to result in event loss during the transition. -## Subscription Lifecycle +### Subscription Lifecycle The lifecycle of a Subscription is independent of that of the channel it refers to in its `spec.channel` field. The `spec.channel` object reference may refer to @@ -385,7 +386,7 @@ additional events to the Subscription's `spec.subscriber` after the deletion. TODO: channel-compatible CRDs (Channelable) --> -## Addressable Resolution +### Addressable Resolution Both Trigger and Subscription have optional object references (`ref` in `spec.subscriber`, `spec.delivery.deadLetterSink`, and `spec.reply` for @@ -422,14 +423,14 @@ and SHOULD include an indication of the error in a condition reason or type. Both Broker and Channel MUST conform to the Addressable partial schema. -# Event Routing +## Event Routing Note that the event routing description below does not cover the actual mechanics of sending an event from one component to another; see [the data plane](./data-plane.md) contracts for details of the event transfer mechanisms. -## Content Based Routing +### Content Based Routing A Broker MUST publish a URL at `status.address.uri` when it is able to receive events. This URL MUST accept CloudEvents in both the @@ -469,7 +470,7 @@ documented to end-users. Implementations MAY avoid using HTTP to deliver event replies to the Broker's event-delivery input and instead use an internal queueing mechanism. -## Topology Based Routing +### Topology Based Routing A Channel MUST publish a URL at `status.address.url` when it is able to receive events. This URL MUST accept CloudEvents in both the @@ -497,7 +498,7 @@ The implementation MAY attach additional event attributes or other metadata distinguishing between these deliveries. The implementation MUST NOT modify the event payload in this process. -## Event Delivery +### Event Delivery Once a Trigger or Subscription has decided to deliver an event, it MUST do the following: @@ -542,17 +543,17 @@ following: example, if the `ref` form of `deadLetterSink` points to a compatible implementation). -# Detailed Resources +## Detailed Resources -## Broker v1 +### Broker v1 -### Metadata: +#### Metadata: Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. -### Spec: +#### Spec:
@@ -563,19 +564,19 @@ resource. - + - +
configKReference
(Optional)		KReference
(Optional)		 A reference to an object which describes the configuration options for the Broker (for example, a ConfigMap). RECOMMENDED
deliveryDeliverySpec
(Optional)		DeliverySpec
(Optional)		 A default delivery options for Triggers which do not specify more-specific options. REQUIRED
-### Status +#### Status @@ -598,7 +599,7 @@ resource. - + @@ -610,15 +611,15 @@ resource.
addressduckv1.Addressableduckv1.Addressable Address used to deliver events to the Broker. REQUIRED
-## Trigger v1 +### Trigger v1 -### Metadata: +#### Metadata: Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. -### Spec: +#### Spec: @@ -635,25 +636,25 @@ resource. - + - + - +
filterTriggerFilter
(Optional)		TriggerFilter
(Optional)		 Event filters which are used to select events to be delivered to the Trigger's destination. REQUIRED
subscriberduckv1.Destination
(Required)		duckv1.Destination
(Required)		 The destination that filtered events should be sent to. REQUIRED
deliveryDeliverySpec
(Optional)		DeliverySpec
(Optional)		 Delivery options for this Trigger. RECOMMENDED
-### Status +#### Status @@ -688,15 +689,15 @@ resource.
-## Channel v1 +### Channel v1 -### Metadata: +#### Metadata: Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. -### Spec: +#### Spec: @@ -713,19 +714,19 @@ resource. - + - +
subscribers[]duckv1.SubscriberSpec[]duckv1.SubscriberSpec Aggregated subscription information; this array should be managed automatically by the controller. RECOMMENDED
deliveryDeliverySpec
(Optional)		DeliverySpec
(Optional)		 A default delivery options for Subscriptions which do not specify more-specific options. REQUIRED
-### Status +#### Status @@ -748,13 +749,13 @@ resource. - + - + @@ -766,15 +767,15 @@ resource.
addressduckv1.Addressableduckv1.Addressable Address used to deliver events to the Broker. REQUIRED
subscribers[]duckv1.SubscriberStatus[]duckv1.SubscriberStatus Resolved addresses for the spec.subscribers (subscriptions to this Channel). RECOMMENDED
-## Subscription v1 +### Subscription v1 -### Metadata: +#### Metadata: Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. -### Spec: +#### Spec: @@ -791,25 +792,25 @@ resource. - + - + - +
subscriberduckv1.Destination
(Required)		duckv1.Destination
(Required)		 The destination that events should be sent to. REQUIRED
replyduckv1.Destination
(Required)		duckv1.Destination
(Required)		 The destination that reply events from spec.subscriber should be sent to. REQUIRED
deliveryDeliverySpec
(Optional)		DeliverySpec
(Optional)		 Delivery options for this Subscription. RECOMMENDED
-### Status +#### Status @@ -832,28 +833,28 @@ resource. - +
physicalSubscriptionPhysicalSubscriptionStatusPhysicalSubscriptionStatus The fully resolved values for spec endpoint references. REQUIRED
-## Addressable v1 +### Addressable v1 Note that the Addressable interface is a partial schema -- any resource which includes these fields may be referenced using a `duckv1.Destination`. -### Metadata: +#### Metadata: Standard Kubernetes [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectmeta-v1-meta) resource. -### Spec: +#### Spec: There are no `spec` requirements for Addressable. -### Status +#### Status @@ -864,15 +865,15 @@ There are no `spec` requirements for Addressable. - +
addressduckv1.Addressableduckv1.Addressable Address used to deliver events to the Broker. REQUIRED
-# Detailed SubResource Objects +## Detailed SubResource Objects -## duckv1.Addressable +### duckv1.Addressable @@ -883,13 +884,13 @@ There are no `spec` requirements for Addressable. - +
urlURLURL (string) Address used to deliver events to the Addressable. REQUIRED
-## duckv1.Destination +### duckv1.Destination Destination is used to indicate the destination for event delivery. A Destination eventually resolves the supplied information to a URL by resolving @@ -905,7 +906,7 @@ Destination eventually resolves the supplied information to a URL by resolving ref - duckv1.KReference
(Optional) + duckv1.KReference
(Optional) An ObjectReference to an Addressable reference to deliver events to. REQUIRED @@ -917,7 +918,7 @@ Destination eventually resolves the supplied information to a URL by resolving -## duckv1.SubscriberSpec +### duckv1.SubscriberSpec SubscriberSpec represents an automatically-populated extraction of information from a [Subscription](#subscription). @@ -955,13 +956,13 @@ from a [Subscription](#subscription). delivery - DeliverySpec + DeliverySpec The resolved Subscription delivery options. The deadLetterSink should use the uri form. REQUIRED -## duckv1.SubscriberStatus +### duckv1.SubscriberStatus SubscriberStatus indicates the status of programming a Subscription by a Channel. @@ -993,13 +994,13 @@ Channel. message - duckv1.Addressable + duckv1.Addressable A human readable message indicating details of ready status. REQUIRED -## DeliverySpec +### DeliverySpec @@ -1010,7 +1011,7 @@ Channel. - + @@ -1034,7 +1035,7 @@ Channel.
deadLetterSinkduckv1.Destinationduckv1.Destination Fallback address used to deliver events which cannot be delivered during the flow. An implementation MAY place limits on the allowed destinations for the deadLetterSink. RECOMMENDED
-## KReference +### KReference KReference is a lightweight version of kubernetes [`v1/ObjectReference`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#objectreference-v1-core) @@ -1072,7 +1073,7 @@ KReference is a lightweight version of kubernetes -## PhysicalSubscriptionStatus +### PhysicalSubscriptionStatus @@ -1101,7 +1102,7 @@ KReference is a lightweight version of kubernetes
-## TriggerFilter +### TriggerFilter From 55ee647e715603aa8b5435d1e916f185363e8b32 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 3 Jun 2021 01:26:08 -0700 Subject: [PATCH 30/46] Address duglin and lionelvillard suggestions --- specs/eventing/control-plane.md | 128 ++++++++++++++++---------------- 1 file changed, 66 insertions(+), 62 deletions(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index 1b7e723cd..ce8a97d62 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -1,4 +1,5 @@ # Knative Control Plane Contract + ## Abstract The Knative Eventing platform provides common primitives for routing CloudEvents @@ -39,11 +40,25 @@ tooling developers, by extension) deploying applications to the environment. event routing resources and manage the software configuration of Knative Eventing and the underlying abstractions. +## Resource Overview + +The Knative Eventing API provides a set of primitives to support both +point-to-point communication channels (`messaging.knative.dev`) and +content-based event routing (`eventing.knative.dev`). This specification +describes API interfaces of Knative Eventing resources as well as the supported +[event routing](./data-plane.md) logic and configuration settings. + +At the moment, the Knative Eventing specification does not contemplate any +non-Kubernetes-backed implementations, and therefore does not specifically +define the mapping of Kubernetes verbs (read, watch, patch, etc) to developer +roles. See the [Overview documentation](./overview.md) for general definitions +of the different API objects. + ## RBAC Profile In order to validate the controls described in [Resource Overview](#resource-overview), the following Kubernetes RBAC profile -may be applied in a Kubernetes cluster. This Kubernetes RBAC is an illustrative +can be applied in a Kubernetes cluster. This Kubernetes RBAC is an illustrative example of the minimal profile rather than a requirement. This Role should be sufficient to develop, deploy, and manage event routing for an application within a single namespace. Knative Conformance tests against "MUST", "MUST NOT", @@ -68,8 +83,8 @@ In order to support resolving resources which meet the [Trigger](#trigger-lifecycle) or [Subscription](#subscription-lifecycle) will need _read_ access to these resources. On Kubernetes, this is most easily achieved using role aggregation; on systems using Kubernetes RBAC, resources -which wish to participate in Addressable resolution should provide the following -`ClusterRole`: +which wish to participate in Addressable resolution are expected to provide the +following `ClusterRole`: ```yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -95,20 +110,6 @@ Ref: - --> -## Resource Overview - -The Knative Eventing API provides a set of primitives to support both -point-to-point communication channels (`messaging.knative.dev`) and -content-based event routing (`eventing.knative.dev`). This specification -describes API interfaces of Knative Eventing resources as well as the supported -[event routing](./data-plane.md) logic and configuration settings. - -At the moment, the Knative Eventing specification does not contemplate any -non-Kubernetes-backed implementations, and therefore does not specifically -define the mapping of kubernetes verbs (read, watch, patch, etc) to developer -roles. See the [Overview documentation](./overview.md) for general definitions -of the different API objects. - ## Error Signalling @@ -257,30 +258,31 @@ transient conditions be indicated with a `severity="Info"`. ### Broker Lifecycle -A Broker represents an Addressable endpoint (i.e. it has a `status.address.url` -field) which can receive, store, and forward events to multiple recipients based -on a set of attribute filters (Triggers). Triggers are associated with a Broker -based on the `spec.broker` field on the Trigger; it is expected that the -controller for a Broker will also control the associated Triggers. When the -Broker's `Ready` condition is `true`, the Broker MUST provide a -`status.address.url` which accepts all valid CloudEvents and MUST forward the -received events for filtering to each associated Trigger whose `Ready` condition -is `true`. As described in the [Trigger Lifecycle](#trigger-lifecycle) section, -a Broker MAY forward events to an associated Trigger which which does not -currently have a `true` `Ready` condition, including events received by the -Broker before the Trigger was created. - -The annotation `eventing.knative.dev/broker.class` may be used to select a -particular implementation of a Broker. When a Broker is created, its -implementation and the `spec.config` field MUST be populated (`spec.config` MAY -be an empty broker) to indicate which of several possible Broker implementations -to use. It is RECOMMENDED to default the `eventing.knative.dev/broker.class` -field on creation if it is unpopulated. Once created, both fields MUST be -immutable; the Broker must be deleted and re-created to change the -implementation class or `spec.config`. This pattern is chosen to make it clear -that changing the implementation class or `spec.config` is not an atomic -operation and that any implementation would be likely to result in event loss -during the transition. +A Broker represents an Addressable endpoint (i.e. it MUST have a +`status.address.url` field) which can receive, store, and forward events to +multiple recipients based on a set of attribute filters (Triggers). Triggers are +associated with a Broker based on the `spec.broker` field on the Trigger; it is +expected that the controller for a Broker will also control the associated +Triggers. When the Broker's `Ready` condition is `true`, the Broker MUST provide +a `status.address.url` which accepts all valid CloudEvents and MUST attempt to +forward the received events for filtering to each associated Trigger whose +`Ready` condition is `true`. As described in the +[Trigger Lifecycle](#trigger-lifecycle) section, a Broker MAY forward events to +an associated Trigger which which does not currently have a `true` `Ready` +condition, including events received by the Broker before the Trigger was +created. + +The annotation `eventing.knative.dev/broker.class` MAY be used to select a +particular implementation of a Broker. When a Broker is created, the +`eventing.knative.dev/broker.class` annotation and the `spec.config` field MUST +be populated (`spec.config` MAY be an empty object) to indicate which of several +possible Broker implementations to use. It is RECOMMENDED to default the +`eventing.knative.dev/broker.class` field on creation if it is unpopulated. Once +created, both fields MUST be immutable; the Broker MUST be deleted and +re-created to change the implementation class or `spec.config`. This pattern is +chosen to make it clear that changing the implementation class or `spec.config` +is not an atomic operation and that any implementation would be likely to result +in event loss during the transition. ### Trigger Lifecycle @@ -290,21 +292,22 @@ The lifecycle of a Trigger is independent of the Broker it refers to in its `false`, and the reason SHOULD indicate that the corresponding Broker is missing or not ready. -The Trigger MUST also set the `status.subscriberUri` field based on resolving -the `spec.subscriber` field before setting the `Ready` condition to `true`. If -the `spec.subscriber.ref` field points to a resource which does not exist or -cannot be resolved via [Addressable resolution](#addressable-resolution), the -Trigger MUST set the `Ready` condition to `false`, and at least one condition -should indicate the reason for the error. +The Trigger's controller MUST also set the `status.subscriberUri` field based on +resolving the `spec.subscriber` field before setting the `Ready` condition to +`true`. If the `spec.subscriber.ref` field points to a resource which does not +exist or cannot be resolved via +[Addressable resolution](#addressable-resolution), the Trigger MUST set the +`Ready` condition to `false`, and at least one condition should indicate the +reason for the error. If the Trigger's `spec.delivery.deadLetterSink` field it set, it MUST be resolved to a URI and reported in `status.deadLetterSinkUri` in the same manner as the `spec.subscriber` field before setting the `Ready` condition to `true`. Once created, the Trigger's `spec.broker` SHOULD NOT permit updates; to change -the `spec.broker`, the Trigger can be deleted and re-created. This pattern is -chosen to make it clear that changing the `spec.broker` is not an atomic -operation, as it may span multiple storage systems. Changes to +the `spec.broker`, the Trigger can instead be deleted and re-created. This +pattern is chosen to make it clear that changing the `spec.broker` is not an +atomic operation, as it could span multiple storage systems. Changes to `spec.subscriber`, `spec.filter` and other fields SHOULD be permitted, as these could occur within a single storage system. @@ -319,7 +322,7 @@ deletion. ### Channel Lifecycle -A Channel represents an Addressable endpoint (i.e. it has as +A Channel represents an Addressable endpoint (i.e. it MUST have a `status.address.url` field) which can receive, store, and forward events to multiple recipients (Subscriptions). Subscriptions are associated with a Channel based on the `spec.channel` field on the Subscription; it is expected that the @@ -343,8 +346,8 @@ event loss during the transition. ### Subscription Lifecycle -The lifecycle of a Subscription is independent of that of the channel it refers -to in its `spec.channel` field. The `spec.channel` object reference may refer to +The lifecycle of a Subscription is independent of the channel it refers to in +its `spec.channel` field. The `spec.channel` object reference may refer to either an `messaging.knative.dev/v1` Channel resource, or another resource which meets the `spec.subscribers` and `spec.delivery` required elements in the Channelable duck type. If the referenced `spec.channel` does not currently exist @@ -432,18 +435,19 @@ mechanisms. ### Content Based Routing -A Broker MUST publish a URL at `status.address.uri` when it is able to receive +A Broker MUST publish a URL at `status.address.url` when it is able to receive events. This URL MUST accept CloudEvents in both the [Binary Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#31-binary-content-mode) and [Structured Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#32-structured-content-mode) HTTP formats. Before sending an HTTP response, the Broker MUST durably enqueue -the event (be able to deliver with retry without receiving the event again). +the event (where durability means that the Broker can retry event delivery +beyond the duration of receiving the event). For each event received by the Broker, the Broker MUST evaluate each associated Trigger **once** (where "associated" means a Trigger with a `spec.broker` which references the Broker). If the Trigger has a `Ready` condition of `true` when -the event is evaluated, the the Broker MUST evaluate the Trigger's `spec.filter` +the event is evaluated, the Broker MUST evaluate the Trigger's `spec.filter` and, if matched, proceed with [event delivery as described below](#event-delivery). The Broker MAY also evaluate and forward events to associated Triggers for which the `Ready` @@ -455,17 +459,17 @@ set the `Ready` condition to `true`.) If multiple Triggers match an event, one event delivery MUST be generated for each match; duplicate matches with the same destination MUST each generate a separate event delivery attempts, one per Trigger match. The implementation MAY -attach additional event attributes or other metadata distinguishing between these -deliveries. The implementation MUST NOT modify the event payload in this +attach additional event attributes or other metadata distinguishing between +these deliveries. The implementation MUST NOT modify the event payload in this process. Reply events generated during event delivery MUST be re-enqueued by the Broker in the same way as events delivered to the Broker's Addressable URL. If the storage of the reply event fails, the entire event delivery MUST be failed and -the delivery to the Trigger's subscriber MUST be retried. Reply events re-enqueued -in this manner MUST be evaluated against all triggers associated with the -Broker, including the Trigger that generated the reply. Implementations MAY -implement event-loop detection; it is RECOMMENDED that any such controls be +the delivery to the Trigger's subscriber MUST be retried. Reply events +re-enqueued in this manner MUST be evaluated against all triggers associated +with the Broker, including the Trigger that generated the reply. Implementations +MAY implement event-loop detection; it is RECOMMENDED that any such controls be documented to end-users. Implementations MAY avoid using HTTP to deliver event replies to the Broker's event-delivery input and instead use an internal queueing mechanism. From 22b85c9618ec05c45ba782f84da9d6db5517a7bb Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 3 Jun 2021 15:46:34 -0700 Subject: [PATCH 31/46] Refactor error signalling to a common location Update with duglin comments --- specs/common/error-signalling.md | 155 +++++++++++++ specs/eventing/control-plane.md | 143 +----------- .../serving/knative-api-specification-1.0.md | 203 +++--------------- 3 files changed, 189 insertions(+), 312 deletions(-) create mode 100644 specs/common/error-signalling.md diff --git a/specs/common/error-signalling.md b/specs/common/error-signalling.md new file mode 100644 index 000000000..f3d87bea9 --- /dev/null +++ b/specs/common/error-signalling.md @@ -0,0 +1,155 @@ +# Error Signalling + + + +Knative APIs use the +[Kubernetes Conditions convention](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) +to communicate errors and problems to the user. Note that Knative customizes the +general Kubernetes recommendation with a `severity` field, and does not include +`lastTransitionTime` for scalability reasons. Each user-visible resource +described in Resource Overview MUST have a `conditions` field in `status`, which +MUST be a list of `Condition` objects of the following form. Fields in the +condition which are not marked as REQUIRED may be omitted to indicate the +default value (i.e. a Condition which does not include a `status` field is +equivalent to a `status` of `"Unknown"`). The actual API object types in an +OpenAPI document may be named `FooCondition` to allow better code generation and +disambiguation between similar fields in the same `apiGroup`. + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Field + Type + Description + Default Value If Unset +
type + string + The category of the condition, as a short, CamelCase word or phrase. +

+This is the primary key of the Conditions list when viewed as a map. +

REQUIRED – No default +
status + Enum:
    + +
  • "True" +
  • "False" +
  • "Unknown"
+ + 		The last measured status of this condition. + "Unknown" +
reason + string + One-word CamelCase reason for the condition's last transition. + "" +
message + string + Human-readable sentence describing the last transition. + "" +
severity + Enum:
    + +
  • "" +
  • "Warning" +
  • "Info"
+ + 		If present, represents the severity of the condition. An empty severity represents a severity level of "Error". + "" +
lastTransitionTime + Timestamp + Last update time for this condition. + (no timestamp specified) +
+ +Additionally, the resource's `status.conditions` field MUST be managed as +follows to enable clients (particularly user interfaces) to present useful +diagnostic and error message to the user. In the following section, conditions +are referred to by their `type` (aka the string value of the `type` field on the +Condition). + +1. Each resource MUST have either a `Ready` condition (for ongoing systems) or + `Succeeded` condition (for resources that run to completion) with + `severity=""`, which MUST use the `"True"`, `"False"`, and `"Unknown"` + status values as follows: + + 1. `"False"` MUST indicate a failure condition. + 1. `"Unknown"` SHOULD indicate that reconciliation is not yet complete and + success or failure is not yet determined. + 1. `"True"` SHOULD indicate that the resource is fully reconciled and + operating correctly. + + `"Unknown"` and `"True"` are specified as SHOULD rather than MUST + requirements because there may be errors which prevent functioning which + cannot be determined by the API stack (e.g. DNS record configuration in + certain environments). Implementations are expected to treat these as "MUST" + for factors within the control of the implementation. + +1. For non-`Ready` conditions, any conditions with `severity=""` (aka "Error + conditions") MUST be aggregated into the "Ready" condition as follows: + + 1. If the condition is `"False"`, `Ready`'s status MUST be `"False"`. + 1. If the condition is `"Unknown"`, `Ready`'s status MUST be `"False"` or + `"Unknown"`. + 1. If the condition is `"True"`, `Ready`'s status can be any of `"True"`, + `"False"`, or `"Unknown"`. + + Implementations MAY choose to report that `Ready` is `"False"` or + `"Unknown"` even if all Error conditions report a status of `"True"` (i.e. + there might be additional hidden implementation conditions which feed into + the `Ready` condition which are not reported.) + +1. Non-`Ready` conditions with non-error severity MAY be surfaced by the + implementation. Examples of `Warning` or `Info` conditions could include: + missing health check definitions, scale-to-zero status, or non-fatal + capacity limits. + +1. Conditions with a `status` other than `"True"` SHOULD provide `message` and + `reason` fields indicating the reason that the `status` is not `"True"`. + Conditions where the `status` is `"False"` MUST provide a failure `reason` + in the condition. (`"Unknown"` conditions may not have been reconciled, and + so may have an empty `reason`.) + +Conditions type names SHOULD be chosen to describe positive conditions where +`"True"` means that the condition has been satisfied. Some conditions MAY be +transient (for example, `ResourcesAllocated` might change between `"True"` and +`"False"` as an application scales to and from zero). It is RECOMMENDED that +transient conditions be indicated with a `severity="Info"`. diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index ce8a97d62..1da37d1b9 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -112,147 +112,8 @@ Ref: ## Error Signalling - - -The Knative API uses the -[Kubernetes Conditions convention](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) -to communicate errors and problems to the user. Each user-visible resource -described in Resource Overview MUST have a `conditions` field in `status`, which -must be a list of `Condition` objects of the following form (note that the -actual API object types may be named `FooCondition` to allow better code -generation and disambiguation between similar fields in the same `apiGroup`): - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Field - Type - Description - Default Value -
type - string - The category of the condition, as a short, CamelCase word or phrase. -

-This is the primary key of the Conditions list when viewed as a map. -

REQUIRED – No default -
status - Enum:
    - -
  • "True" -
  • "False" -
  • "Unknown"
- - 		The last measured status of this condition. - "Unknown" -
reason - string - One-word CamelCase reason for the condition's last transition. - "" -
message - string - Human-readable sentence describing the last transition. - "" -
severity - Enum:
    - -
  • "" -
  • "Warning" -
  • "Info"
- - 		If present, represents the severity of the condition. An empty severity represents a severity level of "Error". - "" -
lastTransitionTime - Timestamp - Last update time for this condition. - "" – may be unset -
- -Additionally, the resource's `status.conditions` field MUST be managed as -follows to enable clients (particularly user interfaces) to present useful -diagnostic and error message to the user. In the following section, conditions -are referred to by their `type` (aka the string value of the `type` field on the -Condition). - -1. Each resource MUST have either a `Ready` condition (for ongoing systems) or - `Succeeded` condition (for resources that run to completion) with - `severity=""`, which MUST use the `True`, `False`, and `Unknown` status - values as follows: - - 1. `False` MUST indicate a failure condition. - 1. `Unknown` SHOULD indicate that reconciliation is not yet complete and - success or failure is not yet determined. - 1. `True` SHOULD indicate that the application is fully reconciled and - operating correctly. - - `Unknown` and `True` are specified as SHOULD rather than MUST requirements - because there may be errors which prevent functioning which cannot be - determined by the API stack (e.g. DNS record configuration in certain - environments). Implementations are expected to treat these as "MUST" for - factors within the control of the implementation. - -1. For non-`Ready` conditions, any conditions with `severity=""` (aka "Error - conditions") must be aggregated into the "Ready" condition as follows: - - 1. If the condition is `False`, `Ready` MUST be `False`. - 1. If the condition is `Unknown`, `Ready` MUST be `False` or `Unknown`. - 1. If the condition is `True`, `Ready` may be any of `True`, `False`, or - `Unknown`. - - Implementations MAY choose to report that `Ready` is `False` or `Unknown` - even if all Error conditions report a status of `True` (i.e. there may be - additional hidden implementation conditions which feed into the `Ready` - condition which are not reported.) - -1. Non-`Ready` conditions with non-error severity MAY be surfaced by the - implementation. Examples of `Warning` or `Info` conditions could include: - missing health check definitions, scale-to-zero status, or non-fatal - capacity limits. - -Conditions type names should be chosen to describe positive conditions where -`True` means that the condition has been satisfied. Some conditions may be -transient (for example, `ResourcesAllocated` might change between `True` and -`False` as an application scales to and from zero). It is RECOMMENDED that -transient conditions be indicated with a `severity="Info"`. +See [the Knative common condition guidance](../common/error-signalling) for how +resource errors are signalled to the user. ## Resource Lifecycle diff --git a/specs/serving/knative-api-specification-1.0.md b/specs/serving/knative-api-specification-1.0.md index 0e1c74698..363c4fc0c 100644 --- a/specs/serving/knative-api-specification-1.0.md +++ b/specs/serving/knative-api-specification-1.0.md @@ -1,6 +1,5 @@ # Knative Serving API Specification - - + @@ -511,7 +516,7 @@ resource. The `apiVersion` is `eventing.knative.dev/v1` and the `kind` is - + @@ -523,7 +528,7 @@ resource. The `apiVersion` is `eventing.knative.dev/v1` and the `kind` is - + @@ -553,7 +558,7 @@ resource. The `apiVersion` is `eventing.knative.dev/v1` and the `kind` is - + @@ -590,7 +595,7 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is - + @@ -626,7 +631,7 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is - + @@ -669,19 +674,19 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is - + - + - + @@ -711,7 +716,7 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is - + @@ -779,8 +784,7 @@ There are no `spec` requirements for Addressable. Destination is used to indicate the destination for event delivery. A Destination eventually resolves the supplied information to a URL by resolving `uri` relative to the address of `ref` (if provided) as described in -[Addressable resolution](#addressable-resolution). `ref` MAY be an -[Addressable](#duckv1-addressable) object or a `v1/Service` Kubernetes service. +[Destination resolution](#destination-resolution).

@@ -148,45 +147,46 @@ deployment scenarios (by frequency). ## Extensions -Extending the Knative resource model allows for custom semantics to be -offered by implementions of the specification. Unless otherwise noted, -implementations of this specification MAY define extensions but those -extensions MUST NOT contradict the semantics defined within this specification. +Extending the Knative resource model allows for custom semantics to be offered +by implementions of the specification. Unless otherwise noted, implementations +of this specification MAY define extensions but those extensions MUST NOT +contradict the semantics defined within this specification. There are several ways in which implementations can extend the model: -* Annotations and Labels
- _Note_: Because this mechanism allows new controllers to be added to the - system without requiring code changes to the core Knative components, it is - the preferred mechanism for extending the Knative interface. + +- Annotations and Labels
_Note_: Because this mechanism allows new + controllers to be added to the system without requiring code changes to the + core Knative components, it is the preferred mechanism for extending the + Knative interface. Allowing end users to include annotations or labels on the Knative resources allows for them to indicate that they would like some additional semantics - applied to those resources. When defining annotations, or labels, it - is STRONGLY RECOMMENDED that they have some vendor-specific prefix to - avoid any potential naming conflict with other extensions or future - annotations defined by the specification. For more information on - annotations and labels, see + applied to those resources. When defining annotations, or labels, it is + STRONGLY RECOMMENDED that they have some vendor-specific prefix to avoid any + potential naming conflict with other extensions or future annotations defined + by the specification. For more information on annotations and labels, see [here](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#label-selector-and-annotation-conventions). -* Additional Properties
- There might be times when annotations and labels can not be used to - properly (or easily) allow end users to convey their desired semantics, - in which case additional well-defined properties might need to be +- Additional Properties
There might be times when annotations and labels can + not be used to properly (or easily) allow end users to convey their desired + semantics, in which case additional well-defined properties might need to be defined by implementations. - In these cases vendor-specific properties MAY be defined and it is - STRONGLY RECOMMENDED that they be named, or prefixed, in such a way - to clearly indicate their scope and purpose. Choosing a name that - is too generic might lead to conflicts with other vendor extensions - or future changes to the specification. + In these cases vendor-specific properties MAY be defined and it is STRONGLY + RECOMMENDED that they be named, or prefixed, in such a way to clearly indicate + their scope and purpose. Choosing a name that is too generic might lead to + conflicts with other vendor extensions or future changes to the specification. + + For example, adding authentication on a per-tag basis via annotations might + look like: - For example, adding authentication on a per-tag basis via annotations - might look like: ``` annotations: knative.vendor.com/per-tag-auth: "{'cannary': true, 'latest': true}" ``` + but, that is not as user-friendly as extending the `traffic` section itself: + ``` spec: traffic: @@ -510,145 +510,8 @@ if allowed, at least JSON Merge patch be made available. # Error Signalling -The Knative API uses the -[Kubernetes Conditions convention](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) -to communicate errors and problems to the user. Each user-visible resource -described in Resource Overview MUST have a `conditions` field in `status`, which -must be a list of `Condition` objects of the following form (note that the -actual API object types may be named `FooCondition` to allow better code -generation and disambiguation between similar fields in the same `apiGroup`): - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Field - Type - Description - Default Value -
type - string - The category of the condition, as a short, CamelCase word or phrase. -

-This is the primary key of the Conditions list when viewed as a map. -

REQUIRED – No default -
status - Enum:
    - -
  • "True" -
  • "False" -
  • "Unknown"
- - 		The last measured status of this condition. - "Unknown" -
reason - string - One-word CamelCase reason for the condition's last transition. - "" -
message - string - Human-readable sentence describing the last transition. - "" -
severity - Enum:
    - -
  • "" -
  • "Warning" -
  • "Info"
- - 		If present, represents the severity of the condition. An empty severity represents a severity level of "Error". - "" -
lastTransitionTime - Timestamp - Last update time for this condition. - "" – may be unset -
- -Additionally, the resource's `status.conditions` field MUST be managed as -follows to enable clients (particularly user interfaces) to present useful -diagnostic and error message to the user. In the following section, conditions -are referred to by their `type` (aka the string value of the `type` field on the -Condition). - -1. Each resource MUST have either a `Ready` condition (for ongoing systems) or - `Succeeded` condition (for resources that run to completion) with - `severity=""`, which MUST use the `True`, `False`, and `Unknown` status - values as follows: - - 1. `False` MUST indicate a failure condition. - 1. `Unknown` SHOULD indicate that reconciliation is not yet complete and - success or failure is not yet determined. - 1. `True` SHOULD indicate that the application is fully reconciled and - operating correctly. - - `Unknown` and `True` are specified as SHOULD rather than MUST requirements - because there may be errors which prevent serving which cannot be determined - by the API stack (e.g. DNS record configuration in certain environments). - Implementations are expected to treat these as "MUST" for factors within the - control of the implementation. - -1. For non-`Ready` conditions, any conditions with `severity=""` (aka "Error - conditions") must be aggregated into the "Ready" condition as follows: - - 1. If the condition is `False`, `Ready` MUST be `False`. - 1. If the condition is `Unknown`, `Ready` MUST be `False` or `Unknown`. - 1. If the condition is `True`, `Ready` may be any of `True`, `False`, or - `Unknown`. - - Implementations MAY choose to report that `Ready` is `False` or `Unknown` - even if all Error conditions report a status of `True` (i.e. there may be - additional hidden implementation conditions which feed into the `Ready` - condition which are not reported.) - -1. Non-`Ready` conditions with non-error severity MAY be surfaced by the - implementation. Examples of `Warning` or `Info` conditions could include: - missing health check definitions, scale-to-zero status, or non-fatal - capacity limits. - -Conditions type names should be chosen to describe positive conditions where -`True` means that the condition has been satisfied. Some conditions may be -transient (for example, `ResourcesAllocated` might change between `True` and -`False` as an application scales to and from zero). It is RECOMMENDED that -transient conditions be indicated with a `severity="Info"`. +See [the Knative common condition guidance](../common/error-signalling) for how +resource errors are signalled to the user. # Resource Lifecycle @@ -694,9 +557,8 @@ Configuration or Route, as follows: - Additional `labels` and `annotations` on the Configuration and Route not specified above MUST be removed. - See the documentation of `spec` in the - [detailed resource fields section](#detailed-resources--v1) for the - mapping of specific `spec` fields to the corresponding fields in Configuration - and Route. + [detailed resource fields section](#detailed-resources--v1) for the mapping of + specific `spec` fields to the corresponding fields in Configuration and Route. Similarly, the Service MUST update its `status` fields based on the corresponding `status` of its owned Route and Configuration. The Service MUST @@ -1362,8 +1224,8 @@ Restrictions to the values of the field are noted in the Description column. ## TrafficTarget -This resource specifies how the network traffic for a particular -Revision or Configuration is to be configured. +This resource specifies how the network traffic for a particular Revision or +Configuration is to be configured. @@ -2848,6 +2710,5 @@ Max: 1 ## Authors -[Dan Gerdesmeier](mailto:dangerd@google.com) -[Doug Davis](mailto:dug@us.ibm.com) +[Dan Gerdesmeier](mailto:dangerd@google.com) [Doug Davis](mailto:dug@us.ibm.com) [Evan Anderson](mailto:evana@vmware.com) From 4af35db982a186e7dfd27a1f73b3d07580d6c401 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Sun, 13 Jun 2021 10:53:23 -0700 Subject: [PATCH 32/46] Address duglin comments --- specs/eventing/images/eventing-overview.svg | 77 +++++++++++++-------- specs/eventing/overview.md | 36 ++++++---- 2 files changed, 70 insertions(+), 43 deletions(-) diff --git a/specs/eventing/images/eventing-overview.svg b/specs/eventing/images/eventing-overview.svg index 4ba72413f..589224995 100644 --- a/specs/eventing/images/eventing-overview.svg +++ b/specs/eventing/images/eventing-overview.svg @@ -1,80 +1,97 @@ -eventing.knative.devmessaging.knative.devBrokerTriggerTriggerChannelSubscriptionSubscription«Knative Service»Addressable«Kubernetes Service»Addressablebrokerbrokersubscriberchannelchannelsubscriberreplyeventing.knative.devmessaging.knative.devBrokerTriggerevents selected from Broker,replies sent back to BrokerTriggerChannelSubscriptionSubscriptionevents sent from Channel to Addressable,replies sent to second Addressable«Knative Service»Addressable«Kubernetes Service»Addressablebrokerbrokersubscriberchannelchannelsubscriberreply +Knative Eventing does not directly specify mechanisms for other event-processing +models, including multi-stage workflows, correlated request-reply, and +sequential (windowed) event processing; these models could be built using the +primitives provided by Knative, or Knative could deliver events to an external +system that implements these models. + +In addition to the primitives needed to express the above patterns, Knative +Eventing defines two [_interface contracts_](#interface-contracts) to allow +connecting multiple types of Kubernetes objects as event senders and recipients +to the core primitives. + ## Interface Contracts In addition to the concrete types described below in the `messaging.knative.dev` From 2479d26cf2759315c4c046af4f848786c02d4e9f Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Sun, 13 Jun 2021 23:02:53 -0700 Subject: [PATCH 33/46] Address Doug's comments --- specs/common/error-signalling.md | 34 +-- specs/eventing/control-plane.md | 237 ++++++++++-------- .../serving/knative-api-specification-1.0.md | 2 +- 3 files changed, 147 insertions(+), 126 deletions(-) diff --git a/specs/common/error-signalling.md b/specs/common/error-signalling.md index f3d87bea9..59b6f895d 100644 --- a/specs/common/error-signalling.md +++ b/specs/common/error-signalling.md @@ -6,14 +6,18 @@ Knative APIs use the [Kubernetes Conditions convention](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) to communicate errors and problems to the user. Note that Knative customizes the general Kubernetes recommendation with a `severity` field, and does not include -`lastTransitionTime` for scalability reasons. Each user-visible resource +`lastHeartbeatTime` for scalability reasons. Each user-visible resource described in Resource Overview MUST have a `conditions` field in `status`, which -MUST be a list of `Condition` objects of the following form. Fields in the -condition which are not marked as REQUIRED may be omitted to indicate the -default value (i.e. a Condition which does not include a `status` field is -equivalent to a `status` of `"Unknown"`). The actual API object types in an -OpenAPI document may be named `FooCondition` to allow better code generation and -disambiguation between similar fields in the same `apiGroup`. +MUST be a list of `Condition` objects described by the following table. + +Fields in the condition which are not marked as "REQUIRED" MAY be omitted to +indicate the default value (i.e. a Condition which does not include a `status` +field is equivalent to a `status` of `"Unknown"`). As `Conditions` are an output +API, an implementation MAY never set these fields; the OpenAPI document MUST +still describe these fields. The actual API object types in an OpenAPI document +might be named `FooCondition` (for a `Foo` resource, for example) to allow +better code generation and disambiguation between similar fields in the same +`apiGroup`.
@@ -33,7 +37,8 @@ disambiguation between similar fields in the same `apiGroup`. @@ -106,7 +111,7 @@ diagnostic and error message to the user. In the following section, conditions are referred to by their `type` (aka the string value of the `type` field on the Condition). -1. Each resource MUST have either a `Ready` condition (for ongoing systems) or +1. Each resource MUST have a summary `Ready` condition (for ongoing systems) or `Succeeded` condition (for resources that run to completion) with `severity=""`, which MUST use the `"True"`, `"False"`, and `"Unknown"` status values as follows: @@ -118,7 +123,7 @@ Condition). operating correctly. `"Unknown"` and `"True"` are specified as SHOULD rather than MUST - requirements because there may be errors which prevent functioning which + requirements because there might be errors which prevent functioning which cannot be determined by the API stack (e.g. DNS record configuration in certain environments). Implementations are expected to treat these as "MUST" for factors within the control of the implementation. @@ -137,16 +142,11 @@ Condition). there might be additional hidden implementation conditions which feed into the `Ready` condition which are not reported.) -1. Non-`Ready` conditions with non-error severity MAY be surfaced by the - implementation. Examples of `Warning` or `Info` conditions could include: - missing health check definitions, scale-to-zero status, or non-fatal - capacity limits. - 1. Conditions with a `status` other than `"True"` SHOULD provide `message` and `reason` fields indicating the reason that the `status` is not `"True"`. Conditions where the `status` is `"False"` MUST provide a failure `reason` - in the condition. (`"Unknown"` conditions may not have been reconciled, and - so may have an empty `reason`.) + in the condition. (`"Unknown"` conditions might not have been reconciled, + and so MAY have an empty `reason`.) Conditions type names SHOULD be chosen to describe positive conditions where `"True"` means that the condition has been satisfied. Some conditions MAY be diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index 1da37d1b9..0781600b7 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -24,8 +24,8 @@ interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119). There is no formal specification of the Kubernetes API and Resource Model. This document assumes Kubernetes 1.21 behavior; this behavior will typically be -supported by many future Kubernetes versions. Additionally, this document may -reference specific core Kubernetes resources; these references may be +supported by many future Kubernetes versions. Additionally, this document +references specific core Kubernetes resources; these references can be illustrative (i.e. _an implementation on Kubernetes_) or descriptive (i.e. _this Kubernetes resource MUST be exposed_). References to these core Kubernetes resources will be annotated as either illustrative or descriptive. @@ -59,7 +59,7 @@ of the different API objects. In order to validate the controls described in [Resource Overview](#resource-overview), the following Kubernetes RBAC profile can be applied in a Kubernetes cluster. This Kubernetes RBAC is an illustrative -example of the minimal profile rather than a requirement. This Role should be +example of the minimal profile rather than a requirement. This Role is sufficient to develop, deploy, and manage event routing for an application within a single namespace. Knative Conformance tests against "MUST", "MUST NOT", and "REQUIRED" conditions are expected to pass when using this profile: @@ -83,7 +83,7 @@ In order to support resolving resources which meet the [Trigger](#trigger-lifecycle) or [Subscription](#subscription-lifecycle) will need _read_ access to these resources. On Kubernetes, this is most easily achieved using role aggregation; on systems using Kubernetes RBAC, resources -which wish to participate in Addressable resolution are expected to provide the +which wish to participate in [Addressable resolution](#addressable-resolution) are expected to provide the following `ClusterRole`: ```yaml @@ -99,7 +99,7 @@ rules: verbs: ["get", "list", "watch"] ``` -This configuration advice SHALL NOT indicate a requirement for Kubernetes RBAC +This configuration advice does not indicate a requirement for Kubernetes RBAC support. -### Addressable Resolution +### Destination Resolution Both Trigger and Subscription have OPTIONAL object references (`ref` in `spec.subscriber`, `spec.delivery.deadLetterSink`, and `spec.reply` for -Subscription) which are expected to conform to the Addressable partial schema -("duck type"). An object conforms with the Addressable partial schema if it -contains a `status.address.url` field containing a URL which can be used to -deliver CloudEvents over HTTP. As a special case, Kubernetes `v1` `Service` -objects are considered to have a `status.address.url` of -`http:///`. +Subscription) contained in the type [`duck.v1.Destination`](#duckv1destination). +Destination provides a mechanism to resolve an object reference to an absolute +URL; two mechanisms MUST be supported: + +1. Objects conforming to the [Addressable partial schema](#duckv1addressable) + ("duck type") contain a `status.address.url` field providing a URL which can + be used to deliver CloudEvents over HTTP. +2. As a special case, Kubernetes `v1/Service` objects are resolved to the + service's cluster-local DNS name (of the form + `http://..svc/`). If both the `ref` field and the `uri` fields are set on a Destination, then the Destination's address MUST be the `uri` interpreted relative to the resolved URI @@ -281,14 +287,14 @@ This Destination would resolve to `http://test./update`, as `/update` would be interpreted relative to the `test` Service's DNS name. -If one of these object references points to an object which does not currently -satisfy this partial schema (because object does not exist, the -`status.address.url` field is empty, or because the object does not have that -field), then the Trigger or Subscription MUST indicate an error by setting the -`Ready` condition to `false`, and SHOULD include an indication of the error in a -condition reason or type. +If a Destination includes a reference to an object which does not resolve to an +absolute URL (because object does not exist, the `status.address.url` field is +empty, etc), then the Trigger or Subscription MUST indicate an error by setting +the `Ready` condition to `false`, and SHOULD include an indication of the error +in a condition reason or type. -Both Broker and Channel MUST conform to the Addressable partial schema. +Both Broker and Channel MUST conform to the [Addressable +partial(#duckv1addressable)] schema. ## Event Routing @@ -300,13 +306,11 @@ mechanisms. ### Content Based Routing A Broker MUST publish a URL at `status.address.url` when it is able to receive -events. This URL MUST accept CloudEvents in both the -[Binary Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#31-binary-content-mode) -and -[Structured Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#32-structured-content-mode) -HTTP formats. Before sending an HTTP response, the Broker MUST durably enqueue -the event (where durability means that the Broker can retry event delivery -beyond the duration of receiving the event). +events. This URL MUST implement the receiver requirements of +[event delivery](#data-plane.md#event-delivery). Before +[acknowledging an event](data-plane.md#event-acknowledgement-and-delivery-retry), +the Broker MUST durably enqueue the event (where durability means that the +Broker can retry event delivery beyond the duration of receiving the event). For each event received by the Broker, the Broker MUST evaluate each associated Trigger **exactly once** (where "associated" means a Trigger with a @@ -324,19 +328,20 @@ If multiple Triggers match an event, one event delivery MUST be generated for each match; duplicate matches with the same destination MUST each generate a separate event delivery attempts, one per Trigger match. The implementation MAY attach additional event attributes or other metadata distinguishing between -these deliveries. The implementation MUST NOT modify the event payload in this -process. +these deliveries. The implementation MUST NOT modify the +[event data](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#event-data) +in this process. Reply events generated during event delivery MUST be re-enqueued by the Broker -in the same way as events delivered to the Broker's Addressable URL. If the -storage of the reply event fails, the entire event delivery MUST be failed and -the delivery to the Trigger's subscriber MUST be retried. Reply events -re-enqueued in this manner MUST be evaluated against all triggers associated -with the Broker, including the Trigger that generated the reply. Implementations -MAY implement event-loop detection; it is RECOMMENDED that any such controls be -documented to end-users. Implementations MAY avoid using HTTP to deliver event -replies to the Broker's event-delivery input and instead use an internal -queueing mechanism. +using the same routing and persistence as events delivered to the Broker's +Addressable URL. Reply events re-enqueued in this manner MUST be evaluated +against all Triggers associated with the Broker, including the Trigger that +generated the reply. If the storage of the reply event in the Broker fails, the +entire event delivery MUST be failed and the delivery to the Trigger's +subscriber MUST be retried. Implementations MAY implement event-loop detection; +it is RECOMMENDED that any such controls be documented to end-users. +Implementations MAY avoid using HTTP to deliver event replies to the Broker's +event-delivery input and instead use an internal queueing mechanism. ### Topology Based Routing @@ -352,27 +357,28 @@ For each event received by the Channel, the Channel MUST deliver the event to each associated Subscription **at least once** (where "associated" means a Subscription with a `spec.channel` which references the Channel). If the Subscription has a `Ready` condition of `true` when the event is evaluated, the -Channel MUST forward the event as described in -[event delivery as described below](#event-delivery). The Channel MAY also -forward events to associated Subscriptions for with the `Ready` condition is not -currently `true`. (One example: a Subscription which is in the process of being -programmed in the Channel data plane might receive _some_ events before the data -plane programming was complete and the Subscription was updated to set the -`Ready` condition to `true`.) +Channel MUST forward the event +[as described in event delivery](#event-delivery). The Channel MAY also forward +events to associated Subscriptions where the `Ready` condition is not currently +`true`. (One example: a Subscription which is in the process of being programmed +in the Channel data plane might receive _some_ events before the data plane +programming was complete. The Subscription would not be updated to set the +`Ready` condition to `true` until after the programming completed.) If multiple Subscriptions with the same destination are associated with the same Channel, each Subscription MUST generate one delivery attempt per Subscription. The implementation MAY attach additional event attributes or other metadata distinguishing between these deliveries. The implementation MUST NOT modify the -event payload in this process. +[event data](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#event-data) +in this process. ### Event Delivery Once a Trigger or Subscription has decided to deliver an event, it MUST do the following: -1. Resolve all URLs and delivery options, using the values in `status` for URL - resolution. +1. Read the resolved URLs and delivery options from the object's `status` + fields. 1. Attempt delivery to the `status.subscriberUri` URL following the [data plane contract](./data-plane.md). @@ -385,7 +391,7 @@ following: retry) and no event is returned, the event delivery is complete. 1. If the delivery attempt is successful (either the original request or a - retry) and an event is returned in the reply, the reply event will be + retry) and an event is returned in the reply, the reply event MUST be delivered to the `status.replyUri` destination (for Subscriptions) or added to the Broker for processing (for Triggers). If `status.replyUri` is not present in the Subscription, the reply event MUST be dropped. @@ -395,10 +401,9 @@ following: times (subject to congestion control), following the `backoffPolicy` and `backoffDelay` parameters if specified. -1. If all retries are exhausted for either the original delivery or the retry, - or if a non-retryable error is received, the event MUST be delivered to the - `deadLetterSink` in the delivery options. If no `deadLetterSink` is - specified, the event is dropped. +1. If an event (either the initial event or a reply) cannot be delivered, the + event MUST be delivered to the `deadLetterSink` in the delivery options. If + no `deadLetterSink` is specified, the event is dropped. The implementation MAY set additional attributes on the event or wrap the failed event in a "failed delivery" event; this behavior is not (currently) @@ -451,7 +456,7 @@ resource. The `apiVersion` is `eventing.knative.dev/v1` and the `kind` is - +
The category of the condition, as a short, CamelCase word or phrase.

-This is the primary key of the Conditions list when viewed as a map. +This is the primary key of the Conditions list when viewed as a map and MUST be +unique across Conditions for a given resource.

REQUIRED – No default
delivery DeliverySpec
(OPTIONAL)		A default delivery options for Triggers which do not specify more-specific options. If a Trigger specifies _any_ delivery options, this field MUST be ignored.A default delivery options for Triggers which do not specify more-specific options. If a Trigger specifies any delivery options, this field MUST be ignored. REQUIRED
@@ -474,7 +479,7 @@ resource. The `apiVersion` is `eventing.knative.dev/v1` and the `kind` is

observedGeneration int64The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction.The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction. REQUIRED
brokerstring
(Required, Immutable)		string
(REQUIRED, IMMUTABLE)		 The Broker to which this Trigger is associated. REQUIRED
subscriberduckv1.Destination
(Required)		duckv1.Destination
(REQUIRED)		 The destination for delivery of filtered events. REQUIRED
observedGeneration int64The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction.The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction. REQUIRED
channelTemplateobject
(Optional)		object
(OPTIONAL)		 Implementation-specific parameters to configure the channel. OPTIONAL
observedGeneration int64The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction.The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction. REQUIRED
channelKubernetes v1/ObjectReference
(Required, Immutable)		Kubernetes v1/ObjectReference
(REQUIRED, IMMUTABLE)		 The channel this subscription receives events from. Immutable. REQUIRED
subscriberduckv1.Destination
(Required)		duckv1.Destination
(OPTIONAL)		 The destination for event delivery. REQUIRED
replyduckv1.Destination
(Required)		duckv1.Destination
(OPTIONAL)		 The destination for reply events from spec.subscriber. REQUIRED
observedGeneration int64The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction.The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction. REQUIRED
@@ -792,13 +796,13 @@ Destination eventually resolves the supplied information to a URL by resolving - + - +
ref duckv1.KReference
(OPTIONAL)		An ObjectReference to an Addressable reference to deliver events to.An ObjectReference to a cluster resource to deliver events to. REQUIRED
uri URL (string)
(OPTIONAL)		A resolved URL to deliver events to.A URL (possibly relative to ref) to deliver events to. REQUIRED
@@ -825,7 +829,7 @@ from a [Subscription](#subscription). generation int64 Generation of the copied Subscription. - RECOMMENDED + REQUIRED subscriberUri @@ -863,7 +867,7 @@ Channel. uid UID (string) UID is used to disambiguate Subscriptions which might be recreated. - RECOMMENDED + REQUIRED generation @@ -1001,7 +1005,7 @@ KReference is a lightweight version of kubernetes map[string]string Event filter using exact match on event context attributes. Each key in the map MUST be compared with the equivalent key in the event context. All keys MUST match (as described below) the event attributes for the event to be selected by the Trigger.
- For each key specified in the filter, an attribute with that name MUST be present in the event to match. If the value corresponding to the key is non-empty, the value MUST be an exact match to attribute value in the event; an empty string MUST match all attribute values. - REQUIREDF + For each key specified in the filter, an attribute with that name MUST be present in the event to match. If the value corresponding to the key is non-empty, the value MUST be an exact (case-sensitive) match to attribute value in the event; an empty string MUST match all attribute values. + REQUIRED diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 282a74ea3..e856ca40e 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -11,7 +11,7 @@ Additionally, these roles can be combined in different ways: - **Event Processors** can be event senders, event recipients, or both. - **Event Sources** are exclusively event senders, and never act as recipients. -- **Event Sinks** are exclusively event recipients, and never act as senders. +- **Event Sinks** are exclusively event recipients, and do not send events as part of their event handling. ## Introduction diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index dec617fef..6ff8607bd 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -112,9 +112,9 @@ reference cluster-external resources such as a virtual machine or SaaS service. **Event Sources** are resources which generate events and may be configured to deliver the events to a **Destination** designated by a `sink` object in the -resource's `spec`. The Knative Eventing spec does not define any specific event -sources, but does define common interfaces for discovering and managing event -sources. +resource's `spec`. The current Knative Eventing spec does not define any +specific event sources; it only specifies the `spec.sink` partial schema at this +time. ## Eventing From ba78f252c20d3474bf85a283df6be202f9f9bb0e Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Tue, 20 Jul 2021 15:05:36 -0700 Subject: [PATCH 39/46] Normalize "Field Type" annotations for spec. Explain spec/status and api extension / schema requirements better. Fix one capitalization bug. --- specs/eventing/control-plane.md | 57 ++++++++++++++++++--------------- specs/eventing/data-plane.md | 2 +- 2 files changed, 33 insertions(+), 26 deletions(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index c3f58fd57..3377e5d64 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -419,15 +419,21 @@ following: ## Detailed Resources The following schema defines a set of REQUIRED or RECOMMENDED resource fields on -the Knative resource types. Whether a field is REQUIRED or RECOMMENDED is -denoted in the "Schema Requirement" column. Additional `spec` and `status` -fields MAY be provided by particular implementations, however it is expected -that most extension will be accomplished via the `metadata.labels` and -`metadata.annotations` fields, as Knative implementations MAY validate supplied -resources against these fields and refuse resources which specify unknown -fields. Knative implementations MUST NOT require `spec` fields outside this -implementation; to do so would break interoperability between such -implementations and implementations which implement validation of field names. +the Knative resource types. All implementations must include the REQUIRED schema +fields in their API, though implementations may implement validation of fields. +Whether a field is REQUIRED or RECOMMENDED is denoted in the "Schema +Requirement" column. Additional `spec` and `status` fields MAY be provided by +particular implementations, however it is expected that most API extensions will +be accomplished via the `metadata.labels` and `metadata.annotations` fields, as +Knative implementations MAY validate supplied resources against these fields and +refuse resources which specify unknown fields. Knative implementations MUST NOT +require `spec` fields outside this implementation; to do so would break +interoperability between such implementations and implementations which +implement validation of field names. + +For fields set in a resource `spec`, the "Field Types" column indicates whether +implementations are REQUIRED to validate that a field is set in requests, or +whether the a request is valid if the field is omitted. ### Broker @@ -601,7 +607,7 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is subscribers - []duckv1.SubscriberSpec + []duckv1.SubscriberSpec (FILLED BY SERVER) Aggregated subscription information; this array MUST be managed automatically by the controller. RECOMMENDED @@ -810,7 +816,8 @@ Destination eventually resolves the supplied information to a URL by resolving ### duckv1.SubscriberSpec SubscriberSpec represents an automatically-populated extraction of information -from a [Subscription](#subscription). +from a [Subscription](#subscription). SubscriberSpec should be populated by the +server. @@ -900,25 +907,25 @@ Channel. - + - + - + - + @@ -938,27 +945,27 @@ KReference is a lightweight version of kubernetes - + - + - - - + + + - - - - + + + +
deadLetterSinkduckv1.Destinationduckv1.Destination (OPTIONAL) Fallback address used to deliver events which cannot be delivered during the flow. An implementation MAY place limits on the allowed destinations for the deadLetterSink. RECOMMENDED
retryintint (OPTIONAL) Retry is the minimum number of retries the sender should attempt when sending an event before moving it to the dead letter sink. RECOMMENDED
backoffDelaystringstring (OPTIONAL) The initial delay when retrying delivery, in ISO 8601 format. RECOMMENDED
backoffPolicyenum
("linear", "exponential")		enum
["linear", "exponential"] (OPTIONAL)		 Retry timing scaling policy. Linear policy uses the same backoffDelay for each attempt; Exponential policy uses 2^N multiples of backoffDelay RECOMMENDED
apiVersionstringstring (REQUIRED) ApiVersion of the target reference. REQUIRED
kindstringstring (REQUIRED) Kind of the target reference. REQUIRED
namespacestringNamespace of the target resource.namestring (REQUIRED)Name of the target resource. REQUIRED
addressstringAddress used to deliver events to the Broker.REQUIREDnamespacestring (OPTIONAL)Namespace of the target resource. If unspecified, defaults to the same namespaceRECOMMENDED
@@ -1002,7 +1009,7 @@ KReference is a lightweight version of kubernetes attributes - map[string]string + map[string]string (OPTIONAL) Event filter using exact match on event context attributes. Each key in the map MUST be compared with the equivalent key in the event context. All keys MUST match (as described below) the event attributes for the event to be selected by the Trigger.
For each key specified in the filter, an attribute with that name MUST be present in the event to match. If the value corresponding to the key is non-empty, the value MUST be an exact (case-sensitive) match to attribute value in the event; an empty string MUST match all attribute values. diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index e856ca40e..934157283 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -118,7 +118,7 @@ extension**. Event recipients SHOULD NOT send these response codes in this spec version, but event senders MUST handle these response codes as errors or success as appropriate and implement described success or failure behavior. -Recipients MUST accept duplicate delivery of events, but they are not REQUIRED +Recipients MUST accept duplicate delivery of events, but they are NOT REQUIRED to detect that they are duplicates. If duplicate detection is implemented, then as specified in the [CloudEvents specification](https://github.com/cloudevents/spec/blob/v1.0.1/primer.md#id), From 8aa07dfc618837fc7d357a88ced5772d4b00d5d7 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Tue, 20 Jul 2021 16:21:42 -0700 Subject: [PATCH 40/46] A few updates to clarify that the control-plane specification indicates *server-side* requirements. --- specs/eventing/control-plane.md | 83 +++++++++++++++------------------ 1 file changed, 38 insertions(+), 45 deletions(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index 3377e5d64..1df532b07 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -10,6 +10,14 @@ An understanding of the Kubernetes API interface and the capabilities of [Kubernetes Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) is assumed. +This document describes the requirements of a conforming implementation of the +Knative Eventing resources [Broker](overview.md#broker), +[Trigger](overview.md#trigger), [Channel](#overview.md#channel) and +[Subscription](overview.md#subscription). Requirements are written from the +point of view of a service implementer; requirements on request formatting (for +example) MUST be enforced by the service implementation (e.g. by providing a 400 +error to poorly-behaved clients). + This document does not define the [data plane event delivery contract](./data-plane.md) (though it does describe how event delivery is configured). This document also does not prescribe @@ -58,8 +66,8 @@ of the different API objects. In order to validate the controls described in [Resource Overview](#resource-overview), the following Kubernetes RBAC profile -can be applied in a Kubernetes cluster. This Kubernetes RBAC is an illustrative -example of the minimal profile rather than a requirement. This Role is +can be applied in a Kubernetes cluster. This Kubernetes RBAC is an _illustrative +example_ of the minimal profile rather than a requirement. This Role is sufficient to develop, deploy, and manage event routing for an application within a single namespace. Knative Conformance tests against "MUST", "MUST NOT", "SHALL", "SHALL NOT", and "REQUIRED" conditions are expected to pass when using @@ -103,14 +111,6 @@ rules: This configuration advice does not indicate a requirement for Kubernetes RBAC support. - - ## Error Signalling See [the Knative common condition guidance](../common/error-signalling.md) for @@ -134,17 +134,16 @@ an associated Trigger destination which which does not currently have a `true` `Ready` condition, including events received by the Broker before the Trigger was created. -The annotation `eventing.knative.dev/broker.class` MAY be used to select a -particular implementation of a Broker. When a Broker is created, the -`eventing.knative.dev/broker.class` annotation and the `spec.config` field -SHOULD be populated (`spec.config` MAY be an empty object) to indicate which of -several possible Broker implementations to use. It is RECOMMENDED to default the -`eventing.knative.dev/broker.class` field on creation if it is unpopulated. Once -created, both fields MUST be immutable; the Broker MUST be deleted and -re-created to change the implementation class or `spec.config`. This pattern is -chosen to make it clear that changing the implementation class or `spec.config` -is not an atomic operation and that any implementation would be likely to result -in event loss during the transition. +The annotation `eventing.knative.dev/broker.class` SHOULD be used to select a +particular implementation of a Broker, if multiple implementations are +available. It is RECOMMENDED to default the `eventing.knative.dev/broker.class` +field on creation if it is unpopulated. Once created, the +`eventing.knative.dev/broker.class` annotation and the `spec.config` field MUST +be immutable; the Broker MUST be deleted and re-created to change the +implementation class or `spec.config`. This pattern is chosen to make it clear +that changing the implementation class or `spec.config` is not an atomic +operation and that any implementation would be likely to result in event loss +during the transition. ### Trigger Lifecycle @@ -199,7 +198,7 @@ events to an associated Subscription which does not currently have a `true` `Ready` condition, including events received by the Channel before the `Subscription` was created. -When a Channel is created, its `spec.channelTemplate` field MUST be populated to +When a Channel is created, its `spec.channelTemplate` field MAY be populated to indicate which of several possible Channel implementations to use. It is RECOMMENDED to default the `spec.channelTemplate` field on creation if it is unpopulated. Once created, the `spec.channelTemplate` field MUST be immutable; @@ -250,10 +249,6 @@ to the Subscription before prior to the Subscription's `Ready` condition being set to `true`. When a Subscription is deleted, the Channel MAY send some additional events to the Subscription's `spec.subscriber` after the deletion. - - ### Destination Resolution Both Trigger and Subscription have OPTIONAL object references (`ref` in @@ -283,9 +278,8 @@ subscriber: uri: "/update" ``` -This Destination would resolve to -`http://test./update`, as `/update` would be -interpreted relative to the `test` Service's DNS name. +This Destination would resolve to `http://test..svc/update`, as +`/update` would be interpreted relative to the `test` Service's DNS name. If a Destination includes a reference to an object which does not resolve to an absolute URL (because object does not exist, the `status.address.url` field is @@ -293,15 +287,15 @@ empty, etc), then the Trigger or Subscription MUST indicate an error by setting the `Ready` condition to `false`, and SHOULD include an indication of the error in a condition reason or type. -Both Broker and Channel MUST conform to the [Addressable -partial(#duckv1addressable)] schema. +Both Broker and Channel MUST conform to the +[Addressable partial](#duckv1addressable) schema. ## Event Routing Note that the event routing description below does not cover the actual mechanics of sending an event from one component to another; see [the data plane](./data-plane.md) contracts for details of the event transfer -mechanisms. +mechanism. ### Content Based Routing @@ -325,7 +319,7 @@ before the data plane programming was complete and the Trigger was updated to set the `Ready` condition to `true`.) If multiple Triggers match an event, one event delivery MUST be generated for -each match; duplicate matches with the same destination MUST each generate a +each match; duplicate matches with the same destination MUST each generate separate event delivery attempts, one per Trigger match. The implementation MAY attach additional event attributes or other metadata distinguishing between these deliveries. The implementation MUST NOT modify the @@ -346,12 +340,11 @@ event-delivery input and instead use an internal queueing mechanism. ### Topology Based Routing A Channel MUST publish a URL at `status.address.url` when it is able to receive -events. This URL MUST accept CloudEvents in both the -[Binary Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#31-binary-content-mode) -and -[Structured Content Mode](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#32-structured-content-mode) -HTTP formats. Before sending an HTTP response, the Channel MUST durably enqueue -the event (be able to deliver with retry without receiving the event again). +events. This URL MUST implement the receiver requirements of +[event delivery](#data-plane.md#event-delivery). Before +[acknowledging an event](data-plane.md#event-acknowledgement-and-delivery-retry), +the Channel MUST durably enqueue the event (be able to deliver with retry +without receiving the event again). For each event received by the Channel, the Channel MUST deliver the event to each associated Subscription **at least once** (where "associated" means a @@ -409,12 +402,12 @@ following: failed event in a "failed delivery" event; this behavior is not (currently) standardized. - If delivery of the reply event fails with a retryable error, the delivery to - the `deadLetterSink` SHOULD be retried up to `retry` times, following the - `backoffPolicy` and `backoffDelay` parameters if specified. Alternatively, - implementations MAY use an equivalent internal mechanism for delivery (for - example, if the `ref` form of `deadLetterSink` points to a compatible - implementation). + If delivery of the dead-letter event fails with a retryable error, the + delivery to the `deadLetterSink` SHOULD be retried up to `retry` times, + following the `backoffPolicy` and `backoffDelay` parameters if specified. + Alternatively, implementations MAY use an equivalent internal mechanism for + delivery (for example, if the `ref` form of `deadLetterSink` points to a + compatible implementation). ## Detailed Resources From 7a678bde2e951197004cac2726664cb510f52c77 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Wed, 21 Jul 2021 13:01:58 -0700 Subject: [PATCH 41/46] Require that Subscriptions reference Channels in the same namespace. --- specs/eventing/control-plane.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index 1df532b07..418c48bd3 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -213,7 +213,9 @@ A Subscription MAY be created before the referenced Channel indicated by its `spec.channel` field. The `spec.channel` object reference MAY refer to either a `messaging.knative.dev/v1` Channel resource, or another resource which meets the `spec.subscribers` and `spec.delivery` required elements in the Channelable duck -type. If the referenced `spec.channel` does not currently exist or its `Ready` +type. The `spec.channel` reference MUST be to an object in the same namespace; +specifically, the `spec.channel.namespace` field must be unset or the empty +string. If the referenced `spec.channel` does not currently exist or its `Ready` condition is not `true`, then the Subscription's `Ready` condition MUST NOT be `true`, and the reason SHOULD indicate that the corresponding Channel is missing or not ready. @@ -673,8 +675,8 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is channel - Kubernetes v1/ObjectReference
(REQUIRED, IMMUTABLE) - The channel this subscription receives events from. Immutable. + KReference
(REQUIRED, IMMUTABLE) + The channel this subscription receives events from. namespace may not be set (must refer to a Channel in the same namespace). Immutable. REQUIRED From fad55f54b6e275aa6abef01a434e4b8fb64a200b Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 22 Jul 2021 11:30:44 -0700 Subject: [PATCH 42/46] Allow additional Destinaiton resolution mechanisms. --- specs/eventing/control-plane.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index 418c48bd3..215e7dd2b 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -266,7 +266,8 @@ URL; two mechanisms MUST be supported: service's cluster-local DNS name (of the form `http://..svc/`). -If both the `ref` field and the `uri` fields are set on a Destination, then the +Implementations MAY implement additional resolution mechanisms. If both the +`ref` field and the `uri` fields are set on a Destination, then the Destination's address MUST be the `uri` interpreted relative to the resolved URI of the `ref` field. This can be used, for example, to refer to a specific URL off a referenced domain name, like so: From 8333aa7e9b9e283f0f3d8dfcb4e8426433667b2d Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 22 Jul 2021 14:35:59 -0700 Subject: [PATCH 43/46] Fix two Doug nits. --- specs/eventing/data-plane.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 934157283..60e983457 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -11,7 +11,8 @@ Additionally, these roles can be combined in different ways: - **Event Processors** can be event senders, event recipients, or both. - **Event Sources** are exclusively event senders, and never act as recipients. -- **Event Sinks** are exclusively event recipients, and do not send events as part of their event handling. +- **Event Sinks** are exclusively event recipients, and do not send events as + part of their event handling. ## Introduction @@ -171,11 +172,10 @@ the `spec.subscriber` address. A recipient MAY reply to any HTTP POST with a `200` response to indicate that the event was processed successfully, with or without a response payload. If the -recipient will _never_ provide a response payload, the `202` response code is -also acceptable. Responses with a `202` response code MUST NOT be processed as -reply events; even if the response can be interpreted as a CloudEvent, the -status monitor for the accepted-but-not-completed request should not be routed -further. +recipient does not produce a response payload, the `202` response code is also +acceptable. Responses with a `202` response code MUST NOT be processed as reply +events; even if the response can be interpreted as a CloudEvent, the status +monitor for the accepted-but-not-completed request SHOULD NOT be routed further. If a recipient chooses to reply to a sender with a `200` response code and a reply event in the absence of a `Prefer: reply` header from the sender, the From bef17aa5bcb14edc90834ffaee4fb5b313cf8952 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 22 Jul 2021 14:41:50 -0700 Subject: [PATCH 44/46] Remove schema requirements. --- specs/eventing/control-plane.md | 96 +++++---------------------------- 1 file changed, 12 insertions(+), 84 deletions(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index 215e7dd2b..82b7ea465 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -414,22 +414,21 @@ following: ## Detailed Resources -The following schema defines a set of REQUIRED or RECOMMENDED resource fields on -the Knative resource types. All implementations must include the REQUIRED schema -fields in their API, though implementations may implement validation of fields. -Whether a field is REQUIRED or RECOMMENDED is denoted in the "Schema -Requirement" column. Additional `spec` and `status` fields MAY be provided by -particular implementations, however it is expected that most API extensions will -be accomplished via the `metadata.labels` and `metadata.annotations` fields, as -Knative implementations MAY validate supplied resources against these fields and -refuse resources which specify unknown fields. Knative implementations MUST NOT -require `spec` fields outside this implementation; to do so would break -interoperability between such implementations and implementations which -implement validation of field names. +The following schema defines a set of REQUIRED resource fields on the Knative +resource types. All implementations MUST include all schema fields in their API, +though implementations may implement validation of fields. Additional `spec` and +`status` fields MAY be provided by particular implementations, however it is +expected that most API extensions will be accomplished via the `metadata.labels` +and `metadata.annotations` fields, as Knative implementations MAY validate +supplied resources against these fields and refuse resources which specify +unknown fields. Knative implementations MUST NOT require `spec` fields outside +this implementation; to do so would break interoperability between such +implementations and implementations which implement validation of field names. For fields set in a resource `spec`, the "Field Types" column indicates whether implementations are REQUIRED to validate that a field is set in requests, or -whether the a request is valid if the field is omitted. +whether the a request is valid if the field is omitted. Field in a resource +`status` MUST be set by the server implementation. ### Broker @@ -447,19 +446,16 @@ resource. The `apiVersion` is `eventing.knative.dev/v1` and the `kind` is Field Name Field Type Description - Schema Requirement config KReference
(OPTIONAL) A reference to an object which describes the configuration options for the Broker (for example, a ConfigMap). - RECOMMENDED delivery DeliverySpec
(OPTIONAL) A default delivery options for Triggers which do not specify more-specific options. If a Trigger specifies any delivery options, this field MUST be ignored. - REQUIRED @@ -470,31 +466,26 @@ resource. The `apiVersion` is `eventing.knative.dev/v1` and the `kind` is Field Name Field Type Description - Schema Requirement conditions See Error Signalling Used for signalling errors, see link. - REQUIRED observedGeneration int64 The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction. - REQUIRED address duckv1.Addressable Address used to deliver events to the Broker. - REQUIRED deadLetterSinkUri URL (string) If spec.delivery.deadLetterSink is specified, the resolved URL of the dead letter address. - REQUIRED @@ -514,31 +505,26 @@ resource. The `apiVersion` is `eventing.knative.dev/v1` and the `kind` is Field Name Field Type Description - Schema Requirement broker string
(REQUIRED, IMMUTABLE) The Broker to which this Trigger is associated. - REQUIRED filter TriggerFilter
(OPTIONAL) Event filters which are used to select events to be delivered to the Trigger's destination. - REQUIRED subscriber duckv1.Destination
(REQUIRED) The destination for delivery of filtered events. - REQUIRED delivery DeliverySpec
(OPTIONAL) Delivery options for this Trigger. - RECOMMENDED @@ -549,31 +535,26 @@ resource. The `apiVersion` is `eventing.knative.dev/v1` and the `kind` is Field Name Field Type Description - Schema Requirement conditions See Error Signalling Used for signalling errors, see link. - REQUIRED observedGeneration int64 The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction. - REQUIRED subscriberUri URL (string) The resolved address of the spec.subscriber. - REQUIRED deadLetterSinkUri URL (string) If spec.delivery.deadLetterSink is specified, the resolved URL of the dead letter address. - REQUIRED @@ -593,25 +574,21 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is Field Name Field Type Description - Schema Requirement channelTemplate object
(OPTIONAL) Implementation-specific parameters to configure the channel. - OPTIONAL subscribers []duckv1.SubscriberSpec (FILLED BY SERVER) Aggregated subscription information; this array MUST be managed automatically by the controller. - RECOMMENDED delivery DeliverySpec
(OPTIONAL) Default delivery options for Subscriptions which do not specify more-specific options. If a Subscription specifies _any_ delivery options, this field MUST be ignored. - REQUIRED @@ -622,37 +599,31 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is Field Name Field Type Description - Schema Requirement conditions See Error Signalling Used for signalling errors, see link. - REQUIRED observedGeneration int64 The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction. - REQUIRED address duckv1.Addressable Address used to deliver events to the Broker. - REQUIRED subscribers []duckv1.SubscriberStatus Resolved addresses for the spec.subscribers (subscriptions to this Channel). - RECOMMENDED deadLetterSinkUri URL (string) If spec.delivery.deadLetterSink is specified, the resolved URL of the dead letter address. - REQUIRED @@ -672,31 +643,26 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is Field Name Field Type Description - Schema Requirement channel KReference
(REQUIRED, IMMUTABLE) The channel this subscription receives events from. namespace may not be set (must refer to a Channel in the same namespace). Immutable. - REQUIRED subscriber duckv1.Destination
(OPTIONAL) The destination for event delivery. - REQUIRED reply duckv1.Destination
(OPTIONAL) The destination for reply events from spec.subscriber. - REQUIRED delivery DeliverySpec
(OPTIONAL) Delivery options for this Subscription. - RECOMMENDED @@ -707,25 +673,21 @@ resource. The `apiVersion` is `messaging.knative.dev/v1` and the `kind` is Field Name Field Type Description - Schema Requirement conditions See Error Signalling Used for signalling errors, see link. - REQUIRED observedGeneration int64 The latest metadata.generation that the reconciler has attempted. If observedGeneration is updated, conditions MUST be updated with current status in the same transaction. - REQUIRED physicalSubscription PhysicalSubscriptionStatus The fully resolved values for spec endpoint references. - REQUIRED @@ -752,13 +714,11 @@ There are no `spec` requirements for Addressable. Field Name Field Type Description - Schema Requirement address duckv1.Addressable Address used to deliver events to the resource. - REQUIRED @@ -771,13 +731,11 @@ There are no `spec` requirements for Addressable. Field Name Field Type Description - Schema Requirement url URL (string) Address used to deliver events to the Addressable. - REQUIRED @@ -793,19 +751,16 @@ Destination eventually resolves the supplied information to a URL by resolving Field Name Field Type Description - Schema Requirement ref duckv1.KReference
(OPTIONAL) An ObjectReference to a cluster resource to deliver events to. - REQUIRED uri URL (string)
(OPTIONAL) A URL (possibly relative to ref) to deliver events to. - REQUIRED @@ -820,37 +775,31 @@ server. Field Name Field Type Description - Schema Requirement uid UID (string) UID is used to disambiguate Subscriptions which might be recreated. - REQUIRED generation int64 Generation of the copied Subscription. - REQUIRED subscriberUri URL (string) The resolved address of the Subscription's spec.subscriber. - REQUIRED replyUri URL (string) The resolved address of the Subscription's spec.reply. - REQUIRED delivery DeliverySpec The resolved Subscription delivery options. The deadLetterSink SHOULD use the uri form. - REQUIRED @@ -864,31 +813,26 @@ Channel. Field Name Field Type Description - Schema Requirement uid UID (string) UID is used to disambiguate Subscriptions which might be recreated. - REQUIRED generation int64 Generation of the copied Subscription. - RECOMMENDED ready kubernetes v1/ConditionStatus Ready status of the Subscription's programming into the Channel data plane. - REQUIRED message string A human readable message indicating details of ready status. - REQUIRED @@ -899,31 +843,26 @@ Channel. Field Name Field Type Description - Schema Requirement deadLetterSink duckv1.Destination (OPTIONAL) Fallback address used to deliver events which cannot be delivered during the flow. An implementation MAY place limits on the allowed destinations for the deadLetterSink. - RECOMMENDED retry int (OPTIONAL) Retry is the minimum number of retries the sender should attempt when sending an event before moving it to the dead letter sink. - RECOMMENDED backoffDelay string (OPTIONAL) The initial delay when retrying delivery, in ISO 8601 format. - RECOMMENDED backoffPolicy enum
["linear", "exponential"] (OPTIONAL) Retry timing scaling policy. Linear policy uses the same backoffDelay for each attempt; Exponential policy uses 2^N multiples of backoffDelay - RECOMMENDED @@ -937,31 +876,26 @@ KReference is a lightweight version of kubernetes Field Name Field Type Description - Schema Requirement apiVersion string (REQUIRED) ApiVersion of the target reference. - REQUIRED kind string (REQUIRED) Kind of the target reference. - REQUIRED name string (REQUIRED) Name of the target resource. - REQUIRED namespace string (OPTIONAL) Namespace of the target resource. If unspecified, defaults to the same namespace - RECOMMENDED @@ -972,25 +906,21 @@ KReference is a lightweight version of kubernetes Field Name Field Type Description - Schema Requirement subscriberUri URL (string) Resolved address of the spec.subscriber. - REQUIRED replyUri URL (string) Resolved address of the spec.reply. - REQUIRED deadLetterSinkUri URL (string) Resolved address of the spec.delivery.deadLetterSink. - REQUIRED @@ -1001,7 +931,6 @@ KReference is a lightweight version of kubernetes Field Name Field Type Description - Schema Requirement attributes @@ -1009,6 +938,5 @@ KReference is a lightweight version of kubernetes Event filter using exact match on event context attributes. Each key in the map MUST be compared with the equivalent key in the event context. All keys MUST match (as described below) the event attributes for the event to be selected by the Trigger.
For each key specified in the filter, an attribute with that name MUST be present in the event to match. If the value corresponding to the key is non-empty, the value MUST be an exact (case-sensitive) match to attribute value in the event; an empty string MUST match all attribute values. - REQUIRED From ac546356a9a1936b08fbc405447423e2a08aac90 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 22 Jul 2021 14:44:03 -0700 Subject: [PATCH 45/46] Align SHOULD NOT / MUST NOT --- specs/eventing/data-plane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 60e983457..6a713cbd4 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -175,7 +175,7 @@ the event was processed successfully, with or without a response payload. If the recipient does not produce a response payload, the `202` response code is also acceptable. Responses with a `202` response code MUST NOT be processed as reply events; even if the response can be interpreted as a CloudEvent, the status -monitor for the accepted-but-not-completed request SHOULD NOT be routed further. +monitor for the accepted-but-not-completed request MUST NOT be routed further. If a recipient chooses to reply to a sender with a `200` response code and a reply event in the absence of a `Prefer: reply` header from the sender, the From f5d15dcbc93e5330ca36592d1a0fd9fca6912646 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Thu, 22 Jul 2021 14:47:28 -0700 Subject: [PATCH 46/46] Capitalize "MAY" --- specs/eventing/control-plane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/eventing/control-plane.md b/specs/eventing/control-plane.md index 82b7ea465..dd36546f6 100644 --- a/specs/eventing/control-plane.md +++ b/specs/eventing/control-plane.md @@ -416,7 +416,7 @@ following: The following schema defines a set of REQUIRED resource fields on the Knative resource types. All implementations MUST include all schema fields in their API, -though implementations may implement validation of fields. Additional `spec` and +though implementations MAY implement validation of fields. Additional `spec` and `status` fields MAY be provided by particular implementations, however it is expected that most API extensions will be accomplished via the `metadata.labels` and `metadata.annotations` fields, as Knative implementations MAY validate