Deterministic CBOR (CDE) and Determinism with Numeric Reduction (dCBOR)

[ChristopherA]

For cryptographic signatures, determinism is essential. The purpose of determinism is to ensure that semantically equivalent data items are encoded into identical byte streams. The IETF CBOR spec RFC-8949 has a section on determinism (§4.2), however, it is underspecified.

In the last year there are two documents advancing to address this underspecification, and there are also libraries in a number of languages that support one or both.

The base document is CBOR Common Deterministic Encoding (CDE) which cleans up those §4.2 issues. It also describe how §3 Application Profiles function.

At this point, there is only one Application Profile, which is dCBOR, which adds Numeric Reduction, and is slightly more restrictive then CDE. Numeric reduction further ensures that semantically equal numeric values (e.g. 2 and 2.0) are encoded into identical byte streams (e.g. 0x02) by encoding "Integral floating point values" (floating point values with a zero fractional part) as integers when possible. This also means that the three representations of a zero number in CBOR (0, 0.0, -0.0 in diagnostic notation) are all reduced to the basic integer 0 (with preferred encoding 0x00).

There are some other sharp edges in the dCBOR I-D that are restricted, such as requiring lexicographic ordering of map keys such as maps must not have duplicate keys, restricts various NAN values. dCBOR also doesn't support bigint.

There are more details on dCBOR at https://developer.blockchaincommons.com/dcbor/ and there are now multiple libraries in multiple languages that support it, including Rust, Typescript, Ruby, and Swift. I have heard that there are more now that support dCBOR, including Java, but I've not validated them.

Both of these deterministic CBOR documents are have been adopted by the CBOR-WG and are relatively mature, and are on different RFC tracks.

I am one of the co-authors of the dCBOR I-D, which inspired the CDE document. Our reasoning for support for numeric reduction is address many of the problems across different languages at a lower level than the semantic layer. You don't need a schema to order maps and validate numbers — schemas instead can focus the semantic layer, rather than the data layer. We leverage this with great utility in our higher level structured data model, triple store, and Merkle tree based Gordian Envelope I-D (more details and code).

I'd like to suggest that this document leverage the CDE/dCBOR drafts and conform with them for determinism. If there is some reason they can't, let the CBOR community know, or consider creating your own Application Profile with your determinism rules.