Switch from typeforce to something else

[junderw]

General musings:

• Typia - https://typia.io/docs/
  • I really like the API and the ease of use. The speed is also very good. However, it's relatively young and is only being maintained by one person, so longevity might be a concern.
• zod - https://zod.dev/
  • This seems like the one that everyone likes. It seems a bit complex, and it's slow. It is also essentially maintained by one person, but they have almost 100 Github sponsors, which at least shows the willingness for people to pay to support the project.

I am also open to other suggestions.

Replacing typeforce in the bitcoinjs ecosystem will be breaking changes most likely, since a lot of these require changes to the typescript types.

Once we've decided on one, I'll go ahead and audit their codebase.

[pajasevi]

I did some research and from what I observed, there isn't a widely-used validation library that has more than one maintainer. Some people might suggest AJV but that doesn't really fit the use-case of bitcoinjs as it's JSON schema based. Other libraries that have good performance either aren't widely used, rely on JSON schema or have complicated API.

Regarding the two libraries already mentioned: Zod is certainly well established in the space and is widely used but the speed might be a concern. Typia on the other hand is super fast due being strictly Ahead Of Time compiled and looks healthy right now in terms of development. Also the API is very intuitive.

One thing to look at is also library dependencies: Typia has 4 dependencies while Zod has zero.

So while I think that Typia might be a better fit due to API, performance and ease-of-use, I would be cautious.

[motorina0]

What are the current/future limitations of typeforce?
Why is the change needed?

[junderw]

@pajasevi mentioned a few things in the PR that mentions this issue...

Perhaps @pajasevi could go into further details.

[pajasevi]

@motorina0 I see several issues with typeforce.

• It is unmantained - last change 2 years ago
• no ES module support - minor issue in this case
• very bad extendability - current set of type checks is very limited and the API is very clunky when you want to add a sophisticated extension, like I tried with Uint8Array of length 32
• no Typescript support - personally my biggest issue with the library is that it doesn't have any type definitions for its API (not aven a third-party @types package) and results of type assertions therefore aren't typed at all

This could be easily solved even by having a standard set of type guard functions that could be reused in the bitcoinjs ecosystem. Using a different third-party library that supports all the features might be easier of course.

[samholmes]

May I suggest Cleaners? Simple API and the conventions lend to trivial extensibility.

[pajasevi]

@samholmes That might be good for JSON types but we're looking for something that can validate low-level JS natives like Uint8Array and with specific length as well. From a quick look, I don't see a way to trivially extend that library to do that.

[samholmes]

@samholmes That might be good for JSON types but we're looking for something that can validate low-level JS natives like Uint8Array and with specific length as well. From a quick look, I don't see a way to trivially extend that library to do that.

Cleaners are just functions that validate some unknown input and return the correct type, or throws if the input is invalid. Therefore an asUint8Array [custom cleaner](https://cleaners.js.org/#/guides?id=writing-custom-cleaners} would look something like:

function asUint8Array(value: unknown): Uint8Array {
  if (value instanceof Uint8Array) return value
  throw new TypeError('Expected Uint8Array')
}

For length, I'd create a generic cleaner over it:

const asLengthed = <T extends {length: number}>(asT: Cleaner<T>, length: number) => (value: unknown) => {
  const out = asT(value)
  if (out.length === length) return out
  throw new TypeError(`Expected 'length' to be ${length}`)
}

Using it:

const as21ByteUint8Array = asLengthed(asUint8Array, 21)

const cleanData = as21ByteUint8Array(unknownData)

This is an example of a generic which takes arguments and returns a new cleaner. It is high-order in that it takes a cleaner another cleaner. Aside from using a generic, another more straight forward approach would be:

const as21LengthObject = (value:unknown) => asUint8Array(asObject({
  length: asValue(21)
})(value))

This cleaner is not generic, and will do all the type checks specific to the example. However, the Uint8Array check only is there to infer the correct output type really (which is important at runtime).

There are many ways to achieve what is needed with cleaners. This gives you the framework to design you runtime type definitions and extend from the initial API contract: plain functions which either throw or return the correct type at runtime.

[pajasevi]

@samholmes To be honest, I don't see much added value over custom type guards in this case. And since I assume that @junderw would probably want to go with something that is commonly used, battle tested and has a good chance of being maintained in the future, this is probably not the right way to go.

[samholmes]

@pajasevi Fair enough. Though to the point of battle testing: cleaners has been battle tested within the Edge's open source codebase. It's actively maintained, and relatively straight-forward to reason about it's implementation (low footprint). The only thing it doesn't have going for it is the "commonly used" aspect, which shouldn't be a show-stopper necessarily (popularity doesn't necessarily mean it's the right choice for the job).

The thing to consider in open source is the level of effort for outside contributors to understand how to contribute. By choosing something like Zod based on popularity, then you gain the network effect of its popularity and all the devs to go with it.

On the another hand, if you choose a library with low API footprint, then you may reach more developer contributions based on it's simplicity to understand.

[pajasevi]

I have found out through direct experience (replacing typeforce on my fork of bip32) that the best replacement is the ow library. It has very similar API to typeforce and has a great support for composition.
And of course it has a full TS support.

The only caveat is that it is pure ESM which means that any project importing it has to be ESM as well.