Integrating local-first authentication with Automerge

[HerbCaudill]

Recently I've been working with @pvh on Automerge Repo and we've been kicking around some ideas on how to provide authentication/authorization/encryption, either using localfirst/auth directly, or stealing ideas from it.

Part of what makes this awkward is that localfirst/auth manages group membership using my own homemade CRDT engine, crdx. I couldn't use Automerge because I needed custom logic around detecting and resolving conflicts.¹

As we all know, to build a real-world local-first app using Automerge, you have to come up with solutions for storage and networking; you need something that's always online to support asynchronous forwarding of changes; you need to orchestrate synchronization; and so on. Automerge Repo will hopefully offer an "official" solution for all that.

Of course crdx (and lf/auth by extension) needs all those things too, but it has different storage formats, sync message formats, etc., so you can't just use the same infrastructure for both: If you're trying to make a local-first app using localfirst/auth for authentication and Automerge for application state, you have to come up with parallel solutions for all that stuff.

I can think of a couple of different (not mutually exclusive) approaches to avoid all that duplication:

• A standards approach would be, for example, to have a sync protocol that can synchronize any DAG, using a hashmap of nodes and associated dependency information, but agnostic to the content of the nodes. So Automerge and lf/auth could use the same sync client code and the same physical sync server.

• An extensibility approach would be for Automerge to support pluggable datatypes, so that instead of using crdx I could add a group membership datatype to Automerge with my custom rules for conflicts. Then lf/auth could piggyback whatever options emerge in the Automerge ecosystem for storage/networking/syncing.

Curious what y'all think.

---

¹For example, if a group member makes a change to the group while concurrently being removed from the group, that's a conflict, and it's resolved by discarding the change.

[HerbCaudill]

As a side note, datatype extensibility would give us a better answer for questions that periodically come up like Generalizing the Counter type, where the underlying issue is that a JSON CRDT just can't represent some kinds of logical conflicts.

[ept]

Some people might want to use Automerge Repo with centralised access control (e.g. a server-side ACL), and for those users the complexity of a decentralised access control system should be avoidable. However, for users with a decentralised setup, I am supportive of integrating localfirst/auth with Automerge Repo, since I believe it's the best decentralised access control system currently available.

Having spent a while thinking about algorithms for decentralised access control, I've also come to the conclusion that it requires a custom CRDT algorithm — it's not possible to achieve sensible semantics with a generic JSON CRDT algorithm like currently provided in Automerge. Whether that custom algorithm is implemented using crdx or otherwise is an implementation detail that I'm not particularly fussed about. I have been thinking about proposing some tweaks to the conflict resolution algorithm in localfirst/auth, but they don't affect the overall architecture so we can discuss them separately.

We will need to figure out is how the Automerge sync protocol interfaces with lf/auth. My initial thought is that when a connection between two peers is set up, lf/auth would first sync the hash graph of ACL operations (to make sure both peers are up-to-date on the group membership), then the peers establish a mutually authenticated, encrypted channel (so that each party knows who it's talking to), and then we run the Automerge sync protocol over that secure channel. With this design, the ACL hash graph sync happens at a different protocol phase from the sync of Automerge document data. Moreover, the lf/auth hash graph has different characteristics (much slower rate of change; cryptographically signed operations) than Automerge document changes. I think it therefore might make sense for these to be two separate protocols, even though they are both hash graph syncs.

At some point we might want an implementation of lf/auth in Rust, so that peers using the native Automerge library outside of a Wasm context can interoperate with browser/Node peers. We could also consider making lf/auth an official part of the Automerge ecosystem, like Automerge Repo. But I'm inclined to keep it a separate library from core Automerge, to keep the core library as lightweight as possible for the users who don't need cryptographic access control.

[pvh]

Good thoughts all around here -- I think we should consider this through a lens of composability and layering. Can we decompose Automerge somehow so that useful pieces can be adopted by other CRDTs? Is the sync protocol the thing to propagate?

[HerbCaudill]

I think we should consider this through a lens of composability and layering. Can we decompose Automerge somehow so that useful pieces can be adopted by other CRDTs?

That might make sense in terms of how this is implemented internally, but I guess as a developer I personally find the "build a plugin to this thing that already works" model much less daunting than the "here are some parts, pur them together yourself" model. It's easier to wrap my head around it because I only have to think about the one thing I'm contributing, I don't have to understand the entire internal architecture of the larger thing.

[HerbCaudill]

To clarify what I have in mind, because I think the title of the discussion makes it sound misleadingly ambitious:

Automerge already offers a few datatypes in addition to the "native" JSON stuff: Counter, Text, etc. Could we let people add their own datatypes but otherwise keep using Automerge the same way?

In the case of lf/auth, this might be a "Team" datatype.

That would be all, for now. Figuring out exactly how authentication, access control, encryption, etc. would work with Automerge would be downstream of that.