support both nats and redis for state stores ?

[gedw99]

Have an idea...

I searched for NATS in this repo and found nothing.

https://github.com/zeromicro/go-queue/blob/master/natsq/consumer.go has a basic nats PUB SUB.
However I dont see it used in https://github.com/zeromicro/go-zero. Any reason why not ?

There is Redis, and I presume this is used for sessions ?
Redis works, but Nats gives other things on top of Redis. Nats can do Sessions of course.

Some not well know things about NAT:

1. can run embedded, as a local LEAF node. so very high perf.
2. can run as a cluster. typically 3 per data centre.
3. can run as a global supper cluster. So you can have 5 data centres, with 3 cluster in each.

NATS Leaf nodes and self autonomous and continue working even if the network dies, but at the same time linked up to the cluster and super cluster. If the network dies can be vital, because the nature of a Message queue is that you want to fire and forget and to know ( for sure ) that the message will get through once the network comes back up.

NATS Leaf node can also be run as In-Proess too. This is kind of a new feature of NATS. It can be useful for certain aspects also.

Here is the In-Process code links:

Here is the Server: https://github.com/nats-io/nats-server/blob/main/server/server.go#L2734

Here is the client: https://github.com/nats-io/nats.go/blob/main/nats.go#L262C6-L262C19

There's lots of options with NATS.

Also you can upgrade NATS Server using canary pattern with zero downtime. It's called lame duck mode in NATS.

The JWTS used with GRPC or really any web apps, can be fully integrated with NATS using the NATS auth Callout feature. So a Users auth token flow all the way through NATS, such that the streams are secured against that user, and so when a User is deleted from the system they are deleted from NATS too.

The other tings you can do with nats is make Databases that are not designed for Scale out easily scale out using the WAL of the DB. SQLite is easy to do this with. This is called the CDC pattern, and NATS can do Multi Master, so you can write to any DB instance and the other other will eventually get it. Write clashes are simply LWW ( Last Write Wins ). You can go further and make it CRDT but it's much more work and generally not needed.

Testing is also really easy with NATS because you dont have to mock, and can just use the NATS embedded server:

import (
	"fmt"
	"log"
	"testing"

	"github.com/nats-io/nats-server/v2/server"
	natsserver "github.com/nats-io/nats-server/v2/test"
	"github.com/nats-io/nats.go"
)

const TEST_PORT = 4222

func RunServerOnPort(port int) *server.Server {
	opts := natsserver.DefaultTestOptions
	opts.Port = port
	return RunServerWithOptions(&opts)
}

func RunServerWithOptions(opts *server.Options) *server.Server {
	return natsserver.RunServer(opts)
}

func TestServerRestart(t *testing.T) {
	s := RunServerOnPort(TEST_PORT)
        s.EnableJetStream(&server.JetStreamConfig{})
	go func() {
		s.Start()
	}()
	time.Sleep(time.Second * 5)

        // use nats as normal
	sUrl := fmt.Sprintf("nats://127.0.0.1:%d", TEST_PORT)
	if nc, err := nats.Connect(sUrl); err != nil {
		panic(err)
	} else {
		js, err := nc.JetStream()
		if err != nil {
			log.Fatal("error creating jetstream context:", err)
		}
		if _, err = js.AddStream(&nats.StreamConfig{
			Name:     "natss",
			Subjects: []string{fmt.Sprintf("%s.*", "natss")},
		}); err != nil {
			log.Fatal("error adding stream:", err)
		}

	}
	defer s.Shutdown()
}

Anyways, just an idea... Wondering what people think.