Framework needs to help test authors understand how to pass state between steps

[n3wscott]

One issue has the testing framework makes it had to pass state between components. How do we solve this? Is it a real problem?

[slinkydeveloper]

It's a real problem, we need to find a solution for this. An example:

	f.Setup("install source", func(ctx context.Context, t *testing.T) {
		u, err := broker.Address(ctx, brokerName, features.Interval, features.Timeout)
		if err != nil || u == nil {
			t.Error("failed to get the address of the broker", brokerName, err)
		}
		eventshub.Install(sourceName, eventshub.StartSenderURL(u.String()), eventshub.InputEvent(event))(ctx, t)
	})

In this case I wanted to compose the Address retrieval (which might be a stepfn itself) with eventshub.Install, propagating the address. It would be nice to have a solution that just compose these two steps together

[n3wscott]

f.Setup("install source", func(ctx context.Context, t *testing.T) {
	// to a bunch of steps that create a thing and it eventually gets a status.address `fooAddress`
  // ...but this might not happen quickly.
}.
Setup("install source", func(ctx context.Context, t *testing.T) {
	// Use that resulting `fooAdddress`
	// This will never work at the moment.
})

OK brain wave....

It was my intention that the requirement timing would let you block until it passes while setup phases were happening so the testing phase works.

BUT! what if we change how Requirement works, and it gets paired with Setup as a prereq for running Setups. The setup phase is optional, example:

f.Setup("install foo", func(ctx context.Context, t *testing.T) {
	// to a bunch of steps that create a thing and it eventually gets a status.address `fooAddress`
  // ...but this might not happen quickly.
}.
Require("install thing targeting foo", func(ctx context.Context, t *testing.T) {
	// Wait for `fooAdddress` to be on the target resource.
}).
Setup("install thing targeting foo", func(ctx context.Context, t *testing.T) {
	// Use that resulting `fooAdddress`
})
Require("wait for all things to be ready", func(ctx context.Context, t *testing.T) {
	// this is an example of something that has no setup pair.
})

How to know a Require is paired with a Setup? Match the name of the step.

ok this does not solve the problem at hand, so I think we also introduce a config store that can be blocking:

f.Setup("install foo", func(ctx context.Context, t *testing.T) {
	// to a bunch of steps that create a thing and it eventually gets a status.address `fooAddress`
  // ...but this might not happen quickly.
}.
Require("install thing targeting foo", func(ctx context.Context, t *testing.T) {
	// Wait for `fooAdddress` to be on the target resource.
  fooAddress := "some valid value pulled from the cluster."
  f.SetConfig(ctx, "fooAddress", fooAddress)
}).
Setup("install thing targeting foo", func(ctx context.Context, t *testing.T) {
	// Use that resulting `fooAdddress`
  f.GetConfig(ctx, "fooAddress", &fooAddress) // maybe a return error, timeout, maybe it blocks? idk yet.
})
Require("wait for all things to be ready", func(ctx context.Context, t *testing.T) {
	// this is an example of something that has no setup pair.
})

feature.Feature looks like this now

type Feature struct {
	Name  string
	Steps []Step
}

But we could change it to be:

type Feature struct {
	Name  string
	Steps []Step
	config.Store
}

and config.Store is something like

type config.Store interface {
  SetConfig(ctx context.Context, key string, value interface{})
  GetConfig(ctx context.Context, key string, target interface{}) // target must be a pointer type?
}

[dprotaso]

Then steps have two places (context &| config) to get things from and they're both untyped and have loose contracts.

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

[dprotaso]

/lifecycle frozen

[dprotaso]

I wonder if step function should change to return a potentially new context

ie. func Step(context.Context, testing.T) context.Context