Limitations of golang.org/x/tools/go/analysis

[scott-cotton]

While prototyping pal, I've learned some cool things about golang.org/x/tools/go/analysis, as that is the initial target.

However, I'm having some problems with some aspects of it.

The dependency relation

go/analysis implements a 2-axis dependency relation where an analyser may depend on other analysers, and at once package import dependencies are supported. This is nice, but it has some limitations w.r.t. our needs in pal.

(Note the API is quite clearly documented as experimental; I am sure it is natural to consider such limitations and build on what we have learned.)

That said:

1. It is only bottom-up. Top down analysis, and iterations in which one analysis can render another more precise, are not supported.
2. Results between two analyses, A, B, where A consumes B, are in one form (arbitrary, but unfortunately using reflect to key different analysers) -- while results shared from one package to another must take another, distinct form: either associated with the package or keyed by type.Object's. This is confusing.
3. Incrementality of type.Objects are filtered by gcexportdata. Pointer analysis needs to make reference to things which are not exported.

Fact "Types"

Nit: Facts are in a field called FactTypes but they are not types.

SSA/Types are pointer heavy

Systems such as soufflé, and probably CodeQL, consider results, including program representations, as pure data. They can easily be stored, made deterministic, etc.

However, SSA and go/types and go/ast are very pointer heavy, and pointers are often map keys. The result is that it is impossible to render deterministic and difficult to store or consider as pure data.

Disk Caching

Caching a la go-cache is in place in some tools, but it is based on commands and their outputs. I think we need to be able to cache intermediary results which can be consumed and even refined over time, with the ability to reference a more semantic ir than go/ast, such as go/ssa.

The heavy use of pointers makes this difficult.

Workarounds?

I am trying to find a way to workaround these problems with pal. The first objective is to make pal work in the existing Go analysis framework. I believe I have a lead.

The idea is to just bite the bullet for caching and referring to SSA, and to compute things which we need based on that framework but without referring directly to it. But I need to avoid using ObjectFacts even though I refer to objects, because the export data filtering is not what we want.

Then the idea is to use a single dummy PkgFact to trigger package-wise deps and to bypass the export mechanism with a side channel for results: each pass will neither produce nor consume any real facts, just a single dummy fact. The result type will be a handle to the side-channel, which stores results for all packages, and this will also be the dummy fact.

If this works, it will provide a first feel of how a per-package results persistant cache could work.

Thoughts?