GSoC 2022 Q&A

[larryshamalama]

Following a discussion with @brandonwillard yesterday, many concepts are still difficult for me to understand and I often find myself with many questions. To facilitate conversation, this discussion board can serve as a place where I ask such questions and receive pointers to hopefully get unstuck in my progress.

[brandonwillard]

I changed this to a general Discussion, since the single Q & A format might not work well for multiple questions and ongoing discussions.

[larryshamalama]

Why is a FunctionGraph necessary as inputs in local optimizers? As in, what information do they provide outside of node? Example here. Do they serve as listeners as we primarily work with node which, to my understand, is a subgraph itself?

Perhaps more generally, what purpose do FunctionGraph objects serve? Reading the description here, I'm sort of reminded of cases where I use something like f = aesara.function(some_inputs, some_outputs)

[brandonwillard]

FunctionGraphs primarily specify the relevant inputs and outputs of a graph, hold all the information about said graph, and provide the basic means to mutate a graph. Also, in a sense, a FunctionGraph specifies a topologically ordered sequence of Apply nodes (i.e. the FunctionGraph.toposort value).

"Local" rewrites are node-specific, which means that without the FunctionGraph object one wouldn't be able to reason about the context in which the given "local" Apply nodes exist (e.g. which nodes depend on said node, or any other information tracked by a Feature in a FunctionGraph). Ideally, one doesn't need any context for a local rewrite, but, in practice, that's not always the case.

Really, the "local" rewrite framing is just a way to specify a simple visitation/traversal pattern.

[larryshamalama]

Also, in a sense, a FunctionGraph specifies a topologically ordered sequence of Apply nodes (i.e. the FunctionGraph.toposort value).

Say we have some fgraph, _, _ = FunctionGraph(...). Is fgraph compiled or have gone through rewrites, either local or global?

[brandonwillard]

FunctionGraph is a Python type, so calling it will return an instance of FunctionGraph. If you take a look at the source for FunctionGraph.__init__, you'll see that a FunctionGraph is mostly just a container that holds a list of inputs and outputs.

One of the more non-trivial things performed in FunctionGraph.__init__—albeit somewhat indirectly—is the construction/collection of client information (i.e. a map between Variables and the Apply nodes that depend on them as inputs), available via FunctionGraph.clients. This happens when FunctionGraph.add_input and FunctionGraph.add_output are called.

[larryshamalama]

In the following example, where does ScalarFromTensor come from? Why is the index variable I_rv being wrapped in it?

srng = at.random.RandomStream(seed=2320)

I_rv = srng.bernoulli(0.5, name="I")
X_rv = srng.normal(-10., 0.1, name="X")
Y_rv = srng.normal(10., 0.1, name="Y")

Z_rv = at.stack([X_rv, Y_rv])[I_rv]
Z_rv.name = "Z"
z_vv = Z_rv.clone()

fgraph, _, _ = construct_ir_fgraph({Z_rv: z_vv, I_rv: i_vv})

fgraph.outputs[0].owner.inputs # [NoneConst, ScalarFromTensor.0, X, Y]

aesara.dprint(fgraph2.outputs[0].owner.inputs[1])
# ScalarFromTensor [id A]
#  |bernoulli_rv{0, (0,), int64, False}.1 [id B] 'I'
#    |RandomGeneratorSharedVariable(<Generator(PCG64) at 0x165706E40>) [id C]
#    |TensorConstant{[]} [id D]
#    |TensorConstant{4} [id E]
#    |TensorConstant{0.5} [id F]

I'm currently trying to work out a local_optimizer for switches as per your suggestion here. I'm at a point where:

aesara.graph.basic.equal_computations(fgraph2.outputs[0].owner.inputs[1], I_rv)

yields False.

[brandonwillard]

You should really dprint fgraph.outputs[0], because without the Op, it's hard to tell why that ScalarFromTensor might've been introduced.

Regardless, some Ops will require ScalarTypeed inputs, so their Op.make_node methods will add a ScalarFromTensor in order to get the correct type of inputs.

I'm assuming the Op in this case is a Subtensor, and, if you look at Subtensor.make_node, you'll see that it converts its inputs to ScalarTypeed Variables. The reason is that array-valued inputs have a very specific interpretation under NumPy's indexing semantics, and Subtensor needs to make sure that its arguments are basic indices (e.g. non-array scalar values).