Make ifelse use Switch when its arguments are ScalarTypeed

[brandonwillard]

Since Switch is quite literally the scalar version of IfElse—at least in terms of its Op.c_code implementation, we should consider making invocations of the ifelse helper function return Switches when its arguments are ScalarTypeed.

This should increase the chances that the resulting Op can/will be fused with other scalar Ops to produce a single, more efficient Composite Op, as well as remove some minor overhead due to TensorFromScalar and ScalarFromTensor.

The same general idea can be applied through rewrites, although we'll probably get a lot of coverage using only changes like this at the interface-level. In cases where TensorFromScalar arguments are passed to ifelse, an undesirable IfElse Op would be used, and this is where a rewrite could be helpful.

Originally posted by @brandonwillard in #1217 (comment)

[jessegrabowski]

I'm taking a look at this. How should the case of list-like inupts to then/else conditions of ifelse be handled? aesara.scalar.switch doesn't seem to allow lists in the same way ifelse does.

[brandonwillard]

I'm taking a look at this. How should the case of list-like inupts to then/else conditions of ifelse be handled? aesara.scalar.switch doesn't seem to allow lists in the same way ifelse does.

Good question. We can always nest Switches, but that doesn't sound great, especially not for the readability of our graphs.

N.B. This reminds me of a couple scalars-related issues: there are almost no scalar rewrites that simplify these basic operations, and Composites can contain other Composites. Both of these can lead to unnecessarily large scalar sub-graphs (e.g. see the scalar sub-graphs constructed by some of the Scan tests). See #1235.

[jessegrabowski]

A list of Switchs is the natural answer, but I agree it's not very beautiful. An alternative could be a full-blown scalar version of ifelse added to the scalar module? If there is enough of a distinction to merit two separate functions Switch and IfElse in the tensor case, does it then follow that there's enough distinction for the scalar case as well?

[brandonwillard]

A list of Switchs is the natural answer, but I agree it's not very beautiful. An alternative could be a full-blown scalar version of ifelse added to the scalar module? If there is enough of a distinction to merit two separate functions Switch and IfElse in the tensor case, does it then follow that there's enough distinction for the scalar case as well?

We should be fine with nested Switches for now, especially since we could rewrite some of them as single Composites (e.g. via #1225).