Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
implement a lazy evaluation / thunk compiler primitive (#25387)
# The Problem
I would like to be able to write module code for "built-in" Chapel
features. That's because some code is much easier to express in plain
Chapel than in AST manipulation. For instance, I would like to be able
to use `on` clauses and `forall` loops to implement language features,
rather than having to build lowered forms of them in the AST. It's also
significantly easier to change the Chapel module implementation than
having to modify the AST-building compiler code. For instance, consider
the following code that I would like to use to implement remote variable
declarations:
```Chapel
var c: owned _remoteVarContainer(inType)?;
on loc do c = new _remoteVarContainer(expr);
return new _remoteVarWrapper(try! c : owned _remoteVarContainer(inType));
```
There are a lot of AST nodes here, and this is just one of the three
versions of this code (it handles the case where both the type and the
value of the remote variable are specified). Writing it as C++
`buildWhateverStatement` is error prone.
And yet, __I can't just turn that code into a Chapel function and call
it__:
```Chapel
proc myFn(loc: locale, type inType, value: inType) {
var c: owned _remoteVarContainer(inType)?;
on loc do c = new _remoteVarContainer(value);
return new _remoteVarWrapper(try! c : owned _remoteVarContainer(inType));
}
var myWrapper = myFn(loc, inType, expr); // not the same!
```
The reason is that arguments to functions are evaluated before they are
given to function bodies. In my example, `expr` will get evaluated
before the call to `myFn`, and thus the computation will not occur on
`loc`. This is currently causing a problem for remote variables (my
second snippet matches my actual implementation). Static variables
suffer from this problem because the compiler-generated C++ code is
brittle and long.
# The Solution
What I'd like to be able to do is to pass in an "expression" into Chapel
code, to be evaluated by the function when needed, be it on a different
locale or conditionally. To this end, what I need is to be able to defer
computations, and that's what this PR adds:
[thunks](https://en.wikipedia.org/wiki/Thunk). Then, I can write code
like this:
```Chapel
proc myFn(loc: locale, type inType, in thunk: _thunkRecord) {
var c: owned _remoteVarContainer(inType)?;
on loc do c = new _remoteVarContainer(__primitive("force thunk", thunk));
return new _remoteVarWrapper(try! c : owned _remoteVarContainer(inType));
}
var myWrapper = myFn(loc, inType, __primitive("create thunk", expr)); // same as the "inlined" form!
```
Another example (from a test file) is the following:
```Chapel
proc executeIfTrue(cond: bool, in thunk: _thunkRecord) {
var temp: thunkToReturnType(thunk.type);
if cond {
temp = __primitive("force thunk", thunk);
}
return temp;
}
writeln(executeIfTrue(true, __primitive("create thunk", new C?(42)))); // prints "calling C.init", then "{42}"
writeln(executeIfTrue(false, __primitive("create thunk", new C?(42)))); // doesn't print, then prints "nil"
```
# Implementation
This works by doing pretty much what we do for iterators and their
records/classes: for each thunked expression (the `create thunk`
primitive), it creates a new builder function `chpl__thunkN` and,
eventually, a record `_tr_chpl__thunkN`. It re-uses the same logic for
capturing outer variables etc, so that the expression being captured can
refer to global or local variables, fields, etc. In detail, the process
is as follows:
1. __Pass 8 - Normalize__: lifts `create thunk` primitives into builder
functions, named `chpl__thunkN`. These functions accept all the captured
variables as formals. Eventually, these functions are marked to return
the thunk record; however, during normalize, the thunk record doesn't
yet exist. The thunk body -- the expression being deferred -- is copied
into these functions.
2. __Pass 12 - Resolve__: after resolving the types of the
`chpl__thunkN` function, the function resolution code creates the thunk
record (but doesn't yet populate its fields; this matches how iterator
records are handled). It also creates the `invoke` method, which is
called by `force thunk` primitives. The 'invoke' method is empty for the
time being (again, like the prototype methods for iterators) and is
populated once the thunk record's fields have been created. The `force
thunk` primitive is simply rewritten to the `invoke` method.
3. __Pass 20 - Lower Iterators__: at this time, the thunk record is
populated with the fields that correspond to the captured variables, and
the body of the `invoke` method is filled in with the original code that
the user provided; references to outer variables are replaced with field
references to the thunk record. The builder function is made to create a
new thunk record and populate its fields with the formals, preparing it
for invocation.
Much of the logic is shared with lowering iterators, and I have factored
out code where appropriate to ensure that little-to-no code duplication
is involved.
# Next steps
Since this PR is relatively large, I didn't want to also include into it
changes to remote or static varibles. However, I believe that the next
step is to switch the remote variable implementation to use thunks,
which will allow it to consistently execute the initialization
expression on the target locale
(#25298).
Reviewed by @e-kayrakli -- thanks!
# Testing
- [x] paratest
- [x] paratest gasnet
- [x] GPU test- Loading branch information
