Spec updates: POI, reduce intents, ... (#16567) by vasslitvinov

Update the spec and RI technote.

Update the Point of Instantiation (POI) rule for function visibility
to reflect the semantic changes in #15948 and #15872.

Improve the presentation of function resolution especially w.r.t.
function visibility.

Specify reduce intents within the specification of task and forall intents.

Update the reduceIntents technote. Defining reduce intents in the spec
made some former parts of this technote unnecessary.

While there, remove occurrences of the phrase "online documentation"
as all documentation is now online.

Signed-off-by: Vassily Litvinov <vasslitvinov@users.noreply.github.com>
r: @mppf @dlongnecke-cray

doc/rst/language/spec/data-parallelism.rst

@@ -18,6 +18,7 @@ This chapter details data parallelism as follows:
 
 -  :ref:`Forall_Intents` specifies how variables from outer
    scopes are handled within forall statements and expressions.
+   :ref:`Task_Private_Variables` provide a related functionality.
 
 -  :ref:`Promotion` describes promotion.
 
@@ -95,8 +96,8 @@ tasks.
 
 This concept will be formalized in future drafts of the Chapel
 specification. For now, the
-:ref:`primer on parallel iterators <primers-parIters>` in the online
-documentation provides a brief introduction.
+:ref:`primer on parallel iterators <primers-parIters>`
+provides a brief introduction.
 Please also refer to *User-Defined Parallel Zippered Iterators in
 Chapel*, published in the PGAS 2011 workshop.
 
@@ -297,17 +298,19 @@ Forall Intents
 If a variable is referenced within the lexical scope of a forall
 statement or expression and is declared outside that forall construct,
 it is subject to *forall intents*, analogously to task intents
-(:ref:`Task_Intents`) for task-parallel constructs. That is, the
-variable is considered to be passed as an actual argument to each task
-function created by the object or iterator leading the execution of the
-loop. If no tasks are created, it is considered to be an actual argument
-to the leader or standalone iterator itself. All references to the
+for task-parallel constructs (see :ref:`Task_Intents`). That is, the
+outer variable is considered to be passed as an actual argument to an
+implicit formal of the iterator leading the execution of the loop.
+From there, it is passed down to each task created by that iterator,
+if any, as an actual argument to an implicit formal of the corresponding
+task function. A top-level task passes it down recursively to its
+child tasks, if any. All references to the 
 variable within the forall construct implicitly refer to a *shadow
 variable*, i.e. the corresponding formal argument of the task function
-or the leader/standalone iterator.
+or the leading iterator.
 
 When the forall construct is inside a method on a record and accesses a
-field of ``this``, the field is treated as a regular variable. That is,
+field of ``this``, the field is treated as an outer variable. That is,
 it is subject to forall intents and all references to this field within
 the forall construct implicitly refer to the corresponding shadow
 variable.
@@ -325,11 +328,17 @@ variables of most types, the ``ref`` intent allows the body of the
 forall loop to modify the corresponding original variable or to read
 its updated value after concurrent modifications. The ``in`` intent is
 an alternative way to obtain task-private variables
-(:ref:`Task_Private_Variables`). A ``reduce`` intent can be used
-to reduce values across iterations of a forall or coforall loop.
-Reduce intents are described in the
-:ref:`Reduce Intents technical note <readme-reduceIntents>` in
-the online documentation.
+(see :ref:`Task_Private_Variables`).
+
+A ``reduce`` forall intent can be used to reduce values across iterations
+of a forall loop. While it is similar to the ``reduce`` task intent
+(see :ref:`Task_Intents`), there is a difference in how values
+are combined at the end of a task. With a ``reduce`` forall intent,
+each child task combines its accumulated value into its parent task
+rather than into an outer variable.
+The ``reduce=`` operator accumulates its right-hand side values
+computed for all iterations executed by a given task into the same
+shadow variable for that task.
 
    *Rationale*.
 

doc/rst/language/spec/domain-maps.rst

@@ -32,12 +32,10 @@ Domain maps are presented as follows:
 -  standard layouts and distributions, such as Block and Cyclic, are
    documented under
    :ref:`Standard Layouts and Distributions <layouts_and_distributions>`
-   in Chapel's online documentation.
 
 -  specification of user-defined domain maps is forthcoming; please
    refer to the
    :ref:`Domain Map Standard Interface technical note <readme-dsi>`
-   in Chapel's online documentation.
 
 .. _Domain_Maps_For_Types:
 

doc/rst/language/spec/error-handling.rst

@@ -14,8 +14,8 @@ a less-permissive mode intended for production code.
 
    Additional information about the current implementation of
    error handling and the *strict* error handling mode, which is not
-   defined here, is available online in the errorHandling technical note
-   :ref:`readme-errorHandling`.
+   defined here, is available in the 
+   :ref:`errorHandling technical note <readme-errorHandling>`
 
 .. _Throwing_Errors:
 

doc/rst/language/spec/generics.rst

@@ -473,50 +473,86 @@ A queried domain may not be modified via the name to which it is bound
 Function Visibility in Generic Functions
 ----------------------------------------
 
-When resolving function calls made within generic functions, there is an
-additional source of visible functions. Besides functions visible to the
-generic function’s point of declaration, visible functions are also
-taken from one of the call sites at which the generic function is
-instantiated for each particular instantiation. The specific call site
-chosen is arbitrary and it is referred to as the *point of
-instantiation*.
+When resolving a function call, as defined in :ref:`Function_Resolution`,
+there is an additional source of visible functions when the call is
+nested within a generic function. The additional source is the functions
+visible from the call site that the enclosing generic function is invoked from.
+This call site is referred to as the *point of instantiation*.
+If there are multiple enclosing generic functions or the call is nested
+within a concrete function that is, in turn, nested in generic function(s),
+the point of instantiation is the call site of the innermost generic function.
+
+If no candidate functions are found during the initial steps of
+identifying visible and candidate functions, function resolution
+continues the search for visible and candidate functions
+at the point of instantiation. If still no candidates are found,
+the search continues to the point of instantiation of the innermost
+generic function that contains the previous point of instantiation.
+Once candidate(s) are found, the search succeeds and
+function resolution proceeds to selecting the most specific functions.
+Otherwise the search will reach a point of instantiation that is not
+within a generic function. For example, it can be at the module level or
+enclosed in only concrete function(s). If no candidates have been found,
+the compiler issues a "call cannot be resolved" error.
 
    *Example (point-of-instantiation.chpl)*.
 
-   Consider the following code which defines a generic function ``bar``:
+   Consider the following code:
 
 
    .. code-block:: chapel
 
-      module M1 {
-        record R {
-          var x: int;
-          proc foo() { }
+      module LibraryA {
+        proc callWorkers(arg) {
+          worker1();
+          worker2();
         }
       }
 
-      module M2 {
-        proc bar(x) {
-          x.foo();
+      module LibraryB {
+        use LibraryA;
+        proc worker1() { writeln("in LibraryB"); }
+        proc libFun(arg) {
+          callWorkers(arg);
         }
       }
 
-      module M3 {
-        use M1, M2;
+      module Application {
+        use LibraryB;
+        proc worker1() { writeln("in Application"); }
+        proc worker2() { writeln("in Application"); }
         proc main() {
-          var r: R;
-          bar(r);
+          libFun(1);
         }
       }
 
-   In the function ``main``, the variable ``r`` is declared to be of
-   type ``R`` defined in module ``M1`` and a call is made to the generic
-   function ``bar`` which is defined in module ``M2``. This is the only
-   place where ``bar`` is called in this program and so it becomes the
-   point of instantiation for ``bar`` when the argument ``x`` is of type
-   ``R``. Therefore, the call to the ``foo`` method in ``bar`` is
-   resolved by looking for visible functions from within ``main`` and
-   going through the use of module ``M1``.
+   .. BLOCK-test-chapeloutput
+
+      in LibraryB
+      in Application
+
+   When resolving the call to ``worker1`` in ``callWorkers()``
+   there are no visible functions at the scope of the call. Since
+   ``callWorkers()`` is a generic function, resolution looks at
+   its point of instantiation, which is its call within ``libFun()``.
+   There, a single candidate function for ``worker1`` is found, so
+   function resolution determines that this is the target function.
+
+   Since the search is complete, no further points of instantiation
+   are visited. Therefore ``LibraryB`` is assured that whenever
+   ``callWorkers()`` looks to its callers for ``worker1``,
+   the implementation in ``LibraryB`` will be used.
+   Other overloads, such ``worker1()`` in module ``Application``,
+   will not be considered.
+
+   When resolving the call to ``worker2`` in ``callWorkers()``,
+   resolution again looks at its point of instantiation, namely
+   its call within ``libFun()``. No visible functions can be found
+   there. Since ``libFun`` is also a generic function, the search
+   continues in turn to its point of instantiation, which is
+   its call in module ``Application``. Since a definition of ``worker2``
+   is visible there, it will be considered the candidate for the call
+   to ``worker2`` in ``callWorkers()``.
 
 If the generic function is only called indirectly through dynamic
 dispatch, the point of instantiation is defined as the point at which

doc/rst/language/spec/lexical-structure.rst

@@ -505,7 +505,8 @@ language:
 **symbols**                                                                                         **use**
 =================================================================================================== =============================
 ``=``                                                                                               assignment
-``+=`` ``-=`` ``*=`` ``/=`` ``**=`` ``%=`` ``&=`` ``|=`` ``^=`` ``&&=`` ``||=`` ``<<=`` ``>>=``     compound assignment
+``+=`` ``-=`` ``*=`` ``/=`` ``**=`` ``%=`` ``&=`` ``|=`` ``^=``                                     compound assignment
+``&&=`` ``||=`` ``<<=`` ``>>=`` ``reduce=``                                                         compound assignment, cont.
 ``<=>``                                                                                             swap
 ``<~>``                                                                                             I/O
 ``..`` ``..<``                                                                                      range specifier

doc/rst/language/spec/procedures.rst

@@ -1182,9 +1182,9 @@ Assignment overloads are not supported for class types.
 Function Resolution
 -------------------
 
-*Function resolution* is the algorithm that determines which function to
-invoke for a given call expression. Function resolution is defined as
-follows.
+*Function resolution* is the algorithm that determines which
+*target function* to invoke for a given call expression.
+Function resolution is defined as follows.
 
 -  Identify the set of visible functions for the function call. A
    *visible function* is any function that satisfies the criteria
@@ -1196,9 +1196,13 @@ follows.
    the set of candidate functions for the function call. A *candidate
    function* is any function that satisfies the criteria
    in :ref:`Determining_Candidate_Functions`. If no candidate
-   function can be found, the compiler will issue an error stating that
+   function can be found and the call is within a generic function,
+   its point of instantiation(s) are visited searching for candidates
+   as defined in :ref:`Function_Visibility_in_Generic_Functions`.
+   If still no candidate functions are found,
+   the compiler will issue an error stating that
    the call cannot be resolved. If exactly one candidate function is
-   found, this is determined to be the function.
+   found, this is determined to be the target function.
 
 -  From the set of candidate functions, determine the set of most
    specific functions. In most cases, there is one most specific
@@ -1213,9 +1217,9 @@ follows.
    best function for each return intent as described in
     :ref:`Determining_Best_Functions`. If there is more than
    one best function for a given return intent, the compiler will issue
-   an error stating that the call is ambiguous. Otherwise, it will
-   choose which function to call based on the calling context as
-   described in :ref:`Choosing_Return_Intent_Overload`.
+   an error stating that the call is ambiguous. Otherwise, it will choose
+   the target function from these best functions based on the calling
+   context as described in :ref:`Choosing_Return_Intent_Overload`.
 
 Notation
 ~~~~~~~~
@@ -1239,14 +1243,28 @@ This section uses the following notation:
 Determining Visible Functions
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Given a function call, a function is determined to be a *visible
-function* if the name of the function is the same as the name of the
-function call and the function is defined in the same scope as the
-function call or a lexical outer scope of the function call, or if the
-function is publicly declared in a module that is used from the same
-scope as the function call or a lexical outer scope of the function
-call. Function visibility in generic functions is discussed
-in :ref:`Function_Visibility_in_Generic_Functions`.
+Given a function call, a function :math:`X` is determined to be a
+*visible function* if its name is the same as the name of the
+function call and one of the following conditions is met:
+
+- :math:`X` is defined in the same scope as the
+  function call or in a lexical outer scope of the function call, or
+
+- :math:`X` is ``public`` and is declared in a module that is used from
+  the same scope as the function call or from its lexical outer scope,
+  see also :ref:`Using_Modules`, or
+
+- :math:`X` is ``public`` and is declared in a module that is imported from
+  the same scope as the function call or from its lexical outer scope,
+  and the call qualifies the function name with the module name,
+  see also :ref:`Importing_Modules`.
+
+   *Open issue*.
+
+   What should be the visibility of methods? Applying the above rules
+   excludes, for example, the methods defined in the same module as the
+   receiver type when that module is neither visible nor reachable
+   through module uses or imports from the scope of the function call.
 
 .. _Determining_Candidate_Functions:
 

doc/rst/language/spec/task-parallelism-and-synchronization.rst

@@ -29,6 +29,7 @@ details task parallelism as follows:
 -  :ref:`Task_Intents` specifies how variables from outer scopes
    are handled within ``begin``, ``cobegin`` and ``coforall``
    statements.
+   :ref:`Task_Private_Variables` are also available.
 
 -  :ref:`Sync_Statement` describes the sync statement, a
    structured way to control parallelism.
@@ -831,14 +832,15 @@ Task Intents
 
 If a variable is referenced within the lexical scope of a ``begin``,
 ``cobegin``, or ``coforall`` statement and is declared outside that
-statement, it is subject to *task intents*. That is, it is considered to
+statement, it is subject to *task intents*. That is, this *outer variable*
+is considered to
 be passed as an actual argument to the corresponding task function at
 task creation time. All references to the variable within the task
 function implicitly refer to a *shadow variable*, i.e. the task
 function’s corresponding formal argument.
 
 When the task construct is inside a method on a record and accesses a
-field of ``this``, the field is treated as a regular variable. That is,
+field of ``this``, the field itself is treated as an outer variable. That is,
 it is passed as an actual argument to the task function and all
 references to the field within the task function implicitly refer to the
 corresponding shadow variable.
@@ -870,16 +872,34 @@ The syntax of the task intent clause is:
 
    task-intent-item:
      formal-intent identifier
+     reduce-scan-operator 'reduce' identifier
+     class-type 'reduce' identifier
      task-private-var-decl
 
 where the following intents can be used as a ``formal-intent``:
 ``ref``, ``in``, ``const``, ``const in``, ``const ref``.
-``task-private-var-decl`` is defined in
-:ref:`Task_Private_Variables`. In addition,
-``task-intent-item`` may define a ``reduce`` intent. Reduce intents
-are described in the
-:ref:`Reduce Intents technical note <readme-reduceIntents>` in
-the online documentation.
+``task-private-var-decl`` is defined in :ref:`Task_Private_Variables`.
+
+The ``reduce`` task intent specifies a reduction into the outer variable,
+which is provided to the right of the ``reduce`` keyword.
+The reduction operator is specified by either the ``reduce-scan-operator``
+or the ``class-type`` in the same way as for a Reduction Expressions
+(see :ref:`reduce`). At the start of each task the corresponding shadow
+variable is initialized to the identity value of the reduction operator.
+Within the task it behaves as a regular variable. In addition, it can be
+the left-hand side of the ``reduce=`` operator, which accumulates its
+right-hand side onto the shadow variable.
+At the end of each task its shadow variable is combined into the outer
+variable.
+
+   *Open issue*.
+
+   How should ``reduce`` task intent be defined for ``begin`` tasks?
+   A reduction is legal only when the task completes before the program
+   has exited the dynamic scope of the outer variable.
+
+   Reduce intents are currently work-in-progress. See also
+   :ref:`Reduce Intents technical note <readme-reduceIntents>`.
 
 The implicit treatment of outer scope variables as the task function’s
 formal arguments applies to both module level and local variables. It