BUG: cannot import a model in a function #114

albop · 2017-09-08T13:18:36Z

This is a bit frightening. Are we doing something wrong with the function generation ?

julia> import Dolo

julia> function solve_julia(fname)
            model = Dolo.Model(fname)
            dr = Dolo.time_iteration(model)
       end
solve_julia (generic function with 1 method)

julia> sol = solve_julia("neoclassical.yaml")

ERROR: MethodError: no method matching controls_lb(::Dolo.Model{Symbol("##836")}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1})
The applicable method may be too new: running in world age 22573, while current world is 22637.
Closest candidates are:
  controls_lb(::Dolo.Model{Symbol("##836")}, ::AbstractArray{T,1} where T, ::AbstractArray{T,1} where T, ::Any) at :0 (method too new to be called from this world context.)
  controls_lb(::Dolo.Model{Symbol("##836")}, ::AbstractArray, ::AbstractArray, ::Any) at :0 (method too new to be called from this world context.)
  controls_lb(::Type{Dolang.Der{0}}, ::Dolo.Model{Symbol("##836")}, ::AbstractArray{T,1} where T, ::AbstractArray{T,1} where T, ::Any) at :0 (method too new to be called from this world context.)
  ...
Stacktrace:
 [1] #time_iteration#141(::Bool, ::Int64, ::Float64, ::Bool, ::Dict{Any,Any}, ::Function, ::Dolo.Model{Symbol("##836")}, ::Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}, ::Dolo.CartesianGrid{1}, ::Dolo.ConstantDecisionRule{1}) at /home/pablo/.julia/v0.6/Dolo/src/algos/time_iteration.jl:199
 [2] time_iteration(::Dolo.Model{Symbol("##836")}, ::Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}, ::Dolo.CartesianGrid{1}, ::Dolo.ConstantDecisionRule{1}) at /home/pablo/.julia/v0.6/Dolo/src/algos/time_iteration.jl:167
 [3] #time_iteration#145(::Dict{Any,Any}, ::Array{Any,1}, ::Function, ::Dolo.Model{Symbol("##836")}, ::Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}, ::Dolo.ConstantDecisionRule{1}) at /home/pablo/.julia/v0.6/Dolo/src/algos/time_iteration.jl:257
 [4] (::Dolo.#kw##time_iteration)(::Array{Any,1}, ::Dolo.#time_iteration, ::Dolo.Model{Symbol("##836")}, ::Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}, ::Dolo.ConstantDecisionRule{1}) at ./<missing>:0
 [5] #time_iteration#148(::Dict{Any,Any}, ::Array{Any,1}, ::Function, ::Dolo.Model{Symbol("##836")}) at /home/pablo/.julia/v0.6/Dolo/src/algos/time_iteration.jl:277
 [6] solve_julia(::String) at ./REPL[3]:3

The text was updated successfully, but these errors were encountered:

sglyon · 2017-09-08T13:31:33Z

Oh man this could be tricky to figure out.

I honestly don't know too much about how the new world/old world stuff works. Let me ask around and see if we can figure out a fix

albop · 2017-09-08T13:35:12Z

could be related to the scope where the functions are evaluated

sglyon · 2017-09-08T13:51:19Z

One idea would be to have Dolo.@Model "neoclassical.yaml" just emit the generated code as opposed to what we do now where Dolo.Model("neoclassical.yaml") builds the code and then calls evalon it.

That would almost certainly work , but might make it so the methods are only callable within the same scope as when the macro is called. Not sure if that is an issue or not...

albop · 2017-09-08T13:57:21Z

it is of course an issue...

quinnj · 2017-09-08T14:04:36Z

That's the approach I took w/ defining custom json formatting for types here. I'm not sure what you mean "methods are only callable within the same scope"? In my case, my macro expands to a generated function that overloads a JSON2 function for the new type, so there aren't any issues calling it from any scope.

sglyon · 2017-09-08T14:10:27Z

Ok cool, thanks for the tip/reference. We'll take a look at that and see if we can use something similar.

Here's an example of what I meant and was worried about:

Suppose that Dolo.@Model "fname.yaml" did something like this:

begin
    Dolo.controls_lb(::Model{:fname}, ...) = nothing
    Model{:fname}(constructor_args...)
end

If the macro were just to place this code at the call site @albop function above would look like this after macro expansion:

function solve_julia(fname)
    model = begin
        Dolo.controls_lb(::Model{:fname}, ...) = nothing
        Model{:fname}(constructor_args...)
    end
    dr = Dolo.time_iteration(model)
end

I was worried that the particular method of Dolo.controls_lb defined by that macro would only be callable from within the solve_julia function and not visible elsewhere in Julia (e.g. within the Dolo.time_iteration function). I thought about it a bit more and don't think that this will actually be an issue...

quinnj · 2017-09-08T14:20:59Z

It seems like you should just move Dolo.@Model "fname.yml" to a top-level expression, not inside a function, or at least have a macro that defines the type/functions at the top-level, then have a separate construct/macro/whatever that you call in the function that actually defined the instance.

albop · 2017-09-08T14:26:35Z

I don't totally understand how the macro thing solves the initial problem. @sglyon and @quinnj : aren't you assuming the model filename is constant, so that the macro can read the file, expand the code and replace it in the function ? So you can't pass the filename to the model_julia function, can you ?
I confess, I don't understand the error message very well...

sglyon · 2017-09-08T14:34:51Z

Well, I found a solution to this problem, but it isn't necessarily pretty:

julia> function solve_julia(fname::String)
           model = Dolo.Model(fname)
           Base.invokelatest(Dolo.time_iteration, model)
       end
solve_julia (generic function with 1 method)

julia> solve_julia("neoclassical.yaml")
------------------------------------------------------------------
It    ηₙ=|xₙ-xₙ₋₁|    λₙ=ηₙ/ηₙ₋₁      Time            Newton steps
------------------------------------------------------------------
0     NaN             NaN             0.00e+00        0
1     2.03e-02        NaN             6.44e+00        3
2     1.01e-02        4.96e-01        2.34e-02        2
3     7.15e-03        7.11e-01        1.75e-02        2
4     5.21e-03        7.29e-01        8.76e-02        2
5     3.87e-03        7.43e-01        3.25e-02        2
6     2.92e-03        7.53e-01        2.62e-02        2
7     2.22e-03        7.62e-01        2.10e-02        2
8     1.71e-03        7.69e-01        1.82e-02        2
9     1.38e-03        8.05e-01        3.08e-02        2
10    1.16e-03        8.44e-01        1.77e-02        2
11    9.80e-04        8.44e-01        6.17e-02        2
12    8.27e-04        8.43e-01        2.30e-02        2
13    6.97e-04        8.43e-01        2.98e-02        2
14    5.87e-04        8.42e-01        2.06e-02        2
15    4.94e-04        8.42e-01        1.82e-02        2
16    4.15e-04        8.41e-01        1.18e-02        1
17    3.49e-04        8.41e-01        1.64e-02        1
18    2.93e-04        8.40e-01        1.02e-02        1
19    2.46e-04        8.39e-01        1.09e-02        1
20    2.06e-04        8.39e-01        1.57e-02        1
21    1.73e-04        8.38e-01        4.84e-02        1
22    1.45e-04        8.37e-01        1.18e-02        1
23    1.21e-04        8.37e-01        1.27e-02        1
24    1.01e-04        8.36e-01        1.31e-02        1
25    8.46e-05        8.35e-01        1.52e-02        1
26    7.06e-05        8.35e-01        1.52e-02        1
27    5.89e-05        8.34e-01        1.24e-02        1
28    4.91e-05        8.34e-01        1.45e-02        1
29    4.09e-05        8.33e-01        1.48e-02        1
30    3.41e-05        8.33e-01        1.63e-02        1
31    2.84e-05        8.32e-01        1.19e-02        1
32    2.36e-05        8.32e-01        1.26e-02        1
33    1.96e-05        8.31e-01        3.98e-02        1
34    1.63e-05        8.31e-01        1.11e-02        1
35    1.36e-05        8.31e-01        1.76e-02        1
36    1.13e-05        8.30e-01        7.41e-03        1
37    9.34e-06        8.30e-01        5.79e-03        1
38    7.75e-06        8.30e-01        8.91e-03        1
39    6.43e-06        8.29e-01        7.76e-03        1
40    5.33e-06        8.29e-01        5.85e-03        1
41    4.42e-06        8.29e-01        4.34e-03        1
42    3.66e-06        8.29e-01        6.00e-03        1
43    3.03e-06        8.28e-01        1.09e-02        1
44    2.51e-06        8.28e-01        1.01e-02        1
45    2.08e-06        8.28e-01        1.67e-02        1
46    1.72e-06        8.28e-01        6.07e-03        1
47    1.42e-06        8.27e-01        9.77e-03        1
48    1.18e-06        8.27e-01        4.97e-03        1
49    9.72e-07        8.27e-01        1.16e-02        1
50    8.04e-07        8.27e-01        6.40e-03        1
51    6.64e-07        8.26e-01        5.38e-03        1
52    5.49e-07        8.26e-01        6.05e-03        1
53    4.53e-07        8.26e-01        1.09e-02        1
54    3.74e-07        8.26e-01        6.81e-03        1
55    3.09e-07        8.25e-01        8.97e-03        1
56    0.00e+00        0.00e+00        1.97e-03        0
------------------------------------------------------------------
Results of Time Iteration Algorithm
 * Complementarities: true
 * Discretized Process type: Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}
 * Decision Rule type: Dolo.CubicDR{Dolo.CartesianGrid{1},Dolo.CartesianGrid{1},1,2}
 * Number of iterations: 56
 * Convergence: true
   * |x - x'| < 1.0e-07: true

Notice that I use Base.invokelatest(Dolo.time_iteration, model) when calling time iteration. I don't understand what is going on well enough to know why that works or how to get around it, but at least that does give us a starting place...

albop · 2017-09-08T14:47:49Z

As long as the manual explains transparently to the user why he needs to do that... ;-)

…

On Fri, Sep 8, 2017 at 3:34 PM Spencer Lyon ***@***.***> wrote: Well, I found *a* solution to this problem, but it isn't necessarily pretty: julia> function solve_julia(fname::String) model = Dolo.Model(fname) Base.invokelatest(Dolo.time_iteration, model) end solve_julia (generic function with 1 method) julia> solve_julia("neoclassical.yaml") ------------------------------------------------------------------ It ηₙ=|xₙ-xₙ₋₁| λₙ=ηₙ/ηₙ₋₁ Time Newton steps ------------------------------------------------------------------ 0 NaN NaN 0.00e+00 0 1 2.03e-02 NaN 6.44e+00 3 2 1.01e-02 4.96e-01 2.34e-02 2 3 7.15e-03 7.11e-01 1.75e-02 2 4 5.21e-03 7.29e-01 8.76e-02 2 5 3.87e-03 7.43e-01 3.25e-02 2 6 2.92e-03 7.53e-01 2.62e-02 2 7 2.22e-03 7.62e-01 2.10e-02 2 8 1.71e-03 7.69e-01 1.82e-02 2 9 1.38e-03 8.05e-01 3.08e-02 2 10 1.16e-03 8.44e-01 1.77e-02 2 11 9.80e-04 8.44e-01 6.17e-02 2 12 8.27e-04 8.43e-01 2.30e-02 2 13 6.97e-04 8.43e-01 2.98e-02 2 14 5.87e-04 8.42e-01 2.06e-02 2 15 4.94e-04 8.42e-01 1.82e-02 2 16 4.15e-04 8.41e-01 1.18e-02 1 17 3.49e-04 8.41e-01 1.64e-02 1 18 2.93e-04 8.40e-01 1.02e-02 1 19 2.46e-04 8.39e-01 1.09e-02 1 20 2.06e-04 8.39e-01 1.57e-02 1 21 1.73e-04 8.38e-01 4.84e-02 1 22 1.45e-04 8.37e-01 1.18e-02 1 23 1.21e-04 8.37e-01 1.27e-02 1 24 1.01e-04 8.36e-01 1.31e-02 1 25 8.46e-05 8.35e-01 1.52e-02 1 26 7.06e-05 8.35e-01 1.52e-02 1 27 5.89e-05 8.34e-01 1.24e-02 1 28 4.91e-05 8.34e-01 1.45e-02 1 29 4.09e-05 8.33e-01 1.48e-02 1 30 3.41e-05 8.33e-01 1.63e-02 1 31 2.84e-05 8.32e-01 1.19e-02 1 32 2.36e-05 8.32e-01 1.26e-02 1 33 1.96e-05 8.31e-01 3.98e-02 1 34 1.63e-05 8.31e-01 1.11e-02 1 35 1.36e-05 8.31e-01 1.76e-02 1 36 1.13e-05 8.30e-01 7.41e-03 1 37 9.34e-06 8.30e-01 5.79e-03 1 38 7.75e-06 8.30e-01 8.91e-03 1 39 6.43e-06 8.29e-01 7.76e-03 1 40 5.33e-06 8.29e-01 5.85e-03 1 41 4.42e-06 8.29e-01 4.34e-03 1 42 3.66e-06 8.29e-01 6.00e-03 1 43 3.03e-06 8.28e-01 1.09e-02 1 44 2.51e-06 8.28e-01 1.01e-02 1 45 2.08e-06 8.28e-01 1.67e-02 1 46 1.72e-06 8.28e-01 6.07e-03 1 47 1.42e-06 8.27e-01 9.77e-03 1 48 1.18e-06 8.27e-01 4.97e-03 1 49 9.72e-07 8.27e-01 1.16e-02 1 50 8.04e-07 8.27e-01 6.40e-03 1 51 6.64e-07 8.26e-01 5.38e-03 1 52 5.49e-07 8.26e-01 6.05e-03 1 53 4.53e-07 8.26e-01 1.09e-02 1 54 3.74e-07 8.26e-01 6.81e-03 1 55 3.09e-07 8.25e-01 8.97e-03 1 56 0.00e+00 0.00e+00 1.97e-03 0 ------------------------------------------------------------------ Results of Time Iteration Algorithm * Complementarities: true * Discretized Process type: Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}} * Decision Rule type: Dolo.CubicDR{Dolo.CartesianGrid{1},Dolo.CartesianGrid{1},1,2} * Number of iterations: 56 * Convergence: true * |x - x'| < 1.0e-07: true Notice that I use Base.invokelatest(Dolo.time_iteration, model) when calling time iteration. I don't understand what is going on well enough to know why that works or how to get around it, but at least that does give us a starting place... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#114 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAQ5KWdZ5k5TDff0qU4mMq2Dyb1cinZAks5sgVCLgaJpZM4PRK1h> .

sglyon · 2017-09-08T14:49:18Z

I would like to brainstorm a bit more and figure out a different way to do this.

invokelatest isn't a magic solution and has drawbacks as highlighted in the docstring:

help?> Base.invokelatest
  invokelatest(f, args...)

  Calls f(args...), but guarantees that the most recent method of f will be executed.
  This is useful in specialized circumstances, e.g. long-running event loops or
  callback functions that may call obsolete versions of a function f. (The drawback
  is that invokelatest is somewhat slower than calling f directly, and the type of
  the result cannot be inferred by the compiler.)

albop · 2017-09-08T14:55:48Z

Here is a reduced form version of our problem: <edit: I removed the generated macro>

struct Model{T}
end

code = """
function squareit(t::Model{T}, x) where T
    x^2
end
"""

function applyfun(model::Model)
    squareit(model, 0.1)
end

function reduced_problem(fname, cc)
    model = Model{fname}()
    eval(parse(cc))
    applyfun(model)
end

reduced_problem(15, code)

It produces the error:

MethodError: no method matching squareit(::Model{15}, ::Float64)
The applicable method may be too new: running in world age 23129, while current world is 23130.
Closest candidates are:
  squareit(::Model{T}, ::Any) where T at none:2 (method too new to be called from this world context.)
reduced_problem(::Int64, ::String) at bug.jl:28
include_string(::String, ::String) at loading.jl:515
include_string(::String, ::String, ::Int64) at eval.jl:30
include_string(::Module, ::String, ::String, ::Int64, ::Vararg{Int64,N} where N) at eval.jl:34
(::Atom.##49#52{String,Int64,String})() at eval.jl:50
withpath(::Atom.##49#52{String,Int64,String}, ::String) at utils.jl:30
withpath(::Function, ::String) at eval.jl:38
macro expansion at eval.jl:49 [inlined]
(::Atom.##48#51{Dict{String,Any}})() at task.jl:80

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: cannot import a model in a function #114

BUG: cannot import a model in a function #114

albop commented Sep 8, 2017 •

edited

Loading

sglyon commented Sep 8, 2017

albop commented Sep 8, 2017

sglyon commented Sep 8, 2017

albop commented Sep 8, 2017

quinnj commented Sep 8, 2017

sglyon commented Sep 8, 2017 •

edited

Loading

quinnj commented Sep 8, 2017

albop commented Sep 8, 2017

sglyon commented Sep 8, 2017

albop commented Sep 8, 2017 via email

sglyon commented Sep 8, 2017 •

edited

Loading

albop commented Sep 8, 2017 •

edited

Loading

BUG: cannot import a model in a function #114

BUG: cannot import a model in a function #114

Comments

albop commented Sep 8, 2017 • edited Loading

sglyon commented Sep 8, 2017

albop commented Sep 8, 2017

sglyon commented Sep 8, 2017

albop commented Sep 8, 2017

quinnj commented Sep 8, 2017

sglyon commented Sep 8, 2017 • edited Loading

quinnj commented Sep 8, 2017

albop commented Sep 8, 2017

sglyon commented Sep 8, 2017

albop commented Sep 8, 2017 via email

sglyon commented Sep 8, 2017 • edited Loading

albop commented Sep 8, 2017 • edited Loading

albop commented Sep 8, 2017 •

edited

Loading

sglyon commented Sep 8, 2017 •

edited

Loading

sglyon commented Sep 8, 2017 •

edited

Loading

albop commented Sep 8, 2017 •

edited

Loading