Generated functions crash in processes

[ten0s]

I was testing a parallel map implementation and immediately found that generated functions crash
when called in processes different from where they were generated. I understand PropEr sets up
some state and even calls it `global' as proper_gen:pick/1 message says. The problem is the state is obviously NOT global as the functions don't work.

1> {ok, F} = proper_gen:pick(proper_types:function1(proper_types:integer())).
WARNING: Some garbage has been left in the process registry and the code server
to allow for the returned function(s) to run normally.
Please run proper:global_state_erase() when done.
{ok,#Fun<proper_gen.19.84225435>}
2> F(0).                                                                     
15
3> Parent = self().                                               
<0.42.0>
4> spawn(fun () -> Parent ! F(0) end). 
<0.49.0>
5> 
=ERROR REPORT==== 27-Jan-2017::15:08:35 ===
Error in process <0.49.0> with exit value:
{function_clause,[{proper_arith,rand_non_neg_float,
                                [undefined],
                                [{file,"src/proper_arith.erl"},{line,305}]},
                  {proper_arith,rand_float,1,
                                [{file,"src/proper_arith.erl"},{line,298}]},
                  {proper_arith,rand_int,1,
                                [{file,"src/proper_arith.erl"},{line,256}]},
                  {proper_gen,generate,3,
                              [{file,"src/proper_gen.erl"},{line,184}]},
                  {proper_gen,generate,1,
                              [{file,"src/proper_gen.erl"},{line,124}]},
                  {proper_gen,function_body,3,
                              [{file,"src/proper_gen.erl"},{line,627}]},
                  {erl_eval,do_apply,5,[{file,"erl_eval.erl"},{line,661}]},
                  {erl_eval,expr,5,[{file,"erl_eval.erl"},{line,470}]}]}

My quick fix was a function like:

-module(pfunction).

-export([wrap/1]).

wrap(Fun) ->
    Size = get('$size'),
    Seed = get('random_seed'),
    CheckSeed = fun () ->
        case get('random_seed') of
        undefined ->
            proper:global_state_init_size_seed(Size, Seed);
        _ ->
            ok
        end
    end,
    {arity, Arity} = erlang:fun_info(Fun, arity),
    case Arity of
    0 -> fun ()        -> CheckSeed(), Fun() end;
    1 -> fun (A)       -> CheckSeed(), Fun(A) end;
    2 -> fun (A,B)     -> CheckSeed(), Fun(A,B) end;
    3 -> fun (A,B,C)   -> CheckSeed(), Fun(A,B,C) end;
    4 -> fun (A,B,C,D) -> CheckSeed(), Fun(A,B,C,D) end
    end.

Wrapping and calling again does the trick.

5> c(pfunction).                                                             
{ok,pfunction}
6> PF = pfunction:wrap(F).                                                   
#Fun<pfunction.2.102553575>
7> f(Parent), Parent = self().         
<0.42.0>
8> spawn(fun () -> Parent ! PF(0) end).
<0.58.0>
9> flush().
Shell got 15

[kostis]

I am trying to understand what is the issue that bothers you here. Do you simply want us to rename global which is actually only used in the name of the function (global_state_erase) to something else?

The warning you got warned you that "Some garbage has been left in the process registry" which means that you cannot expect that this works in some other process with a different process registry in general.

Anyway, is there some concrete action that you want us to do?

[ten0s]

What bothers me is that for the function

pmap(Fun, Xs) when is_function(Fun, 1), is_list(Xs) ->
    Parent = self(),
    collect([
        spawn_monitor(fun () -> Parent ! {self(), Fun(X)} end) || X <- Xs
    ]);
pmap(_, _) ->
    exit(badarg).

collect([]) -> [];
collect([{Pid, MRef} | Rest]) ->
  receive
    {Pid, Res} ->
      erlang:demonitor(MRef, [flush]),
      [Res | collect(Rest)];
    {'DOWN', MRef, process, Pid, Reason} ->
      [{error, Reason} | collect(Rest)]
  end.

the property

prop_pmap_equals_map() ->
    ?FORALL({Fun, List}, {function1(term()), list(term())},
    begin
        lists:map(Fun, List) =:= pmap(Fun, List)
    end).

fails with

proper:quickcheck(qc:prop_pmap_equals_map()).
.!
Failed: After 2 test(s).
{#Fun<proper_gen.17.132935905>,[{}]}

Shrinking .(1 time(s))
{#Fun<proper_gen.17.132935905>,[0]}
false
4> 
=ERROR REPORT==== 28-Jan-2017::20:37:03 ===
Error in process <0.64.0> with exit value:
{badarith,[{proper_gen,real_any_gen,1,
                       [{file,"src/proper_gen.erl"},{line,581}]},
           {proper_gen,generate,3,[{file,"src/proper_gen.erl"},{line,184}]},
           {proper_gen,generate,1,[{file,"src/proper_gen.erl"},{line,124}]},
           {proper_gen,function_body,3,
                       [{file,"src/proper_gen.erl"},{line,627}]},
           {qc,'-pmap/2-fun-0-',3,[{file,"qc.erl"},{line,10}]}]}

=ERROR REPORT==== 28-Jan-2017::20:37:03 ===
Error in process <0.65.0> with exit value:
{badarith,[{proper_gen,real_any_gen,1,
                       [{file,"src/proper_gen.erl"},{line,581}]},
           {proper_gen,generate,3,[{file,"src/proper_gen.erl"},{line,184}]},
           {proper_gen,generate,1,[{file,"src/proper_gen.erl"},{line,124}]},
           {proper_gen,function_body,3,
                       [{file,"src/proper_gen.erl"},{line,627}]},
           {qc,'-pmap/2-fun-0-',3,[{file,"qc.erl"},{line,10}]}]}

=ERROR REPORT==== 28-Jan-2017::20:37:03 ===
Error in process <0.66.0> with exit value:
{badarith,[{proper_gen,real_any_gen,1,
                       [{file,"src/proper_gen.erl"},{line,581}]},
           {proper_gen,generate,3,[{file,"src/proper_gen.erl"},{line,184}]},
           {proper_gen,generate,1,[{file,"src/proper_gen.erl"},{line,124}]},
           {proper_gen,function_body,3,
                       [{file,"src/proper_gen.erl"},{line,627}]},
           {qc,'-pmap/2-fun-0-',3,[{file,"qc.erl"},{line,10}]}]}

=ERROR REPORT==== 28-Jan-2017::20:37:03 ===
Error in process <0.67.0> with exit value:
{badarith,[{proper_gen,real_any_gen,1,
                       [{file,"src/proper_gen.erl"},{line,581}]},
           {proper_gen,generate,3,[{file,"src/proper_gen.erl"},{line,184}]},
           {proper_gen,generate,1,[{file,"src/proper_gen.erl"},{line,124}]},
           {proper_gen,function_body,3,
                       [{file,"src/proper_gen.erl"},{line,627}]},
           {qc,'-pmap/2-fun-0-',3,[{file,"qc.erl"},{line,10}]}]}

=ERROR REPORT==== 28-Jan-2017::20:37:03 ===
Error in process <0.68.0> with exit value:
{badarith,[{proper_gen,real_any_gen,1,
                       [{file,"src/proper_gen.erl"},{line,581}]},
           {proper_gen,generate,3,[{file,"src/proper_gen.erl"},{line,184}]},
           {proper_gen,generate,1,[{file,"src/proper_gen.erl"},{line,124}]},
           {proper_gen,function_body,3,
                       [{file,"src/proper_gen.erl"},{line,627}]},
           {qc,'-pmap/2-fun-0-',3,[{file,"qc.erl"},{line,10}]}]}

but if you wrap the generated function with

wrap(Fun) ->
    Size = get('$size'),
    Seed = get('random_seed'),
    CheckSeed = fun () ->
        case get('random_seed') of
        undefined ->
            proper:global_state_init_size_seed(Size, Seed);
        _ ->
            ok
        end
    end,
    {arity, Arity} = erlang:fun_info(Fun, arity),
    case Arity of
    0 -> fun ()        -> CheckSeed(), Fun() end;
    1 -> fun (A)       -> CheckSeed(), Fun(A) end;
    2 -> fun (A,B)     -> CheckSeed(), Fun(A,B) end;
    3 -> fun (A,B,C)   -> CheckSeed(), Fun(A,B,C) end;
    4 -> fun (A,B,C,D) -> CheckSeed(), Fun(A,B,C,D) end
    end.

then the property

prop_pmap_equals_map_wrapped() ->
    ?FORALL({Fun, List}, {function1(term()), list(term())},
    begin
        lists:map(Fun, List) =:= pmap(wrap(Fun), List)
    end).

works

proper:quickcheck(qc:prop_pmap_equals_map_wrapped()).  
....................................................................................................
OK: Passed 100 test(s).
true

I would expect the generated functions work in other processes without any additional tricks.