How can I redirect or silence stderr in Process(...)?

[kiryph]

The CARAT binary Name emits additional information to stderr next to its result which goes to stdout:

gap> List([1..10], el -> CaratName(SpaceGroupIT(3,el)));
father: level 1 number of orbits 1 lengths 3
father: level 1 number of orbits 1 lengths 3
father: level 1 number of orbits 1 lengths 3
father: level 1 number of orbits 1 lengths 3
father: level 1 number of orbits 1 lengths 3
father: level 1 number of orbits 1 lengths 3
father: level 1 number of orbits 1 lengths 3
father: level 1 number of orbits 1 lengths 3
[ "min.6-1.1-0", "group.5-1.1-0", "min.7-1.1-0", "min.7-1.1-0", "min.7-1.2-0", "min.8-1.1-0", "min.8-1.1-0",
  "min.8-1.2-0", "min.8-1.2-0", "group.6-1.1-0" ]

I would like to not show the lines father: level 1 number of orbits 1 lengths 3 and just the result. The lines do not indicate an error.

As said in the title the additional output goes to stderr:

https://github.com/lbfm-rwth/carat/blob/21bfdb516cc21c8e634a9fc99c469a07b8c7490c/functions/ZZ/ZZ_zclass.c#L350-L353

The GAP method CaratName is written by myself using the package CaratInterface as well:

InstallGlobalFunction( CaratCommandOStream, function( command, args, output )

    local program, err;

    # find executable
    program := Filename( CARAT_BIN_DIR, command );    
    if program = fail then
        Error( Concatenation( "Carat program ", command, " not found." ) );
    fi;

    # execute command
    err    := Process( DirectoryCurrent(), program, InputTextNone(), 
                       output, CaratStringToWordList( args ) );

end );


InstallMethod( CaratName, 
    "with Carat program Name",
    true, [ IsSpaceGroup ], 0,
function( grp )

    local grpfile, resfile, gen, res, out, input, dim, mat, output;

    dim := DimensionOfMatrixGroup( grp ) - 1;

    if dim <> 2 and dim <> 3 then
        Error( "sorry, only groups of dimension 2 and 3 are supported" );
    fi;

    if not IsRationalMatrixGroup( grp ) then
        Error( "grp must be a rational matrix group" );
    fi;

    # Special case for p1 and P1
    # binary Name segfaults
    # https://github.com/lbfm-rwth/carat/issues/112
    if IsTrivial(PointGroup(grp)) then
      if dim = 2 then
	return "min.2-1.1-0";
      elif dim = 3 then
	return "min.6-1.1-0";
      fi;
    fi;

    # Get temporary file name
    grpfile := CaratTmpFile( "grp" );  

    # Write Carat input to file; Carat binary "Name" does not read from stdin
    #  Group is expected to have as translation lattice Z^n (not centered unit cells) and
    #  act from the left in CARAT.
    if IsAffineCrystGroupOnRight( grp ) then
      gen := GeneratorsOfGroup( StandardAffineCrystGroup( TransposedMatrixGroup( grp ) ) );
    else
      gen := GeneratorsOfGroup( StandardAffineCrystGroup( grp ) );
    fi;
    CaratWriteMatrixFile( grpfile, gen );

    # execute Carat program
    res := "";
    output := OutputTextString(res, true);
    CaratCommandOStream( "Name", Concatenation( grpfile, " -c" ), output );;
    CloseStream(output);

    # remove temporary file
    RemoveFile( grpfile );

    return Chomp(res);

end );

The documentation of Process says, I cannot redirect, e.g. stderr to /dev/null with 2> /dev/null as additional option:

options is a list of strings which are passed to the process as command line argument. Note that no substitutions are performed on the strings, i.e., they are passed immediately to the process and are not processed by a command interpreter (shell). Further note that each string is passed as one argument, even if it contains space characters. Note that input/output redirection commands are not allowed as options.

Furthermore, using Exec instead of Process appears not to be an option, since the documentation says:

Exec calls the more general operation Process (11.1-1).

Therefore, I assume shell redirection is also not possible with Exec.

My GAP version:

 ┌───────┐   GAP 4.12.2 of 2022-12-18
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: x86_64-apple-darwin21-default64-kv8
 Configuration:  gmp 6.2.1, GASMAN, readline

So what can I do?

[kiryph]

However, the cmd argument to Exec is processed by the shell, i.e. use Exec(cmd) where cmd contains already all flags and redirections.

But catching stdout directly without disc IO does not seem to be possible:

InstallGlobalFunction( CaratExec, function( cmd, args, stdout_file, stderr_file )

    local program;

    # find executable
    program := Filename( CARAT_BIN_DIR, cmd );    
    if program = fail then
        Error( Concatenation( "Carat program ", cmd, " not found." ) );
    fi;

    # execute command
    Exec( Concatenation(program, " ", args, " > ", stdout_file , " 2> ", stderr_file ) );

end );


InstallMethod( CaratName, 
    "with Carat program Name",
    true, [ IsSpaceGroup ], 0,
function( grp )

    local grpfile, resfile, gen, res, out, input, dim, mat, output;

    dim := DimensionOfMatrixGroup( grp ) - 1;

    if dim <> 2 and dim <> 3 then
        Error( "sorry, only groups of dimension 2 and 3 are supported" );
    fi;

    if not IsRationalMatrixGroup( grp ) then
        Error( "grp must be a rational matrix group" );
    fi;

    # Special case for p1 and P1
    # binary Name segfaults
    # https://github.com/lbfm-rwth/carat/issues/112
    if IsTrivial(PointGroup(grp)) then
      if dim = 2 then
	return "min.2-1.1-0";
      elif dim = 3 then
	return "min.6-1.1-0";
      fi;
    fi;

    # Get temporary file names
    grpfile := CaratTmpFile( "grp" );  
    resfile := CaratTmpFile( "res" );

    # Write Carat input to file; Carat binary "Name" does not read from stdin
    #  Group is expected to have as translation lattice Z^n (not centered unit cells) and
    #  act from the left in CARAT.
    if IsAffineCrystGroupOnRight( grp ) then
      gen := GeneratorsOfGroup( StandardAffineCrystGroup( TransposedMatrixGroup( grp ) ) );
    else
      gen := GeneratorsOfGroup( StandardAffineCrystGroup( grp ) );
    fi;
    CaratWriteMatrixFile( grpfile, gen );

    # execute Carat program
    CaratExec( "Name", Concatenation( grpfile, " -c" ), resfile, "/dev/null" );

    # read Carat result from file, and remove temporary files
    input := InputTextFile( resfile );
    res := ReadLine( input );
    CloseStream(input);

    RemoveFile( grpfile );
    RemoveFile( resfile );

    return Chomp(res);

end );

[fingolfin]

Unfortunately Process does not currently allow redirecting stderr (see also issue #4657).

You can however use a trick and instead of directly executing the target program, execute another program (such as a shell) which performs the redirection. This is indeed what some code does by using Exec, e.g. this:

Exec(Concatenation("firefox \"file://", url,"\" >/dev/null 2>&1 &"));

However, Exec is otherwise a bad tool, as it does not allow check exit codes (to see if the command actually worked) and if you want to hide or capture output, it must always go through temporary files). So in general I recommend not using it. Relatedly, PR #5103 adds an alternative "better" version of Exec.

Luckily Exec does not perform magic, and you can just look at its source code in GAP library (use LocationFunc(Exec) to find out which file and line it is in, or use Display to show its code).

So you could adapt it (and/or my code for Exec2) to achieve what you want.

[kiryph]

Thanks for your answer with the pointer to the existing issue and PR and as well as the suggestions.

However, Exec is otherwise a bad tool, as it does not allow check exit codes (to see if the command actually worked) and if you want to hide or capture output, it must always go through temporary files). So in general I recommend not using it.

Agreed.

However, I will go with Exec as it is without handling of the return value.
Right now, the code is only used by myself and not others where proper error reporting should certainly be done.