🧑‍🌾 CmdLine test regressions on gz-launch5-win

[Crola1702]

Environment

• OS Version:
  • Windows
• Source build
  • ign-launch5 branch

Description

• Expected behavior: No test regressions
• Actual behavior: 6 CmdLine test regressions

Steps to reproduce

1. Run a job in ign_launch-win CI
2. See test regressions fail.

Output

CmdLine.HelpVsCompletionFlags:

C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:136
Expected: (std::string::npos) != (helpOutput.find(flag)), actual: 18446744073709551615 vs 18446744073709551615
Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'

CmdLine.Ls:

C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:70
Value of: output.find("CMakeFiles") != std::string::npos
  Actual: false
Expected: true
Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'

CmdLine.EchoSelf:

C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:80
Expected equality of these values:
  filePath
    Which is: "C:\\Jenkins\\workspace\\ign_launch-ign-5-win\\ws\\gz-launch\\test\\config\\echo.ign"
  output
    Which is: "Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled\nC:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)\n\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'\n\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'\n"
With diff:
@@ -1,1 +1,4 @@
-C:\\Jenkins\\workspace\\ign_launch-ign-5-win\\ws\\gz-launch\\test\\config\\echo.ign
+Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
+C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
+\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
+\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'\n

Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'

CmdLine.HelpSelf:

C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:88
Expected: (std::string::npos) != (output.find("Introspect Ignition launch")), actual: 18446744073709551615 vs 18446744073709551615
Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'

CmdLine.EchoErb:

C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:97
Expected equality of these values:
  "erb1234"
  output
    Which is: "Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled\nC:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)\n\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'\n\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'\n"
With diff:
@@ -1,1 +1,4 @@
-erb1234
+Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
+C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
+\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
+\tfrom C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'\n

Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'

CmdLine.EchoBadErb:

C:\Jenkins\workspace\ign_launch-ign-5-win\ws\gz-launch\src\cmd\ign_TEST.cc:106
Expected: (std::string::npos) != (output.find("is wrong for erb")), actual: 18446744073709551615 vs 18446744073709551615
Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'

They all have: ignition-tools-backward.dll not found in their output

[Crola1702]

This happened today in https://build.osrfoundation.org/job/ign_launch-ign-5-win/38/console

[azeey]

It turns out, at least from what I can tell, none of our libraries, except ign-launch2 and ign-launch5 run UNIT_ign_TEST/UNIT_gz_TEST on Windows 😅. I'm going to investigate why, but if anyone knows, please chime in.

[azeey]

I was able to resolve the error regarding the invalid path:

C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
	from C:/Jenkins/workspace/ign_launch-ign-5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'

with azeey@e6e7805, but now the UNIT_ign_TEST times out after 240s. Both the UNIT_Manager_TEST and now the UNIT_ign_TEST timeout while running dir or ls, which should return instantly. I'm also not sure if the UNIT_Manager_TEST ever passed. The original implementation (#120) indicates 6 failures, which is what we're seeing right now. So I propose we disable both tests on windows.

I'll try to apply azeey@e6e7805 to other gz libraries and hopefully enable UNIT_ign_TEST/UNIT_gz_TEST.

[Crola1702]

This issue is failing consistently in gz-launch5-win

Reference build: https://build.osrfoundation.org/job/gz_launch-ign-launch5-win/36/

Log output is different this time:

C:\J\workspace\gz_launch-ign-launch5-win\ws\gz-launch\src\cmd\ign_TEST.cc:136
Expected: (std::string::npos) != (helpOutput.find(flag)), actual: 18446744073709551615 vs 18446744073709551615
Library error: ignition-tools-backward.dll not found. Improved backtrace generation will be disabled
C:/J/workspace/gz_launch-ign-launch5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in ``': Invalid argument - C:/J/workspace/gz_launch-ign-launch5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/C:/J/workspace/gz_launch-ign-launch5-win/ws/build/ignition-launch5/bin/Release/ign-launch.exe --version (Errno::EINVAL)
	from C:/J/workspace/gz_launch-ign-launch5-win/ws/build/ignition-launch5/test/lib/Release/ruby/ignition/cmdlaunch5.rb:40:in `execute'
	from C:/J/workspace/gz_launch-ign-launch5-win/ws/install/ignition-tools/bin//ign:301:in `<main>'