Can't get lines and snippets displayed

[aleks-f]

I'm having trouble getting lines and snippets displayed.

$ uname -a
Darwin M2.local 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:04 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6020 arm64

$ c++ --version
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

With -DCPPTRACE_GET_SYMBOLS_WITH_LIBDL -DCPPTRACE_DEMANGLE_WITH_CXXABI -DCPPTRACE_UNWIND_WITH_LIBUNWIND

I get this (no line numbers or snippets):

$ bin/Darwin/arm64/traced
Stack trace from functionB:
Stack trace (most recent call first):
#0 0x594100010088c537 in functionB() at /Users/alex/poco/Trace/samples/trace/bin/Darwin/arm64/traced
#1 0x687200010088c67f in functionA() at /Users/alex/poco/Trace/samples/trace/bin/Darwin/arm64/traced
#2 0x9e1a00010088c69b in main at /Users/alex/poco/Trace/samples/trace/bin/Darwin/arm64/traced
Poco::Exception caught: Exception: Stack trace (most recent call first):
#0 0xff48800100e2826f in Poco::Trace::StackTrace::StackTrace(unsigned long) at /Users/alex/poco/lib/Darwin/arm64/libPocoTraced.103.dylib
#1 0xda54000100e2830b in Poco::Trace::StackTrace::StackTrace(unsigned long) at /Users/alex/poco/lib/Darwin/arm64/libPocoTraced.103.dylib
#2 0x3b1b80010088c293 in functionC() at /Users/alex/poco/Trace/samples/trace/bin/Darwin/arm64/traced
#3 0x993800010088c54f in functionB() at /Users/alex/poco/Trace/samples/trace/bin/Darwin/arm64/traced
#4 0x687200010088c67f in functionA() at /Users/alex/poco/Trace/samples/trace/bin/Darwin/arm64/traced
#5 0x9e1a00010088c69b in main at /Users/alex/poco/Trace/samples/trace/bin/Darwin/arm64/traced

I hoped libbacktrace would give a better output, so I changed -DCPPTRACE_GET_SYMBOLS_WITH_LIBDL to -DCPPTRACE_GET_SYMBOLS_WITH_LIBBACKTRACE (and linked with libbacktrace, of course)

But sadly, no:

bin/Darwin/arm64/traced
Stack trace from functionB:
Stack trace (most recent call first):
#0 0x0000000000000000
#1 0x0000000000000000
#2 0x0000000000000000
Poco::Exception caught: Exception: Stack trace (most recent call first):
#0 0x0000000000000000
#1 0x0000000000000000
#2 0x0000000000000000
#3 0x0000000000000000
#4 0x0000000000000000
#5 0x0000000000000000

I'm calling cpptrace::generate_trace().print() and cpptrace::generate_trace().print_with_snippets() but cant get the line numbers or snippets displayed. What am I doing wrong?

[jeremy-rifkin]

Hi, with libdl it's expected that there's no line info since it uses dynamic export information instead of debug info. Libbacktrace I don't know about, I am not sure I've ever tested that on macos. My advice would be try using -DCPPTRACE_GET_SYMBOLS_WITH_DBGHELP=On, which is the default for the library and the most well-supported back-end (and it's tested on macos fairly thoroughly).

[aleks-f]

How do I build on macos with DBGHELP? I thought that was windows only (and cpptrace documentation lists is as such)

** Creating dependency info for src/symbols_with_dbghelp.cpp
In file included from src/symbols_with_dbghelp.cpp:5:
/Users/alex/poco/Trace/src/platform/dbghelp_syminit_manager.hpp:9:10: fatal error: 'windows.h' file not found
#include <windows.h>
         ^~~~~~~~~~~
1 error generated.

[jeremy-rifkin]

My apologies, I meant to copy -DCPPTRACE_GET_SYMBOLS_WITH_LIBDWARF=On. Dbghelp is windows, libdwarf is the preferred back-end everywhere else.

[aleks-f]

yeah, I tried dwarf and addr2line, but it was worse - those did not give function names; I'll see how it does on Linux, that is where I primarily need it

[aleks-f]

Ah no, it is not -g.

I'm not sure you'll be willing to get into our legacy build system intricacies (we officially use cmake and there will be cmake build, but I'm no fan so it's still gmake phase).

Anyway, here's what I was trying on mac: https://github.com/pocoproject/poco/blob/a86a6ea150b14214f180d6e0bbfd86544a741555/Trace/Makefile-Trace#L16

And the good news is that on Linux/g++ the backtrace/cxxabi/unwind combination works fine, line numbers and snippets:

[aleks-f]

sadly, the release build is crippled:

[jeremy-rifkin]

Thanks for linking what you're working on, I can probably poke around this weekend though I'll have to learn how the custom build setup works.

I see you're trying pull in the cpptrace code and have it built as part of everything else. This is possible but the first thing that comes to mind is that the cpptrace cmake does a lot of heavy lifting to find a configuration that works and do feature testing etc (this is how it manages to be portable). The cpptrace cmake config also fetches libdwarf and zstd if needed if libdwarf chosen - likely these would need to be vendored too if you wanted to vendor cpptrace and use libdwarf. One thing I've ran into with libdwarf in the past is that if it can't find zlib and zstd during building it won't have decompression support. It seems gcc/clang can emit compressed dwarf by default. I'm wondering if this might explain why dwarf wasn't working when you tested.

[jeremy-rifkin]

As to why it's not working in release, I'm not sure. I'd want to double check debug symbols are actually there of course. If not that, it's likely a problem with libbacktrace. I've had issues with libbacktrace before which was part of the motivation for doing custom dwarf handling.

[aleks-f]

yes, stripping symbols was the problem, but still somewhat hit-and-miss:

[aleks-f]

@jeremy-rifkin FWIW, here's a bit more systematic description of what we're doing. POCO-esquecpptrace wrap (Poco::Trace) is a thin one, without dependencies on any poco libs, builds out of the box with make

On linux, backtrace works fine. on mac, dl is passable, but I'd like to get dwarf working if possible, preferably the clang internal one. if you can assist in any way, please do