Meson needs native understanding of symbol version scripts

[hughsie]

Meson doesn't seem to understand much about linkers; most of the projects porting to meson do something like this:

mapfile = 'libgusb.ver'
vflag = '-Wl,--version-script,@0@/@1@'.format(meson.current_source_dir(), mapfile)
gusb = shared_library(
  'gusb',
  sources : [
    'gusb-context.c',
  ],
  soversion : lt_current,
  version : lt_version,
  dependencies : [
    libusb,
  ],
  link_args : vflag,
  link_depends : mapfile,
  install : true
)

Which is fine, unless you discover that OS-X doesn't support --version-script : hughsie/libgusb#11

Rather than have to do evil things to the meson.build file to support OS-X, I would much prefer fr meson to know what a version-script is, rather than just dumping it in a command line. In the case of OS-X the version script could just be ignored. Ideas welcome, thanks!

[nirbheek]

Just for the record, could you explain why you need a version script instead of using symbol visibility prototypes?

[hughsie]

The version script provides a record of the ABI/API over time for each version.

[inigomartinez]

Several gnome and freedesktop packages are also using version scripts, so this feature would be very appreciated.

I would also like to point out that trying to workaround this doesn't seem to be an easy task, given the number of existing linkers and that their behaviour does not seem to be always the same.

[npmccallum]

I'm also using linker scripts in my projects. I'd very much appreciate this.

[emersion]

On macOS, Clang supports -exported_symbols_list, which is a list of exported symbols. Here is an example meson.build that supports both: https://github.com/mkhl/mrsh/blob/f94361fdff16f3a19a764691a418fb6734a634ad/meson.build#L40-L48

I'd like to help with this issue, however I'm not sure what the best design is. Make Meson parse version scripts and convert it to a symbol list when required (not all platforms support symbol versioning)? Or the other way around: make Meson upgrade symbol lists to version scripts if necessary (GCC doesn't support symbol lists), allow users to provide both if they want to have symbol versioning?

[jpakkane]

For symbol exporting the preferred choice is export attributes. If it is at all possible to switch to using those you should. Not only are they the only solution that is properly cross platform (meaning Windows) they also allow the compiler to do optimizations it otherwise would not be able to do.

[emersion]

Export attributes is a GNU extension. I'd rather have two compiler-specific lines in my build file rather than a GNU-specific attribute in all my headers. This also helps having more readable headers.

Windows does support symbol lists (https://docs.microsoft.com/en-us/cpp/build/reference/exports?view=vs-2017).

[emersion]

Anyway, if I understand correctly you'd prefer having Meson use version scripts when supported, and convert to symbol lists otherwise?

[npmccallum]

@emersion My preference is to support a data type in Meson which defines versioned symbol exports. Meson should then do the right thing with this data type. When the target linker doesn't support symbol versioning, Meson should fall back to unversioned symbol exports. Where the target linker doesn't support either versioned or unversioned symbol exports, export all symbols. However, I think all linkers provide some symbol export mechanism.

It should also be possible to detect what the linker/platform supports (similar to meson.get_compiler()). This way developers can override the default behavior.

[jpakkane]

Please bring your comments to #4747.

[mon]

I think the title of this issue should be changed or scope expanded as #4747 specifically refers to symbol exports, whereas linker scripts can be used for other things.

Specifically, I use a linker script for cross compiling to ARM, where it specifies the memory layout. I currently use the (hacky) solution of:

link_args = [
    '-T' + join_paths(meson.current_source_dir(), 'nrf52.ld'),
    '-L' + join_paths(meson.current_source_dir(), SDK_ROOT + 'components/toolchain/gcc'),
]

The -L is due to the linker script using IMPORT and needing references there.

Without very specific args such as link_includes and link_script, folders() or something like .absolute for file paths, this will remain "messy".

[phillipjohnston]

I am also using linker scripts for ARM (very common for embedded). Another benefit of understanding linker scripts is we could trigger a re-link of that file is modified.

(Currently, it will say "no work to do")

[mon]

@phillipjohnston as a workaround, I have had great success with link_depends.

[phillipjohnston]

Thanks for pointing that out. Works with an executable, but not with declare_dependency, which is a bummer for my particular project's organization scheme. (I have linker files organized by embedded platform. You select the target platform for your final executable as a dependency)

[blitz]

I'm trying to build a project with meson that has a linker script that needs to be preprocessed (because it includes headers that define the memory layout). This seems to be way harder to achieve than necessary...

[marc-hb]

Just FYI: Is BUILD_DIR = executable.get_id() OK? For: configure_file('linker_script.ld', BUILD_DIR) #5995

[marc-hb]

New! Add a binutils module #6063

[zackw]

/subscribe

I would like to point out that ELF version scripts are more powerful than symbol visibility attributes or OSX/Windows export lists. They give you the ability to define multiple versions of a single symbol, which is valuable for preserving backward binary compatibility. See https://sourceware.org/binutils/docs/ld/VERSION.html for documentation; see https://github.com/besser82/libxcrypt for a relatively straightforward example of a library that actually uses this.

[smcv]

I would like to point out that ELF version scripts are more powerful than symbol visibility attributes or OSX/Windows export lists. They give you the ability to define multiple versions of a single symbol, which is valuable for preserving backward binary compatibility.

Having versioned symbols is also necessary when two copies of the same library (like libcrypto.so.1.0.2 and libcrypto.so.1.1) might get pulled into the namespace of the same executable, or when two unrelated libraries define symbols with the same naming convention (json-c, jansson and json-glib all have symbols starting with json_, causing crashes when the dependency chain GLib -> libmount -> libcryptsetup -> json-c caused json-c symbols to appear in the namespace of executables that use jansson or json-glib).

There are several possible modes for using ELF version scripts, in increasing order of complexity

• to disambiguate between libraries: define a single version that contains all the exported symbols. In at least GNU ld, -Wl,--default-symver will reuse the SONAME as the name of that single version (build: Add a symbol version to all exported symbols akheron/jansson#540, https://gitlab.gnome.org/GNOME/json-glib/-/merge_requests/28); or you can use a simple version-script (like IJG libjpeg6b)
• to disambiguate between libraries as above, and also have explicit export control over exports as an alternative to visibility attributes (I don't have a good example of this right now)
• to disambiguate between libraries and control exports as above, and also track dependencies by the version in which each referenced symbol was added (build: Add a symbol version to all exported symbols json-c/json-c#639, util-linux, OpenSSL)
• all of the above, and also have older "hidden" versions of the same symbol (glibc is one of the few libraries that does this)

[jpakkane]

Since there are knowledgeable people on this thread, one thing that I have wondered about but never gotten a reasonable answer to is whether "linker scripts" and "symbol version scripts" are the same thing or not? That is, is there ever a case where you would use both for the same output file at the same time?

[emersion]

I don't think they're the same, and I think there are use-cases in which they're both used as the same time (e.g. libc implementations).

[zackw]

@jpakkane They are two different things.

A "linker script" specifies how to put together all the sections of the object files going into the link, to make the executable or shared library or whatever. It specifies things like "the text segment of the executable should be loaded at absolute address 0x800_0000, should be marked readable and executable, and should contain the data from the .text, .init, .fini, etc. sections of the input object files." They're needed for any kind of link operation, but the linker has a built-in script that's correct for "normal" programs; you only need to provide one yourself if you're linking something unusual, like an OS kernel or a bootloader.

A "symbol version script" specifies the names of all the symbols that are to be exported from a shared library, with the version tags to attach to each. They're only needed for shared libraries and for executables that load plugins, but all of those can and probably should have one; it's not nearly as unusual a need.

(Version tags let you offer backward binary compatibility on a per-symbol basis. For instance, suppose the function frobnicate used to take three arguments but now you want to add a fourth. Instead of adding a new name like frobnicate2, you can define frobnicate@MYLIBRARY_1.0 that takes only three arguments, and frobnicate@@MYLIBRARY_1.1 that takes four. The prototype in mylibrary.h would specify four arguments, and newly compiled programs would link against the four-argument version -- that's what the double @@ means -- but old binaries still get the three-argument function that they expect. Internally, the three-argument function would probably tack on a default value for the fourth argument and call the four-argument function. Sadly, the tooling for this is not what it should be; in addition to the version script, you need to put a horrible pile of macros that expand to assembly inserts in your C source files. The compiler devs are working on improvements but I'm not holding my breath.)

GNU ld lets you embed a version script inside a linker script; but it also lets you put each in its own file, and usually that's more convenient for maintenance. I don't know about other linkers.

---

Ideally, Meson's executable() and shared_library() targets would have separate optional arguments for linker scripts and symbol version scripts (it can make sense to have more than one of either) and would know how to pass each of these to each supported linker. If I were the project manager, though, I'd prioritize the support for symbol version scripts, as this is the more commonly needed feature.

[smcv]

Even though its title mentions linker scripts, this issue report was originally about symbol-version scripts, as introduced by -Wl,--version-script with GNU ld (and so was my recent comment). It would probably be clearer to retitle it to be about symbol-version scripts, which are what the original issue reporter wanted, and have a separate issue for linker scripts.

Some of the people who later commented on this issue genuinely do want linker scripts (for embedded stuff), and not symbol-version scripts.

If I were the project manager, though, I'd prioritize the support for symbol version scripts, as this is the more commonly needed feature.

That's what I'd say too, but perhaps I'm biased by using symbol-version scripts somewhat frequently, and never having wanted a linker script :-)

[phillipjohnston]

If I were the project manager, though, I'd prioritize the support for symbol version scripts, as this is the more commonly needed feature.

That's what I'd say too, but perhaps I'm biased by using symbol-version scripts somewhat frequently, and never having wanted a linker script :-)

This is definitely a "what do you use" bias - I've never used a symbol version script in my life, while I use linker scripts every day. How could I ever agree that's "more commonly needed"? =P

[jpakkane]

For symbol versioning scripts there is already some work in #4747. Visual Studio has a similar thing and so we'd probably want to support both with one syntax.

[antonysigma]

I think the title of this issue should be changed or scope expanded as #4747 specifically refers to symbol exports, whereas linker scripts can be used for other things.

Specifically, I use a linker script for cross compiling to ARM, where it specifies the memory layout. I currently use the (hacky) solution of:

link_args = [
    '-T' + join_paths(meson.current_source_dir(), 'nrf52.ld'),
    '-L' + join_paths(meson.current_source_dir(), SDK_ROOT + 'components/toolchain/gcc'),
]

The -L is due to the linker script using IMPORT and needing references there.

Without very specific args such as link_includes and link_script, folders() or something like .absolute for file paths, this will remain "messy".

Hi, I suppose #3047 is still within the scope of "replacing linker script" by -Wl,-T,path/to/linker_script.ld?

The ad hoc solution at #3047 (comment) works for me, except that clang complains about the wrong placement of the -Wl,-T,... argument in the link command. Here's my patch to resolve the bug: #7505. Could you take a look and try it out with clang toolchain?

[dcbaker]

For the issue of joining strings and files for arguments #12287 maybe of help