Add a file_argument function

[dcbaker]

A common problem we run into is passing arguments to compilers that need to have a file included in the argument, and cannot be split. Take for example the classic gnu linker script -Wl,--linker-script,<path>. if <path> is a File object, this can't be easily represented. A workaround was to use str.format, however, that as never meant to work and has been marked as deprecated (broken, actually).

There are a number of ways this could be fixed:

1. We could special case the addition (+) operator for str and file, but that feels a little gross, and is likely to backfire in spectacular ways.
2. We could add a method to convert files to strings, but that has some caveats as well, since file objects can be built, and therefore not always available. This would break the ordering guarantees that Meson can create by using files
3. we can add a new function for handling this

I've taken route 3, which allows both getting the correct string value, but also creating the correct ordering guarantees. It works like this

build_target(
  ...,
  link_args : file_argument('-Wl,--version-script,', files('file.map')),
)

This returns an opaque argument that Meson reduces to a string argument, which is passed to the compiler/linker, and a correct dependency on the file object being passed.

This is built on top of the first three patches of #11981

Fixes: #12259

[medhefgo]

Looks like this only supports two args in a fixed order? That seems overly restrictive to me. It would be nicer if it supported any number of strings and files in any order. And maybe even lists of those and some separator/joiner kwarg.

[dcbaker]

What is the use case for more than one file Do you have an example of an argument that works like that? The only uses I've run into or seen submited are things like --arg=<path> or -Wl,--arg,<path>. Generally, we try to be conservative and not address issues that we don't have a current use case for.

[tristan957]

what about a .to_list() method on a files object that creates a list of filenames?

[dcbaker]

what about a .to_list() method on a files object that creates a list of filenames?

files() has a signature like files(...str) -> array[file]. So we wouldn't have any place to attach a to_list method to.

And with to_str on file itself you're still left having to write:

mapfile = files('foo.map)
build_target(
  link_args : ['-Wl,--linker-script,' + mapfile[0].to_str()],
  link_depends : mapfile,
)

Which is easier to get wrong than using the file_argument() approach, where Meson can ensure the dependency is correctly inserted in a DRY manner

[tristan957]

In the past, someone has discussed adding better support for linker scripts. Maybe @eli-schwartz

[medhefgo]

Do you have an example of an argument that works like that?

I don't have a concrete use-case. But I've seen several meson functions in the docs become variadic when they would've clearly been sensible to be so from the start. This one strikes me as one.

Btw, yes, in systemd we use format to string for linker scripts (and already set a link dependency for it). And I just tested this PR and it seems to works.

Now if only meson --fatal-meson-warnings wouldn't make it a fatal deprecation warning unless you actually target >= 1.3.0 (since there would literally be nothing you can do about it until then)… sigh

[dcbaker]

Now if only meson --fatal-meson-warnings wouldn't make it a fatal deprecation warning unless you actually target >= 1.3.0 (since there would literally be nothing you can do about it until then)… sigh

hmmmm... normally that's how deprecations work. Let me see...

[antonysigma]

I can add another use case of the file_argument function: functional testing of user code with binary test vectors.

For example, the following meson build rule scans the project for a raw data file, decode its absolute path, then add it to the compiler flags. In the unit-test file, we invoke fopen to read the binary data from disk for data validations.

unit_test_exe = executable('test-main',
  sources: 'test-main.cpp',
  cpp_args: file_argument('-DRAW_DATA_PATH="@0@"', files('data/raw-data.bin')),
)

test('Raw data validation test with hardcoded binary datafile path', unit_test_exe)

And, the pseudo-code of a typical unit test file:

#ifndef RAW_DATA_PATH
#error preprocessor flag "RAW_DATA_PATH" not yet defined
#endif
TEST_CASE("Can decode the raw binary file") {
    constexpr char file_path[] { RAW_DATA_PATH };
    const std::vector<byte> raw_data = my_fopen(file_path);
    
    assert(raw_data.size > 0);
    assert(std::equal(raw_data, my_simulated_data));
}

[eli-schwartz]

For example, the following meson build rule scans the project for a raw data file, decode its absolute path, then add it to the compiler flags. In the unit-test file, we invoke fopen to read the binary data from disk for data validations.

This seems like a bad use case, actually... adding a build ordering rule here that forces the executable to be recompiled every time the test data file is updated would be somewhat wasteful unless the executable actually used #embed (which itself seems odd for a test-time executable). And the design of this PR assumes that the correct thing to do is to helpfully add that rebuild dependency for you.

[eli-schwartz]

Now if only meson --fatal-meson-warnings wouldn't make it a fatal deprecation warning unless you actually target >= 1.3.0 (since there would literally be nothing you can do about it until then)… sigh

It very much depends on what sort of deprecation we are talking about.

The general rule of thumb is that most deprecations are "XXX was supported but we have changed our mind and would like to remove it. We promised you the ability to use XXX and we are not going to retract that". In such cases we do not issue a warning at all unless you're targeting a high enough minimum. There's a final summary at "notice" level for things which will become deprecated in the future but this does not trigger fatal-meson-warnings since it's not actionable.

The other case of deprecation warnings is when something is asserted by meson developers to be "always wrong" regardless of version. Maybe we never promised you it would work. Maybe it's just "so darned buggy" that we regard it as an ecosystem hazard if we allow it to be used even on ancient versions of meson that had no choice.

In this case we are warning about something that was never guaranteed to work, does not work reliably, and has no guidance other than "have you considered manually constructing your raw strings with meson.current_source_dir()".

The fact that an unrelated PR proposes to provide ergonomics around one limited use case for the deprecated functionality, doesn't have any bearing on the timeline for deprecation (it is deprecated for a vast number of scenarios, most of which don't involve compiler arguments).

[mon]

We could special case the addition (+) operator for str and file, but that feels a little gross, and is likely to backfire in spectacular ways.

It would be nice to special case str+file anyway to throw an error saying "you probably want to use file_argument", as that's the first approach anyone is going to try when adding a linker script.