Pattern matching when names contain space

[goebbe]

A Vorta user had a hard time matching a folder-name with a space on MacOS:
borgbase/vorta#2156 (reply in thread)
Since matching names that contain a single space is a common use case, I decided to report, here.

I tried to reproduce the issues using Vorta 0.10.1 with borg 1.4 on LinuxMint 22.

My folder was named test folder, with a single space in the middle of the folder-name.
I used --patterns-from with the standard sh: and the fm: matcher and tried to exclude the folder from my backup.

The following did not work:

• Putting the file path into quotation marks (single or double).
• Escaping the space with \.
• Replacing the space with \s.

As code:

-'home/user/test folder'
-"home/user/test folder"
-home/user/test\ folder

In short, I did not find a simple way to match the folder name, using --pattern matching with the standard sh: or fm: matcher.

The following worked for me. However, I had to use the re: matcher (regular expressions), in order to match the single space by \s.
-re:home/user/test\sfolder

Currently, matching names with (a single) space seems not to be documented.
If this is the expected behaviour (i.e. not a bug) - maybe we could add a working example (with re:) to the docs?

[ThomasWaldmann]

Did you try this as a line within your patterns file?:

-home/user/test folder

[goebbe]

Did you try this as a line within your patterns file?:

-home/user/test folder

Oh dear, inside a pattern-file this works! 👍 :-)
The original discussion was about using --pattern=-home/user/test folder via Vorta, on the command line - which did not work. (I just tested this to confirm)

So when searching a solution, I was just using my existing --patterns-from file.
I wrongly assumed that the same issue arise independent of --pattern or --patterns-from - and I did not try to reproduce the original issue using a pattern file. :-(

So the following --pattern do not work, when provided with borg create (Vorta) as command line option:

--pattern=-home/user/test folder 
--pattern=-'home/user/test folder'
--pattern=-home/user/test\ folder

[ThomasWaldmann]

Giving command line arguments with spaces inside is a problem because the shell usually splits arguments at spaces, so one needs to work around that. This is not borg specific, but just a shell thing.

The patterns file works on a line-by-line basis and if the line starts with +/-, the processing just removes that and uses the path "as is", thus no workarounds needed.

[dgraziotin]

Hello, thanks @goebbe for opening the issue. I confirm as well that --patterns-from works as expected.

@ThomasWaldmann I also report that escaping with \s does not work, contrary to what I believed.

Giving command line arguments with spaces inside is a problem because the shell usually splits arguments at spaces, so one needs to work around that. This is not borg specific, but just a shell thing.

I agree, but it appears that there is no known way to escape, at least from Vorta and macOS. All these formats fail:

-'home/user/test folder'
-"home/user/test folder"
-home/user/test\ folder
-home/user/test\sfolder

[ThomasWaldmann]

The plus/minus belongs to the option value and values with spaces need quoting (for example), so can you try?:

--pattern='-home/user/test folder'

Values starting with a minus char could lead to parsing problems, because short options also start with a minus char.

So, in general, you can save yourself a lot of pain by using --patterns-from and not having such paths/filenames on the commandline.

[jdchristensen]

All of these seem to work correctly:

$ borg create --list --dry-run '--pattern=-subdir/test folder' repo::space subdir
$ borg create --list --dry-run --pattern='-subdir/test folder' repo::space subdir
$ borg create --list --dry-run --pattern=-subdir/test\ folder repo::space subdir

producing

- subdir/hello
x subdir/test folder
x subdir/test folder/foo
- subdir

So maybe you're seeing a bug in Vorta?

[goebbe]

@jdchristensen Thank you!
I just tested your examples on the command line and can confirm, that on the command line all three variants indeed work.
The only sensible conclusion is that this must be an issue with Vorta itself.

Here are the relevant bits from the Vorta logs, that let me to the (wrong) conclusion that this is a borg issue:

2024-12-06 09:31:58,478 - vorta.borg.jobs_manager - DEBUG - Start job on site: 5
2024-12-06 09:31:58,496 - vorta.borg.borg_job - INFO - Running command /usr/bin/borg create --remote-path=/usr/local/bin/borg --list --progress --info --log-json --json --filter=AM -C lz4 '--pattern=-home/User/test folder' --patterns-from /home/User/patterns-vorta-repo.lst ssh://backupUser@192.168.0.5:22/~/backup/vorta-repo-EB845::User-EliteBook-845-2024-12-06-093158 /home/User
2024-12-06 09:31:58,587 - vorta.borg.borg_job - WARNING - usage: borg create [-h] [--critical] [--error] [--warning] [--info] [--debug]
2024-12-06 09:31:58,588 - vorta.borg.borg_job - WARNING - [--debug-topic TOPIC] [-p] [--iec] [--log-json]
2024-12-06 09:31:58,588 - vorta.borg.borg_job - WARNING - [--lock-wait SECONDS] [--bypass-lock] [--show-version]
2024-12-06 09:31:58,588 - vorta.borg.borg_job - WARNING - [--show-rc] [--umask M] [--remote-path PATH]
2024-12-06 09:31:58,588 - vorta.borg.borg_job - WARNING - [--remote-ratelimit RATE] [--upload-ratelimit RATE]
2024-12-06 09:31:58,590 - vorta.borg.borg_job - WARNING - [--remote-buffer UPLOAD_BUFFER]
2024-12-06 09:31:58,590 - vorta.borg.borg_job - WARNING - [--upload-buffer UPLOAD_BUFFER] [--consider-part-files]
2024-12-06 09:31:58,590 - vorta.borg.borg_job - WARNING - [--debug-profile FILE] [--rsh RSH] [-n] [-s] [--list]
2024-12-06 09:31:58,590 - vorta.borg.borg_job - WARNING - [--filter STATUSCHARS] [--json] [--no-cache-sync]
2024-12-06 09:31:58,590 - vorta.borg.borg_job - WARNING - [--stdin-name NAME] [--stdin-user USER]
2024-12-06 09:31:58,590 - vorta.borg.borg_job - WARNING - [--stdin-group GROUP] [--stdin-mode M]
2024-12-06 09:31:58,590 - vorta.borg.borg_job - WARNING - [--content-from-command] [--paths-from-stdin]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--paths-from-command] [--paths-delimiter DELIM]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [-e PATTERN] [--exclude-from EXCLUDEFILE]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--pattern PATTERN] [--patterns-from PATTERNFILE]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--exclude-caches] [--exclude-if-present NAME]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--keep-exclude-tags] [--exclude-nodump] [-x]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--numeric-owner] [--numeric-ids] [--noatime] [--atime]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--noctime] [--nobirthtime] [--nobsdflags] [--noflags]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--noacls] [--noxattrs] [--sparse] [--files-cache MODE]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--read-special] [--comment COMMENT]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--timestamp TIMESTAMP] [-c SECONDS]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - [--chunker-params PARAMS] [-C COMPRESSION]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - ARCHIVE [PATH ...]
2024-12-06 09:31:58,591 - vorta.borg.borg_job - WARNING - borg create: error: argument ARCHIVE: "'--pattern=-home/User/test": No archive specified

Note that when copying the exact borg command from the logs to the command line, without any modification, the borg create works as expected, without warning or complaint and the --pattern is applied as expected.

/usr/bin/borg create --remote-path=/usr/local/bin/borg --list --progress --info --log-json --json --filter=AM -C lz4 '--pattern=-home/User/test folder' --patterns-from /home/User/patterns-vorta-repo.lst ssh://backupUser@192.168.0.5:22/~/backup/vorta-repo-EB845::User-EliteBook-845-2024-12-06-093158 /home/User

@ThomasWaldmann I am not a programmer, but to me this looked like a borg Warning - do you have any idea/ hint what the problem might be?

I will open an issue on Vorta.

[goebbe]

From the borg side of view, I would like to suggest adding the following to the pattern docs.
I am aware this might be obvious to devs - but it might not be to the common user. :-)

Pattern matching special characters,e.g. white space, on the command line requires some adjustments - such as escaping special characters using \.
Example: The following pattern would match and exclude "item name" on the command line:
--pattern=-path/item\ name
However, when using --patterns-from, special characters in pattern files must not be escaped.

[dgraziotin]

All of these seem to work correctly:

$ borg create --list --dry-run '--pattern=-subdir/test folder' repo::space subdir
$ borg create --list --dry-run --pattern='-subdir/test folder' repo::space subdir
$ borg create --list --dry-run --pattern=-subdir/test\ folder repo::space subdir

producing

- subdir/hello
x subdir/test folder
x subdir/test folder/foo
- subdir

So maybe you're seeing a bug in Vorta?

It appears so, as Vorta does not let me escape the whole option either (like in the option by @ThomasWaldmann, --pattern='-home/user/test folder'). There were some attempts and screenshots in the related Vorta discussion before opening this bug.

[goebbe]

The underlying issue has been fixed in Vorta.
Background: There can be a subtle difference between the borg command (Shell command) that can be found in the logs of Vorta and the list of arguments that are actually executed. This is due to the way the command line options are handed over, using Popen().
This was the reason why Vorta /borg raised an error and printed a borg warning - even though the borg command (Shell command) printed in the Vorta logs was actually correct.

@ThomasWaldmann I plan to add a comment/ phrase to the "borg help patterns" docs about matching special characters on the command line, if this is o.k. with you.
See: #8578 (comment)

[ThomasWaldmann]

Maybe rather use single quotes than backslash-escaping, that's cleaner imho.

Also, explain that the quoting is shell specific and it is due to the shell doing argument splitting at spaces that one needs to do that. For a file read directly by borg, there is no shell involved and thus the quoting is not needed.