LTFS 2.4.2.0 (Pre0)

This is a pre-release of LTFS 2.4.2.0. (This is not related into IBM's release)

The purpose of this release is to support 3rd party backend collection (LTFS backends 0.1 in https://github.com/LinearTapeFileSystem/ltfs-backends).

LTFS backends 0.1 shall work file under this release at least.

Equivalent of IBM Spectrum Archive SDE 2.4.1.1 (10227)

IBM released IBM Spectrum Archive 2.4.1.1 (10227) at 6adff9f. So the LTFS project releases new version 2.4.1.1.

Change Log:

• Fix the percent encoder
• ltfsck writes down new index at the end of data stream

Detail description of

• The percent encoder patch
  • Fix a percent encoder bug for '%' in the filename, fixed encoder can encode '%' '%25' correctly
  • Relax the index reader for indices written by previous buggy encoder
  • Do not follow previous index backward when an index is unpersable because of invalid name at ltfsck
  • ltfsck writes down new index at the end of data stream (not the end of previous good index) to keep data salvage possibility

Symptom of new index parser against bad encoding in the previous index generator

0-0. No percent encode triggers in the all filenames and directories on a tape

Mountable and recoverable

0-1. There is percent triggers but no '%' in the all filenames and directories on a tape

Mountable and recoverable

1. There is percent triggers and '%XX' which can be decoded to correct ASCII characters expect '/' or UNIT SEPARATOR

Mountable and recoverable.

The index is corrupted but LTFS can mount the tape. Some file name or directory name may be modified automatically.

For example, file "aaa:%20" is written and unmount the tape. After that the tape is mounted, the file name may be "aaa: " instead of "aaa:%20"

The same thing happens "%XX%XX%XX" sequence is valid UTF-8 characters.

2. There is percent triggers and '%1f' or '%2f'

Mountable and recoverable.

'%2f' is encoded to '%2f', so '%2f' is decoded to '/'. And '%2f' is encoded to '%2f', so '%2f' is decoded to a UNIT SEPARATOR.

Both '/' and UNIT SEPARATOR is invalid character for file name and directory name. So new index reader revert them to '%2f' or '%1f'.

3. There is percent triggers and '%XX%XX%XX' which can't be decoded to correct UTF-8 characters

Mount fail and recovery fail.

Invalid UTF-8 character is not valid for file name or directory name. So mount failure happens. At recovery process, index seeking is stopped when this type of error happens, so this kind of tape cannot be recovered in this version. (We will handle this on a future release.)

IBM Spectrum Archive SDE 2.4.0.2-1 (10071)

Equivalent of IBM Spectrum Archive SDE 2.4.0.2 (10071) with the percent encoder patch.

• The percent encoder patch
  • Fix a percent encoder bug for '%' in the filename, fixed encoder can encode '%' '%25' correctly
  • Relax the index reader for indices written by previous buggy encoder
  • Do not follow previous index backward when an index is unpersable because of invalid name at ltfsck
  • ltfsck writes down new index at the end of data stream (not the end of previous good index) to keep data salvage possibility

Symptom of new index parser against bad encoding in the previous index generator

0-0. No percent encode triggers in the all filenames and directories on a tape

Mountable and recoverable

0-1. There is percent triggers but no '%' in the all filenames and directories on a tape

Mountable and recoverable

1. There is percent triggers and '%XX' which can be decoded to correct ASCII characters expect '/' or UNIT SEPARATOR

Mountable and recoverable.

The index is corrupted but LTFS can mount the tape. Some file name or directory name may be modified automatically.

For example, file "aaa:%20" is written and unmount the tape. After that the tape is mounted, the file name may be "aaa: " instead of "aaa:%20"

The same thing happens "%XX%XX%XX" sequence is valid UTF-8 characters.

2. There is percent triggers and '%1f' or '%2f'

Mountable and recoverable.

'%2f' is encoded to '%2f', so '%2f' is decoded to '/'. And '%2f' is encoded to '%2f', so '%2f' is decoded to a UNIT SEPARATOR.

Both '/' and UNIT SEPARATOR is invalid character for file name and directory name. So new index reader revert them to '%2f' or '%1f'.

3. There is percent triggers and '%XX%XX%XX' which can't be decoded to correct UTF-8 characters

Mount fail and recovery fail.

Invalid UTF-8 character is not valid for file name or directory name. So mount failure happens. At recovery process, index seeking is stopped when this type of error happens, so this kind of tape cannot be recovered in this version. (We will handle this on a future release.)

Equivalent of IBM Spectrum Archive SDE 2.4.1.0 (10219)

IBM released IBM Spectrum Archive 2.4.1.0 (10219) at fd4bbe7. So the LTFS project releases new version 2.4.1.0.

!! WARNING !!

This version is corrupted.

The percent encoder for the LTFS index is corrupted. This corruption creates an unparsable LTFS index on the tape. An index corruption happens when following 2 conditions are met.

1. File name or directory name has ':' (0x3A) or C0 control codes (0x01-0x1f) except \t(0x09), \n(0x0A) and \r(0x0d) at least (Let's call a percent encode trigger)
2. File name or directory name has '%' (0x25)

That's because the percent encoder modified into 2.4.0.1 is incomplete and it doesn't encode '%' at all. In other words, '%' shall be encoded to "%25" be the percent encoder but '%' remains as '%' in the 2.4.0.1 encoder.

Corruption Variations

0-0. No percent encode triggers in the all filenames and directories on a tape

There is no corruption

0-1. There is percent triggers but no '%' in the all filenames and directories on a tape

There is no corruption

1. There is percent triggers and '%XX' which can be decoded to correct ASCII characters expect '/' or UNIT SEPARATOR

The index is corrupted but LTFS can mount the tape. Some file name or directory name may be modified automatically.

For example, file "aaa:%20" is written and unmount the tape. After that the tape is mounted, the file name may be "aaa: " instead of "aaa:%20"

The same thing happens "%XX%XX%XX" sequence is valid UTF-8 characters.

2. There is percent triggers and '%1f' or '%2f'

'%2f' is encoded to '%2f', so '%2f' is decoded to '/'. And '%2f' is encoded to '%2f', so '%2f' is decoded to a UNIT SEPARATOR.

Both '/' and UNIT SEPARATOR is invalid character for file name and directory name. As a result, the index reader detects an error on fine name or directory name.

In the recovery, the indices which has an invalid character is skipped. As a result, LTFS rolls back to the latest good index. In this time, LTFS overwrites the data or the indices after latest good index. So data loss may happen.

3. There is percent triggers and '%XX%XX%XX' which can'y be decoded to correct UTF-8 characters

Invalid UTF-8 character is not valid for file name or directory name. So same thing on case 2 above, mount failure and data loss at recovery, happens.

Original descriptions

Change Log:

• Supported TS1160 tape drive

Equivalent of IBM Spectrum Archive SDE 2.4.0.2 (10071)

IBM released IBM Spectrum Archive 2.4.0.2 (10071) at 9bceaf8. So the LTFS project releases new version 2.4.0.2.

!! WARNING !!

This version is corrupted.

The percent encoder for the LTFS index is corrupted. This corruption creates an unparsable LTFS index on the tape. An index corruption happens when following 2 conditions are met.

1. File name or directory name has ':' (0x3A) or C0 control codes (0x01-0x1f) except \t(0x09), \n(0x0A) and \r(0x0d) at least (Let's call a percent encode trigger)
2. File name or directory name has '%' (0x25)

That's because the percent encoder modified into 2.4.0.1 is incomplete and it doesn't encode '%' at all. In other words, '%' shall be encoded to "%25" be the percent encoder but '%' remains as '%' in the 2.4.0.1 encoder.

Corruption Variations

0-0. No percent encode triggers in the all filenames and directories on a tape

There is no corruption

0-1. There is percent triggers but no '%' in the all filenames and directories on a tape

There is no corruption

1. There is percent triggers and '%XX' which can be decoded to correct ASCII characters expect '/' or UNIT SEPARATOR

The index is corrupted but LTFS can mount the tape. Some file name or directory name may be modified automatically.

For example, file "aaa:%20" is written and unmount the tape. After that the tape is mounted, the file name may be "aaa: " instead of "aaa:%20"

The same thing happens "%XX%XX%XX" sequence is valid UTF-8 characters.

2. There is percent triggers and '%1f' or '%2f'

'%2f' is encoded to '%2f', so '%2f' is decoded to '/'. And '%2f' is encoded to '%2f', so '%2f' is decoded to a UNIT SEPARATOR.

Both '/' and UNIT SEPARATOR is invalid character for file name and directory name. As a result, the index reader detects an error on fine name or directory name.

In the recovery, the indices which has an invalid character is skipped. As a result, LTFS rolls back to the latest good index. In this time, LTFS overwrites the data or the indices after latest good index. So data loss may happen.

3. There is percent triggers and '%XX%XX%XX' which can'y be decoded to correct UTF-8 characters

Invalid UTF-8 character is not valid for file name or directory name. So same thing on case 2 above, mount failure and data loss at recovery, happens.

Original descriptions

Change Log:

• Handle read-only combination of tape and drive correctly (Bug Fix)
• Tiny fix of plugin help print

Equivalent of IBM Spectrum Archive SDE 2.4.0.1 (10062)

IBM released IBM Spectrum Archive 2.4.0.1 (10062) at fc26267. So the LTFS project releases new version 2.4.0.1.

!! WARNING !!

This version is corrupted.

The percent encoder for the LTFS index is corrupted. This corruption creates an unparsable LTFS index on the tape. An index corruption happens when following 2 conditions are met.

1. File name or directory name has ':' (0x3A) or C0 control codes (0x01-0x1f) except \t(0x09), \n(0x0A) and \r(0x0d) at least (Let's call a percent encode trigger)
2. File name or directory name has '%' (0x25)

That's because the percent encoder modified into 2.4.0.1 is incomplete and it doesn't encode '%' at all. In other words, '%' shall be encoded to "%25" be the percent encoder but '%' remains as '%' in the 2.4.0.1 encoder.

Corruption Variations

0-0. No percent encode triggers in the all filenames and directories on a tape

There is no corruption

0-1. There is percent triggers but no '%' in the all filenames and directories on a tape

There is no corruption

1. There is percent triggers and '%XX' which can be decoded to correct ASCII characters expect '/' or UNIT SEPARATOR

The index is corrupted but LTFS can mount the tape. Some file name or directory name may be modified automatically.

For example, file "aaa:%20" is written and unmount the tape. After that the tape is mounted, the file name may be "aaa: " instead of "aaa:%20"

The same thing happens "%XX%XX%XX" sequence is valid UTF-8 characters.

2. There is percent triggers and '%1f' or '%2f'

'%2f' is encoded to '%2f', so '%2f' is decoded to '/'. And '%2f' is encoded to '%2f', so '%2f' is decoded to a UNIT SEPARATOR.

Both '/' and UNIT SEPARATOR is invalid character for file name and directory name. As a result, the index reader detects an error on fine name or directory name.

In the recovery, the indices which has an invalid character is skipped. As a result, LTFS rolls back to the latest good index. In this time, LTFS overwrites the data or the indices after latest good index. So data loss may happen.

3. There is percent triggers and '%XX%XX%XX' which can'y be decoded to correct UTF-8 characters

Invalid UTF-8 character is not valid for file name or directory name. So same thing on case 2 above, mount failure and data loss at recovery, happens.

Original descriptions

Change Log:

• Supported initialized LTO 8 Type M cartridge
• Support percent-encoding attribute in all name types in LTFS Format Spec 2.4
• Make the sg backend default in Linux
• Use persistent reserve command for drive reservation in the sg backend and the iokit backend

Changes for developers

• Add files for pkg-config to use libltfs
• Use "signed char" explicitly in "char"
• Add following options in the configure script
  • SNMP support switch (--enable-snmp [default: enable])
  • lin_tape support switch (--enable-lintape [default: disable])
• Implements message argument checker [--enable-message-checker (efault: disable)]
  • You cannot build a binary work correctly from the source configured with --enable-message-checker

Equivalent of IBM Spectrum Archive SDE 2.4.0.0 (10022)

LTFS original code is moved to GitHub!

From the next release IBM will fetch the code from here, make their changes for IBM Spectrum Archive SDE, build the binary and release in FixCentral. Only binary is uploaded to FixCentral.

Change log:

• Supported LTFS format specifications 2.4.0.
• Problem fixes reported in the previous versions.