s2: Read skippable block at the beginning of the stream

[Jille]

Thanks for the great library Klaus :)

I'm designing to have a skippable block at the beginning of the compressed S2 stream. The block will contain information about what's in the actual data (my own index format).

It would be nice if calling Reader.SkippableCB() would immediately have a peek at the first block to see if that is a skippable block. Or is there any other way to do this that I've missed?

[klauspost]

In general the Reader does no do any "ahead-of-time" reading.

However, it seems like doing a 0 byte read like this (*Reader).Read([]byte{}) should trigger it to read ahead and should return your skippable block.

If that works, I wouldn't mind making this "official API" and document/test this behavior.

[Jille]

Thanks for the quick reply. That'd work for me. I've written a test, but sadly it fails, even after #916.

func TestLeadingSkippableBlock(t *testing.T) {
  var buf bytes.Buffer
  w := NewWriter(&buf)
  if err := w.AddSkippableBlock(0x80, []byte("skippable block")); err != nil {
    t.Fatalf("w.AddSkippableBlock: %v", err)
  }
  if _, err := w.Write([]byte("some data")); err != nil {
    t.Fatalf("w.Write: %v", err)
  }
  if err := w.Close(); err != nil {
    t.Fatalf("w.Close: %v", err)
  }
  r := NewReader(&buf)
  var sb []byte
  r.SkippableCB(0x80, func(sr io.Reader) error {
    var err error
    sb, err = io.ReadAll(sr)
    return err
  })
  if _, err := r.Read(make([]byte, 1)); err != nil {
    t.Errorf("one byte read failed: %v", err)
  }
  if !bytes.Equal(sb, []byte("skippable block")) {
    t.Errorf("didn't get correct data from skippable block: %q", string(sb))
  }
  data, err := io.ReadAll(r)
  if err != nil {
    t.Fatalf("r.Read: %v", err)
  }
  if !bytes.Equal(data, []byte("ome data")) {
    t.Errorf("didn't get correct compressed data: %q", string(data))
  }
}

=== RUN   TestLeadingSkippableBlock
    reader_test.go: one byte read failed: s2: corrupt input
    reader_test.go: didn't get correct data from skippable block: "skippable block\x01\r\x00\x00"
    reader_test.go: r.Read: s2: corrupt input
--- FAIL: TestLeadingSkippableBlock (0.00s)

[Jille]

Found it: #917

[Jille]

And #918 to add the test and documentation update.

[klauspost]

Awesome - added this mostly at a user request - guess they didn't need it after all ;)

Anyway - adding the changes.

I will probably also do a release shortly.