multi-track index file and loading

[nkundiushuti]

I have some doubts with the structure of phenicx anechoic. The dataset object in 0.3 changed a bit the way the data is loaded and it needs some changes to adapt to multi-track
The moment I have this structure in the index:

"beethoven": {
    "score-bassoon": [
      "annotations/beethoven/bassoon.txt",
      "57054d355fc2e9f49d0933a2e50a7b99"
    ],
    "score_original-bassoon": [
      "annotations/beethoven/bassoon_o.txt",
      "528135e3ec149e02f637c12f0eb40693"
    ],

for multi-tracks beethoven, bruckner, mozart, mahler with annotations and audio files.
And in the loader I have the Track and MultiTrack defined as in CONTRIBUTING docs.
This currently gives an error when calling dataset.load_tracks() because it tries to identify the track ids by the dictionary in which we have the multi-tracks as keys.

I remember seeing this in a conversation between @rabitt and @magdalenafuentes on the dataset object. Should we have a load_multitrack()? And then, to extrapolate, for the lastfm datasets a load_record() method or similar?
I can add the load_multitrack() to the core.py as a method in Dataset.

[rabitt]

For the index, we'd envisioned having something like:

{
  'version': 1,
  'tracks': {
     "beethoven-basson": {'annoation': ('path/to/file', checksum), 'audio': ...}
     "mahler-basson" : {...}
     ...
  },
  'multitracks': {
    "beethoven": {
         'tracks': ['beethoven-bassoon', 'beethoven-clarinet', ...],    # list of keys from 'tracks'
         'audio': ('beethoven/mix/path', checksum), 'melody': ('melody/annoation/path', checksum), ...  # any other mix level audio/annoations if applicable
     }
    "mahler": ...
  },
  'metadata': {
    'some-file-key': (path, checksum),  # move all the manually defined metadata paths from the loaders to the index with checksums
    ...
  },
  'records': {
      'some-record-key': (path, checksum)   # to eventually support record data
  }
}

Where everything under the top-level key "tracks" is identical to the current indexes. Obviously we'd need for #277 to be finalized.

This currently gives an error when calling dataset.load_tracks() because it tries to identify the track ids by the dictionary in which we have the multi-tracks as keys.

In the meantime, for the multitracks, what about manually defining just the multitrack_id --> list of track ids mapping in the loader as a global, which we can move to the index when #277 is done?

Should we have a load_multitrack()? And then, to extrapolate, for the lastfm datasets a load_record() method or similar?

Yes! that was our plan. I didn't implement load_multitrack yet but I was hoping to include it for Phenix-Anechoic as the first use case! And once we have record-based data we can add that as well.

I can add the load_multitrack() to the core.py as a method in Dataset.

Yes sure! I think we miiiight need #277 done to do it smoothly (to get all the mutltrack IDs from the index). If you can find a way to do it generally though, feel free to.