Store: we use a dictionary instead of an array

[lionel-alves]

I have been using Codable for easy persistance for a while, I really like what you have done here!

I was wondering if you considered using a dictionary instead of an array in the Store?
We would be able to access an item from the cacheIdentifier without having to filter the array, which can be very nice if we work on very big data set.

I guess the best would be to be able to choose the data structure depending of the use case. A Set might be useful as well for example.

[mergesort]

Hey @lionel-alves, first off thanks for the kind words. Your suggestion is great, I've actually been thinking about doing pretty much exactly this in fact, with only a very small difference! I wanted to release Boutique before doing this knowing this is an optimization that wouldn't be exposed to users, so it wouldn't be an API breaking change. I also don't expect it to be a huge performance problem initially, you would only start seeing benefits when users start using it with some scale, especially given how optimized Array is in Swift.

Now for the small difference. Maintaining order is important so that rules out using a Dictionary and a Set, but luckily the Swift Collections package provides us with an OrderedDictionary that does just that! And that's what I plan to do soon to avoid the problem you describe (especially since filtering the array is a quadratic operation. 🫠)

How does that sound to you? 🙂

[subdigital]

Agree that order is important, but I think we already lose that, as we save these records in order, they get a hashed filename, and when Bodega reads in the files it does this in filename order, which is different.

(unless I'm missing a mechanism for influencing the filename to induce a natural order)

[mergesort]

That's true, when it's read from disk it's loaded into the array without a specified order. There is one exception to the expectation of it being unsorted though, when you call .add(item) the item is appended to the end of the array. I do this because I think it's a reasonable expectation when you add something to a list it goes to the end of the list. I haven't decided if it's an important consideration because if order matters to the developer then they're going to apply their own item sorting mechanism somewhere, likely in the View, Controller, or other domain object that's interacting with the data.

Do you have any thoughts about that @subdigital?

[subdigital]

Yes order matters, but since my data is returned in the right order, and the in-memory array shows it correctly, it was surprising that I got a different order on subsequent launch as it was read from the cache.

You're right that I should be applying a sort, but this seems potentially costly with large arrays.

Supplying a sort parameter in the filename would be nice, but this does lead to issues that if the filename changes (due to dependent data changing) then you'd end up with no way to delete that old file. This alone is probably the biggest reason not to allow touching the filenames.

Sounds like we need an index 😄 🙈

[lionel-alves]

That's is more related to Bodega, but I will ask here if that's ok. From what @subdigital is saying, I understand that each item of the array is stored individually in a file. Could you explain what are the benefits of that compare to writing the whole array in one file (which would prevent the order on disk)? Thanks!