Move from rgba8 internally to rgba32f

[Quackdoc]

Currently oculante uses rgba8 internally. this is fine for a general purpose image viewer, but oculante has a lot of features that make it suitable for use in professional environments. however for a lot of those environments the internal 8bit handling makes this unusable. migrating to rgba32f will open the doors for a lot of workloads, Currently you can already use oculante to for getting specific RGB values. these values are always limited to 0-255.

one such workload is when verifying HDR mapping it's useful to be able to view the images RGB values, however for HDR 10bit+ is needed, and often working with 16bit images (thanks to uint16) which have the ranges of 0-1023 and 0-65535 respectively. and the rounding error can be quite significant. It is useful for many users to be working with 0-255. so it would be optimal to have a way to balance both of course.

working with high bitdepths internally opens the doors for a lot of use cases, from being necessary to HDR support to image verification.

[woelper]

Thanks! This is really high up on my wishlist too. Here are a couple of thoughts:

• The editing operations are already in float and are only reduced to 8 bit in the end. There should be almost no code change necessary in that area.
• rgba32 needs much more texture memory and system memory. Not sure if this is really problematic for the average user, but it could be significant for very large images.
• Relating to above, I've thought about adding two modes, 8 and 32 bit. This would of course make matters more complicated.
• This would potentially gamma issues like Incorrect gamma scaling #91

[woelper]

Wow, this is almost a year old now :) I still think this should be done, and I would love some help!

I've stated a new dynamicimage branch and got as far as getting it to compile with every image sent as DynamicImage.

I've updated the ImageExt trait in utils.rs to work for DynamicImage, but we would need to use actual texture settings to benefit from it. For example if we receive a greyscale image, we should create a greyscale image with the appropriate image format. Maybe this is something you could help with? If you don't have already, I can give you access to work on the branch directly.

I would try to make the current_image in the state also a DynamicImage and make sure image processing works with it. For that, I would probably convert the image into rgba float and back too keep it flexible and simple.

[B-LechCode]

Most of the image processing should be easily adaptable for varying channel sizes and bit depths. Converting to 4 channels and back would still pose memory shortage problems. But as a starter it's a perfect idea!

I will have a look at it and will come back to you when I have time :)

[woelper]

On some of the pixel operations this might be easier, but at least some libraries require rgba images. Even for pixel ops, there are a couple of optimisations, for example processing them in chunks of four for rgba to make use of SIMD extensions. Making them generic for every type of image buffer could end up being quite difficult.

[B-LechCode]

We could filter the operations depending on the type. If you would want operations requiring rgba images, you would need to place an "Convert to RGBA" operation before.

[woelper]

That is a good idea. It might also be practical to auto-convert rgb8 to rgba8 for editing. Many users might find it confusing if they load a jpeg and can't edit it. Some other takeaways from my progress in the branch so far:

• while we save memory on rgb8 or gray, we increase it for HDR images. The good stuff is that we gain more flexibility in the end.
• now, HDR images are tone mapped and converted to rgba8. Once we leave them as HDR, tone mapping should happen for the texture, maybe as a shader. If we don't do that, some images are not visible at all.
• the image channel selector should now show the actual channels present in the image
• now I pass the underlying mutable rgba buffer (if any) to the editing functions. Later, as you say, we need to branch if we want to reap the benefits or insert a to rgba8 operator.