Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use wuffs for fast & secure ZIP/LZW decoding/encoding #186

Open
syoyo opened this issue Dec 20, 2022 · 3 comments
Open

Use wuffs for fast & secure ZIP/LZW decoding/encoding #186

syoyo opened this issue Dec 20, 2022 · 3 comments

Comments

@syoyo
Copy link
Owner

syoyo commented Dec 20, 2022

Describe the issue

wuffs is a fast, secure and portable algorithm library(zip, huffman decoding for LZW, etc)

https://github.com/google/wuffs

Currently TinyEXR uses codes from OpenEXR, which is not optimized, difficult to interpret code(hard to apply fixes found by fuzzers), and also not secure(although various fixes already added in TinyEXR thanks to fuzzer)

replacing such code part with wuffs would make TinyEXR more faster, secure and portable.

This change also open the possiblity to relicense TinyEXR to MIT or Apache 2.0 by removing OpenEXR code part.
(We also need to rewrite PIZ and other pixel encodings though)

To Reproduce

N/A

Expected behavior

TinyEXR becode faster, secure and portable.

Environment

N/A

@meshula
Copy link

meshula commented Sep 24, 2023

In case it is of interest, we recently switched OpenEXR from using zlib, to using libdeflate.

A nice aspect of libdeflate is that we can use it as a pseudo-single-file library; in fact the same is true for the OpenEXRCore C core.

https://github.com/PixarAnimationStudios/OpenUSD/blob/release/pxr/imaging/hio/OpenEXR/OpenEXRCoreUnity.h

The interesting bit is that you can incorporate the sources of libdeflate as follows:

#include "deflate/lib/lib_common.h"
#include "deflate/common_defs.h"
#include "deflate/lib/utils.c"
#include "deflate/lib/arm/cpu_features.c"
#include "deflate/lib/x86/cpu_features.c"
#include "deflate/lib/deflate_compress.c"
#undef BITBUF_NBITS
#include "deflate/lib/deflate_decompress.c"
#include "deflate/lib/adler32.c"
#include "deflate/lib/zlib_compress.c"
#include "deflate/lib/zlib_decompress.c"

and thus eliminate the need to have a separate library build and link.

@syoyo
Copy link
Owner Author

syoyo commented Sep 25, 2023

hoho,

I have forget to mention to this issue. TinyEXR now started to add nanozlib(based on wuffs) for ZIP decoding.

9bad737

@meshula
Copy link

meshula commented Sep 25, 2023

Ah, cool :) I thought it was still under investigation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants