Polyfills

[thheller]

$jscomp related Polyfills have been a problematic area in shadow-cljs for a while now. The problems are somewhat evasive and hard to reproduce properly. I would like to fix this at some point but I'm not actually sure how.

The problem is somewhat in the Closure Compiler or rather how shadow-cljs uses it. CLJS itself mostly doesn't need polyfills and is not much affected by this but node_modules code often needs and uses them. I'm trying to avoid the situation where polyfills are injected by babel (only used on some JS code) and then maybe again by Closure.

shadow-cljs will often run the compiler incrementally on node_modules code (eg. when you add a new (:require ["some-package"]) it won't recompile the ones it already has). It tries to collect polyfill information and maintain a set of used polyfills which it then generates at a later separate stage in the build. This seems to work most of the time but somehow doesn't sometimes. I don't know if its just lacking information from the Closure Compiler or shadow-cljs not collecting/maintaining it properly. Closure itself doesn't have this issue since it doesn't compile incrementally and is meant to receive all code at once when doing the polyfilling. In shadow-cljs however the node_modules code is part of a different compilation than the CLJS :advanced parts. Things seem to be OK when only the node_modules code uses polyfills but sometimes things appear to go wrong when using polyfill-related code in CLJS or classpath-js.

I do however feel that Polyfills are somewhat overused currently because of the default :output-feature-set :es5. Most problems would go away if this was bumped to :es6 which is very widely supported at this time. Pretty much all problems would go away when bumping this so :es8.

Things would be much easier when only doing polyfills for release builds. Or maybe I change my strategy to do polyfilling as a separate build step and not incrementally. Maybe that works better.

I wonder how many of you actually need/want polyfills while developing? How many people still have to tolerate IE as a runtime target? Without IE support :es8 would be mostly fine given that all other browsers have supported these features for years now.

[gmp26]

I find I need polyfills to support IE9 in production for NHS applications in the UK. It may soon be possible to increase the minimum requirement to IE11. I don't yet know what will be needed there.

[thheller]

I'd expect that you also run the watch or compile output with IE9? Or do you just test locally with modern browser and release with the requirement of that working in IE9.

[mk]

@thheller Thanks for starting this discussion. So you're saying if we bump the :output-feature-set to :es8 (we're already using :es6) in dev only we should be good? Is that still true if we cache .shadow-cljs in ci for the release build?

Btw if you're looking to reproduce the issue, it should be pretty easy in our repo and I have the feeling the errors started showing up after we brought in https://github.com/d3/d3-require.

On your question of needing polyfills for dev, it wouldn't be a big issue for us if they went away or if it was a more manual step like testing out an advanced build.

[thheller]

Bumping to :es8 will eliminate all polyfills that were previously required. Most prominently that would be async/await code. There might be ES2020 or so code that would still require polyfills but this is still kinda rare. It's an uphill battle trying to keep up with all the stuff being added to JS these days. You can set :es-next which will never polyfill anything. The Closure Compiler may not support the absolute bleeding edge of some JS stuff but most "standard" stuff is supported.

It is actually better to never polyfill anything if you know your clients can support it. Not only is the code smaller it may even have better performance. In the JS world it is now considered best-practice to have one "modern" bundle and a fallback polyfilled bundle for "older" clients. If you use a lot of npm packages with "modern" that may something that may be worth considering too. Example:

;; modern bundle with :es8 or so configured (sufficient for modern browsers)
shadow-cljs release app
;; same build into separate folder with polyfills for older browsers
shadow-cljs release app --config-merge '{:output-dir "public/old" :compiler-options {:output-feature-set :es5}}'

This won't do much for most CLJS builds but it is an option. The hard part is figuring out how to "detect" which bundle should be used in the HTML.

My stance on polyfilling has always been that development should match release as much as possible. As I want to avoid stuff working in development but not release, or rather to break as early as possible. Luckily I don't use any npm packages for my work projects so I don't have to worry about this anymore as polyfills are mostly a non-issue for CLJS.

Of course this still should work properly for everyone but it is really difficult to track down since no reports have been reproducible yet. :(