should Rspack support typescript | superset javascript as first class?

[hardfist]

Currently, in rspack, the default behavior is to enable SWC transpilation for all modules. which brings some benefits to rspack

• provide out of box experience and users no need to configure loaders
• performant friendly, no need to do double parse like loaders

But this also leads to the following issues:

• It is not possible to disable SWC transpilation for specific modules (e.g., corejs may have issues after SWC transpilation).
• Configuring SWC options at the module level is difficult since we use global configuration for swc transform options

We have addressed these issues by implementing the builtin:swc-loader. However, there are still some remaining problems:

• We have already enabled SWC transpilation by default in rspack. If users configure swc-loader at their level, it can result in double transpilation. Therefore, we need a way to support disabling the built-in SWC transpilation.

There are several possible solutions to this problem, we should make sure no breaking changes are introduced in the patch version:

• Disable the default SWC transpilation using a custom type, such as type: 'javascript/no-swc'. When this option is configured, JavaScript modules will not undergo SWC transpilation by default.
• Use a global switch to disable this transpilation behavior by default, such as futureDefaults. When futureDefaults is enabled, we disable the default transpilation behavior for JavaScript/{auto,cjs,esm}, ts, etc. Otherwise, it is enabled by default. In the next major version, we will disable this transpilation behavior by default

In the long term, we need to decide whether to keep the features like type: typescript or type: javascript/{auto,cjs,esm} + swc. Currently, both options have their pros and cons:

• support javascript/swc-{auto/cjs/esm:type: javascript/swc-auto is more user-friendly than configuring a loader. but this is not consistent with webpack and it still has the problem of not supporting custom options.
• not support javascript/swc-{auto/cjs/esm: user has to configure loaders for typescript and javascript module but this is consistent with webpack.

we discussed internally and we all agree DX friendly and webpack consistent are both very important for rspack. so we come along with another way

so we think of a new way to both keep DX friendly and consistent with webpack.
since webpack begin to introduce preset webpack/webpack#17171, preset may solve our problem

• we only support type that is supported in webpack, which means we wouldn't support type like ts tsx or javascript/swc-auto
• Introduce preset like react typescript and so on
• via preset or other switches: preset: ['ts']. for example

// file=preset-react.js
module.exports = {
defaultRules: [
 {
    test: /.ts/,
    use: ['swc-loader', {
      react: 'xxx'
    }]
}]

// file=rspack.config.js
preset: [preset_react(reactOptions)]

but there're some drawbacks about this method

• should we distribute preset along with rspack or ship seperate packages for it( which hurts DX)
• the preset is not highly composable like module.rules which may hurt DX in complex application

[hardfist]

@alexander-akait @ScriptedAlchemy @octogonz we need your valuable advice here, thanks

[alexander-akait]

Yeah, I see and undestand the problem

We have addressed these issues by implementing the builtin:swc-loader. However, there are still some remaining problems:

We have already enabled SWC transpilation by default in rspack. If users configure swc-loader at their level, it can result in double transpilation. Therefore, we need a way to support disabling the built-in SWC transpilation.

Technically inside the loader you have a _module and buildInfo (maybe we can even implement an public API on the loader level that allow to store some infomation and so loader can pass some information for other loaders, but we need to think about it more, also cache works great with it). But yes, our problem is that after the transformation we do not have ranges, so it's not entirely possible to avoid double parsing at the moment.

I would even say that this feature would be useful not only in the current context. I see enough others and its uses.

Disable the default SWC transpilation using a custom type, such as type: 'javascript/no-swc'. When this option is configured, JavaScript modules will not undergo SWC transpilation by default.

support javascript/swc-{auto/cjs/esm:type: javascript/auto + esm is more user-friendly than configuring a loader. but this is not consistent with webpack and it still has the problem of not supporting custom options.

I just recommend to be align with ESM/CommonJS/etc, i.e. javascript/esm+swc (or javascript/esm+no-swc, same for auto/dynamic), because swc can output ES modules and CommonJS (and other), so if developer will use javascript/no-swc/javascript/no-swc we can't undestand is it pure ESM module or not and so some things may not be optimized and some things may be broken - there is an interop example - webpack/webpack#17288 and you can see different output between javascript/auto and javascript/esm.

javascript/auto + esm is a bad idea, because interop will be broken (especial default export ...;)

In the long term, we need to decide whether to keep the features like type: typescript or type: javascript/{auto,cjs,esm} + swc. Currently, both options have their pros and cons:

so we think of a new way to both keep DX friendly and consistent with webpack.
since webpack begin to introduce preset webpack/webpack#17171, preset may solve our problem

• we only support type that is supported in webpack, which means we wouldn't support type like ts tsx or javascript/swc-auto
• Introduce preset like react typescript and so on
• via preset or other switches: preset: ['ts']. for example

My opinion is that we need both things - and presets and allow to implement own types.

Because presets can encapsulate more logic - custom extra loaders/plugins and options, for example you can have the react preset + swc (and it can be used with custom options for swc and with built-in options) and the react preset + babel (or even surcase or other things). Therefore, it seems to me that we need to implement presets in any case. Yes, this will slightly increase time to maintance them, but improve DX. Presets can even be used within the same configuration, for example presets: [sassPreset(options), swcPreset(options), polyfillNodeGlobalsPreset(options)], it is just a small blocks of our configurations which will be merge into the one configuration.

It's also a good idea to declare your own types. There may be more in the future, so supporting custom types is a pretty good idea, not only for javascript For example what if we want to change our CSS modules spec (new keywords or we need to change current logic - I even can provide an example for some cases 😄 ) - without this feature we will need to do breaking change or implement global options to change logic, it means bad and hard migration (you need to enable it globally or disable), but with types we can implement css/modules-next (or css/modules-v2), same here for swc - javascript/auto+swc-next (for the next major release of swc). With it we don't need to do extra releases with breaking changes and allow our developers migrate smoothly, I see only wins here.

There is a good question here - how should we behave by default? Now webpack behaves as compatible as possible - i.e. no transpiling/tranformations, only pure Javascript/CSS/etc by spec. Yes, unfortunately this brings some issues to the DX, but this allows webpack to be more stable - only stage 4, only supported by browsers things, no extra magic options under the hood. Unfortunately specifications are subject to change, somtimes it was on stage 3 and then return to stage 2 (look at import ... from ... assets {} syntax). swc (you can change it on babel/etc, this is more of a general conversation) can also have a change, so if we rely on this, we can get a precedent - it was just a bug for swc, but it is a breaking change for us (webpack/rspack).

We also should not forget that the concept of loaders was created specifically not to bring everything into the core, so we should not forget about other things (for example mdx), in fact they should have the same capabilities and API. Also, a bloated kernel can negatively affect the maintenance of the project as a whole and extra logic (so that in a year not to return to the question of who wrote it and how it works 😄 ), we shouldn't forget about this.

[hardfist]

sorry auto+esm is typo what i mean is auto+swc

[hardfist]

we have some concerns about declare rspack own types,

• custom type comes with custom options, if we add custom type we may also add custom parser and generator options and how these options interop with current javascript module parser | generator options ?
• if rspack add custom type, will it conflict with wepback's type in the future ?
• now we have type: javascript/auto+swc and swc-loader to handle module transformation, will it conflict in some edge case, what if user set these options together?

[hardfist]

opps, if we support auto+swc for typescript we may also need to deal with webpack/webpack#17288

[alexander-akait]

custom type comes with custom options, if we add custom type we may also add custom parser and generator options and how these options interop with current javascript module parser | generator options ?

Technically new types are just supertypes of javascript/auto, javascript/dynamic and javascript/esm, that is why I suggest to use javascript/type+something naming (i.e. javascript/esm+swc), this will make it easy to determine what type it is. We need to test it (and maybe improve on the webpack side too) - i.e. try to create a small own type, test can reuse logic from original javascript parser to avoid extra work and look how it will be communicate with our code (honestly I can't give an answer right now because it will take some time to research it, but I'm happy to help), but I consider it as an enhancement and will allow us be more modular

if rspack add custom type, will it conflict with wepback's type in the future ?

Yes, but we can improve error message on both sides, for example if you have javascript/esm+swc we can generate a helpful message - Sorry, "javascript/esm+swc" is not supported, please use "javascript/esm" or maybe it can be just a warning and we will use javascript/esm

now we have type: javascript/auto+swc and swc-loader to handle module transformation, will it conflict in some edge case, what if user set these options together?

There can be two situations:

• The user really wanted it, for example developer has your own plugin for swc to handle something exotic and want to transform it. but want to use built-in transformation, so it's not a mistake, using 2 and 3 or more loaders is normal
• The user makes a configuration error and setup loader and setup type.

Since we can't know the behavior for sure and the autodetector is not really possible then I guess we can implement API for buildInfo on the loader level, i.e. swc-loader set transformedBySwc to true (maybe better name) in buildInfo on the swc-loader side, and rspack check this and emit an warning about it, to hide this warning developer should set { loader: "swc-loader", options: { force: true } } (and swc-loader will set another value for transformedBySwc).

Shortly - transformedBySwc can be undefined | true | "force" and rspack emit a warning when type: "javscript/auto+swc" and devloper run swc-loader without the force option. So I see such a solution, because it will solve all problems, unfortunately for exotic situations you have to add an option for the swc-loader, but it seems to me that people who set up such things have more experience and know what they do

[Boshen]

Therefore, we need a way to support disabling the built-in SWC transpilation.

Do we really need to detect this automatically? Are we allowed to throw a warning if we are double transpiling? Auto detecting user intention doesn't sound right to me.

[hardfist]

Do we really need to detect this automatically? Are we allowed to throw a warning if we are double transpiling? Auto detecting user intention doesn't sound right to me.

we don't do automatically detect here, it's that we enable swc transpilation by default now and we need to disable this behavior when we want people use builtin:swc-loader to avoid double transpiling

[TheLarkInn]

I'm +1 for the approach here: align with webpack, make a preset, and ship preset along side.

but there're some drawbacks about this method

should we distribute preset along with rspack or ship seperate packages for it( which hurts DX)
the preset is not highly composable like module.rules which may hurt DX in complex application

I should sync with @alexander-akait on this design but we could have some core presets living in webpack core:

import {presets} from 'webpack';

module.exports = {
  presets: [presets.ts(), ...]
}

I think in comparison to what we have done in the past (which is just let users figure out how to use these separately maintained packages), that we own the recommended approach for many of these core experiences (react, ts, vue, css?, assets) and provide them to users automatically.

Thoughts?

[alexander-akait]

Yeah, a good idea and a good catch, so consumer will be webpack or rspack itself (not string with versions), so yes, also it will solve a problem with transitive dependecies, when you have multiple versions of webpack (or rspack) and so different API

[alexander-akait]

Updated my proposal

[hardfist]

talking about webpack.xxxPlugin, we find there're two ways for plugin author to import webpack.xxxPlugin, by webpack.xxxPlugin or by require('webpack/lib/xxxPlugin'), I'm not sure it's by design or by accident, and we're starting to implement these webpack's internal plugins and not sure we should be consistent with these patterns.

[hardfist]

the overall design seems good enough, but it seems that the ecosystem are heavilly using webpack-chain, I'm not sure whether we should support modify config in webpack-chain way, @chenjiahan may have some advice here

[alexander-akait]

talking about webpack.xxxPlugin, we find there're two ways for plugin author to import webpack.xxxPlugin, by webpack.xxxPlugin or by require('webpack/lib/xxxPlugin'), I'm not sure it's by design or by accident, and we're starting to implement these webpack's internal plugins and not sure we should be consistent with these patterns.

Yeah, require('webpack/lib/xxxPlugin') is a legacy way and should be avoided, webpack.PluginName is right usage, if we don't export something we should improve it and start to export. Global require("webpack").PluginName is dangerous, especial when you have multiple version with breaking changes, we faced with a lot of problems between webpack v4 and webpack v5.

the overall design seems good enough, but it seems that the ecosystem are heavilly using webpack-chain, I'm not sure whether we should support modify config in webpack-chain way, @chenjiahan may have some advice here

there is also webpack-merge, but I think our goal is to replace them and make our DX is better without extra tools and with simple logic

[hardfist]

Perhaps something like "webpack-presets" package? Which has these preconfigured? What do you think about this?

this seems more ideal and vite use this way

[hardfist]

btw does webpack have plan to maintain webpack and webpack-cli in monorepo，it is not easy to know the corresponding version between webpack and webpack-cli，and once we have presets it seems getting worse？

[alexander-akait]

I think not yet, it's a bit difficult because we didn't do it in the first place.
I don't think this will create a problem with presets because we will support them at the webpack level, so if you have webpack that support presets it will work.

I think it's worth mentioning here - now webpack webpack init is not good supported, I mean it is very limited and doesn't support a lot of cases, presets can help to solve it, because webpack init or rspack init will generate just pure webpack.config.js/rspack.config.js and ask about presets, but because we will have many of them (in future and if we union here), we will not have such problems and have wide support for many cases. At least it helps a lot for beginners well.

[hardfist]

yes, webpack init with preset template is a good combo, vite's has rich template which is super friendly to beginners