Auto-fix Engine Grooming

[daniilsapa]

Here I quickly drafted my high-level vision of how the engine should behave. So, everything is criticizable and discussable, that is why the feature of creating discussions exists in the first place :) Feel free to complement and highlight/improve the bad parts.

Related issues: #20

Phase 1: Determine project specifics

There should be 2 ways to determine:

• provide explicitly in Steiger config
• infer from the codebase or/and other config files (tsconfig, package.json, ...)

Things to determine:

• Language JS vs TS
• Import/export style (CommonJS vs ESM)
• Aliases for folders/files (@/shared, @/entities, etc)
• ...

Phase 2: Regroup diagnostics

Right now fixes are grouped by rule and that can cause conflicts when one rule moves the file and then another fix tries to change imports inside it but tries to access it by the old path.

A possible solution can be regrouping the diagnostics by a single file/folder and then applying them in batch keeping the link to the file/folder

Phase 3: Apply fixes

The solution should behave in a way that only fixes that can be fully done without errors happening in the FS. Because I think we need to avoid situations where some fix is applied partially and then throws an error, but some changes are already done in the FS. I think these situations will 100% irritate the hell out of the users.

Here we can manipulate some kind of virtual file system and then when we are sure that a needed fix (or a sequence of fixes, e.g. "move" -> "rename" -> "fix import statements inside") is possible to apply without errors, we commit it to the real FS.

Phase 4: Notify

Besides the errors that cannot be auto-fixed, we have to explicitly notify the user about what was auto-fixed and where. If for some reason, an auto-fix fails, notify about that too

[illright]

ESLint

I'd like to add some information about how ESLint's auto-fix engine behaves — they avoid dealing with conflict resolution by applying fixes in 10 passes. If two fixes conflict during a pass, then only one of them is applied, and after that, the rule is run again to generate a new fix that incorporates the applied fix already. If some fixes are still unapplied after 10 passes, it gives up (though I've never seen it happen in real life).

We can use this approach ourselves too. And of course, we can still be smart about reordering fixes in such a way as to minimize conflicts.

[daniilsapa]

Interesting, I've never heard of that approach but it is viable in our case too. It looks like you've gone deep into ESLint because you have so many examples from it

[illright]

The format of fixes and conflict resolution

The modify fix is a bit too crude at the moment — it requires every rule writer to bring in their own language parser and printer, and it's also essentially incompatible with two fixes to the same file. This last problem is especially relevant, since it's not unusual for even just one rule to emit several fixes to the same file. Therefore, we might want to consider a different format for modify fixes, perhaps, one that works with the AST rather than raw file contents.

Alternatively, we can have an internal merge algorithm such as ones used by Git, to merge two fixes to a file (if they can be merged cleanly)

[daniilsapa]

Yes, good points for consideration.

Rules and policies can be different for different custom plugins but I think fixes are pretty static and fall under just several types like rename, move, add file, modify yet again, etc.

...it requires every rule writer to bring in their own language parser and printer...

But isn't the purpose of the rule simply to provide diagnostics and include data for fixes if possible (what to fix and where)? The rest is handled by the auto-fix engine built in Steiger.

Also, based on your idea of providing the users with a toolkit for creating their own rules (from the config structure discussion) cant that toolkit additionally provide something for fixes

[illright]

Right now, in order to create a modify fix, a rule must provide the entire new content of a file, which means that it has to generate that content somehow. The autofix engine, as it is right now, is not very helpful in this task.

We could solve this problem in a toolkit, or we could redesign the API of the autofix engine. I'm not sure yet what the pros and cons are for these two options

[illright]

Language servers

The most difficult fix to process might be rename, because ideally we'd want to also update imports on all the references to the file that we're renaming. Language servers (such as TypeScript's server) are already capable of that, but we need to run them, let them gather initial information about the project, send a rename command, and then verify the result. Not to mention that the frontend is quite diverse, so we would need to support JS/TS(X), Vue, Svelte as first priority, then potentially also Angular, Astro, Qwik, Elm, Marko, Dart, etc. Lucky for us, most of these languages have language servers already, and the Language Server Protocol provides a unified way of sending commands to them, but it's still going to be quite the integration

[daniilsapa]

This is also a valuable addition 👍