Handling deletion of old/incompatible files when updating

[derrod]

OBS 26.1 has again shown an issue we've had experienced a few times before, where an update to OBS or its dependencies breaks older plugins and leads to failures that are not visible to users (silent crashes, etc.) and with no immediately obvious cause. Since OBS does not have any plugin manager we're currently limited to communicating this to users through changelogs and support requests. However a number of users is not aware of those warnings and will run into an issue where OBS simply appears to be no longer functional after updating.

Going forward it may therefore be a good idea to have an option to delete files as part of the update process that are known to cause issues or are simply not needed anymore. Of course users should be informed of that in case it affects their setup (e.g. plugins) but it should also be able to happen silently in cases where the users awareness is not required (e.g. old x264/libav DLLs).

One idea I've had to handle this is the ability of the update manifest to specify a list of files that should be deleted as part of the update. These entries would optionally contain a list of hashes of known bad versions and only delete those, rather than any file matching the specified path.

For example, entries could look like this:

[
  {
    "hashes": [
      "8aed2cfa661da80e0fad1c60ede1d1be30fd8c2831e303677d414b0da761d894"
    ],
    "name": "OBS Websocket Plugin",
    "path": "obs-plugins/64bit/obs-websocket.dll",
    "reason": "Incompatible with OBS XX.X",
    "silent": false
  },
  {
    "hashes": [],
    "name": "DeckLink Output",
    "path": "obs-plugins/64bit/decklink-ouput-ui.dll",
    "reason": "File was renamed",
    "silent": true
  }
]

When the updater process starts the installed files would be checked against this list, if there are any matches that are not marked as "silent" the user would be informed in a prompt that those files will be deleted when continuing and also be given the specfieid reason why.

An example warning could look like this (imagine a WinForms dialog):

= Plugin Incompatibilities =
The following files will be deleted when updating to OBS version XX.X due to known incompatibilities.
Make sure to reinstall the latest XX.X compatible versions of any plugins that will be removed as part of this process.

Files:
* obs-websocket.dll (OBS Websocket Plugin) - Incompatible with OBS XX.X

[Cancel] [Continue]

This idea is mainly for the built-in updater, so unfortunately a different setup would have to be created for the installer. I'm not familiar enough with how that works unfortunately.

[coral]

Could this approach also be built into a separate small executable which you run as a "pre-install" script as part of the installer. That way you could share this approach inside the auto updater and the installer.

Correct me if I'm wrong but this sort of looks like a denylist that would be appended with plugins that are known to be broken. My question regarding that is that how do you know prior to releasing what plugins has to go on the denylist? I assume there's some testing but could you instead host this file obsproject.com so the installer/auto-updater could get the latest version even though no new OBS release has to be deployed.

[WizardCM]

For reference, the built in updater on Windows downloads updates for itself first, then launches.

[coral]

@WizardCM That's cool but I don't think that 100% solves what I'm touching on. Let's say OBS 28.0.0 is released with a denylist and 2 weeks later you figure out that one plugin doesn't work and causes weird issues for users. Wouldn't you have to release 28.0.1 with an update to the denylist for the auto-updater to do it's thing, rather than just update the denylist and have the installer for 28 automatically get that version of the denylist.

Downsides to this approach is that it breaks the "standalone-ness" of the standalone installer.

[WizardCM]

Yeah I can definitely see a use case for a shared file. I recently experimented with an NSIS plugin that would allow for visible/silent downloads in our installer which we could then read/execute, and this may be a great use for it.

In terms of the "standalone-ness" of the installer, I think this'd be fine - it'd be an optional action that occurs if the user is online, and in most cases it'll only have to do this with a very small or even nonexistent list.

[coral]

Makes sense. The good part about this is that you can also rescind denying. For example if you have a plugin that relies on an external piece of software, let's say a driver and it breaks. The developer might fix the driver so the plugin still works, without updating the plugin which means you could "rescind" the deny. (although I agree this is unlikely to happen)

[derrod]

To clarify: the list would be part of the update manifest that the updater downloads when it is ran. So it can be changed at a moment's notice.

Moving it to a separate executable might be an option, that way it could be included in the normal installer (and could run offline if it can't fetch a new list). If we ever get around to adding some sort of "safe mode" to OBS it could also be an option to run it as a sort of "cleanup" script that removes known bad plugins. Unfortunately at this moment we do not have that, so if an installation is broken after the update it'll require manual fixing anyway.

Edit: although for the second case we could run the updater in a sort of "repair" mode instead. That's something we've been pondering in general to provide some way of verifying and redownloading OBS binaries without having to just re-run the installer.

[WizardCM]

Would it be worth also cleaning up secondary files? Some plugins come with dependencies that aren't in the main DLL, and in some cases may be worth removing too.

[coral]

Personally I think you should treat a plugin as a "package". If you clean up a package you clean the entire contents of that package, not just the main DLL.

[WizardCM]

Yeah, and it's one of the mistakes of the current plugin structure - as they're not in subdirectories, it's possible for one plugin to replace another plugin's dependencies.

[derrod]

Yeah I'd say we could add a list of other files a plugin ships with to delete. The remaining issue then would be overlap (e.g. two plugins shipping the same dependency, but only one is broken).