Create a custom 404 page with absolute URLs hardcoded.
pip install sphinx-notfound-page
Add this extension in your conf.py file as:
extensions = [
# ... other extensions here
'notfound.extension',
]Check out the full documentation at https://sphinx-notfound-page.readthedocs.io/
Strongly based on @ericholscher's solution from readthedocs/readthedocs.org#353