Structurize, update and extend the documentation

[tnozicka]

Description of your changes:
This PR isn't fixing all the docs issues we have but it's an attempt to:

• make the documentation hierarchical
• fix or remove the pages that are incorrect
• fix the outdated steps that don't apply anymore
• describe the overall architecture
• describe the components and install flow
• describe the GitOps flow (although that could be simplified by publishing the manifests e.g. to GH releases - this just describes how it's done today)
• describe at least the basics of how storage is set up
• avoid requiring users to clone the operator repository
• gets rid of using the unsupported demo scrips (we don't want to be in the business of teaching people Kubernetes set up or automation), although we have retained the simple set up idea with quickstarts that have just a single path to start with
• add a few diagrams
• slightly better UX for copy pasting and other small things here and there
• and some more that I could fit into the limited cycles I had allocated

Which issue is resolved by this Pull Request:
Resolves #1578 (and more)

Requires

• Allow updating Sphinx-Substitution-Extensions and update dependencies sphinx-scylladb-theme#1295

[tnozicka]

/hold
to test the flows before merging it (if we agree that this is how we do it in review)

[rzetelskik]

It fails building locally for me

/go/github.com/scylladb/scylla-operator/docs/source/resources/scylladbmonitorings.md:7: WARNING: Include file '/go/github.com/scylladb/scylla-operator/examples/monitoring/v1alpha1/scylladbmonitoring.yaml' not found or reading it failed [docutils]
...
sphinx-sitemap: sitemap.xml was generated for URL https://operator.docs.scylladb.com in /go/github.com/scylladb/scylla-operator/docs/_build/dirhtml/sitemap.xml
build finished with problems, 1 warning (with warnings treated as errors).
make: *** [Makefile:37: dirhtml] Error 1

command in README-dev.md should be updated with -v="$( pwd )/examples:/go/$( go list -m )/examples:Z"

preferably to:

podman run -it --pull=Always --rm -v="$( pwd )/docs:/go/$( go list -m )/docs:Z" -v="$( pwd )/examples:/go/$( go list -m )/examples:Z"  --workdir="/go/$( go list -m )/docs" -p 5500:5500 quay.io/scylladb/scylla-operator-images:poetry-1.8 bash -euExo pipefail -O inherit_errexit -c 'poetry install && make --warn-undefined-variables dirhtml SPHINXOPTS="--keep-going" && make --warn-undefined-variables redirects && make preview'

to make it closer to what we do in CI

[zimnx]

lgtm
/assign rzetelskik

[rzetelskik]

two conversations remaining, rest lgtm

[rzetelskik]

/lgtm
thanks for the updates

[tnozicka]

I am testing the quickstarts - pushed a few fixes as well as using a bit smaller machines so it gets through with default GKE quotas (that I've also hit).

[tnozicka]

ok, the GKE went fine, I need to try EKS again next week (it failed spinning up today)

[tnozicka]

well, the EKS got as far as expected given EKS is broken with current master

    - lastTransitionTime: "2024-11-25T09:38:14Z"
      message: 'can''t create mounts: can''t ensure units: can''t get unit statuses:
        can''t list units by names "mnt-persistent\\x2dvolumes.mount": Unknown method
        ''ListUnitsByNames'' or interface ''org.freedesktop.systemd1.Manager''.'

#2180

/hold cancel

[rzetelskik]

/lgtm

[scylla-operator-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rzetelskik, tnozicka, zimnx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tnozicka,zimnx]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment