Improve Shardy documentation

[tomnatan30]

Shardy currently has the following WIP documentation:

1. A very basic README file
2. Auto generated MLIR docs from shardy/dialect/sdy/ir/*.td and shardy/dialect/sdy/transforms/*/passes.td.
3. An RFC that is a bit outdated.

Proposed documentation structure:

• README (should be improved to provide a good user facing entry point to the repo)
• Dialect / IR documentation (auto generated, can be improved).
• Transforms / passes documentation (auto generated, can be improved).
• Concepts: Sharding and Propagation - higher level understanding of how sharding / propagation works, and what compiler API we provide.
• Using Shardy from JAX (user journey, perhaps with a colab).
• Adding sharding to your compiler / dialect (or frontend framework like JAX).
• Tooling
• Debugging sharding annotations / propagation (follow up with internal users on what was useful in debugging).

The Dialect / IR documentation can be improved in general, but it's specifically missing the following:

1. Invariants of each operation / attribute, those are verified in the code, and the code comments there can be used as reference. The Invariants should be documented in a similar way to StableHLO spec.
2. Parameters/arguments of operations/attribute are sometimes missing a summary that populates the Description column in the auto generated doc.

More information will be added to this feature request later on.

[melissawm]

Now that #175 is live, there are a couple of follow-ups that need work:

• The Dialect / IR documentation (as stated above)
  • Add missing docstrings to dialect documentation #265
  • Add constraints to attrs.td and ops.td #275
  • Improve propagation passes docstrings #276
• How to register an automatic partitioner and trigger auto partitioning
• How to use Shardy with MyDialect
  • Dialect-agnostic sharding docs #271
• Improvements to Propagation doc: focus on data flow edge ops
  • Data flow edge ops documentation #263

Please add more as required.