Large scale GraphQL APIs

[jakerobinson]

Describe the bug

I'm working with a GraphQL API with over 8000 types. Here is the storage used after generation:

6.4G	docs/types
1.2M	docs/introduction
1.0G	docs/mutations
1.2G	docs/queries
272K	docs/_app/immutable/nodes
 36K    docs/_app/immutable/entry                                                                     
4.5M	docs/_app/immutable/chunks
1.5M	docs/_app/immutable/assets
6.3M	docs/_app/immutable
6.3M	docs/_app
8.6G	docs

Each type html file comes in at around 1.2M. I believe this is due to the side bar being statically generated for every page. I'd like for Magidoc to only generate the side bar once and include it in each type page. This would theoretically cut down on generation time, memory resources, page load time, and overall storage usage.

Reproduction

Schema available upon request.

Logs

No response

System Info

System:
    OS: macOS 13.5
    CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
    Memory: 846.94 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.3.0 - /usr/local/bin/node
    npm: 9.6.7 - /usr/local/bin/npm
    pnpm: 8.6.12 - /usr/local/bin/pnpm
  Browsers:
    Chrome: 116.0.5845.110
    Safari: 16.6

### Severity

Serious, but I can work around it

[pelletier197]

Hi, thanks for raising this issue. You're right, this is a big problem. However, I'm not sure if what you propose is possible. The template uses svelte (svelte-kit), and since we want to output static html to make this easier and cheaper to deploy, it's more or less possible to have the page request a part of the HTML like it's possible to do with SSR. SSR has its advantages and its disadvantages as well.

Maybe a possible solution would be to completely exclude the types from the navbar ? This could be an option provided in order to skip it. To be fair, I can hardly see a world where someone scrolls through an 8000 elements list anyway.

It would still be possible to navigate to types through search or by navigating the queries, but not through the navbar.

What do you think ?

[jakerobinson]

it's more or less possible to have the page request a part of the HTML like it's possible to do with SSR

Would it not be possible to generate the side bar as it's own static html and include it with client-side javascript? That's a real question, I'm totally ignorant. :) Otherwise, I agree that it's probably best that an option to not include the sidebar is probably going to be best for performance.

I can hardly see a world where someone scrolls through an 8000 elements list anyway.

The challenge for GraphQL API consumers is that it may not be evident what the name of a specific query or type is, so they still need to browse a little. I've seen some solutions handle this by including a text box to do a little filtering, if they have a small sense of what they are looking for. The magicdoc search function does that, but brings up a different modal and doesn't appear to give all results, only the first 10 or so. Probably a separate issue I can go into more detail on.

[andrew-demb]

Optional enabling reusing partial content (sidebar) can make deploy process simpler - instead of changing all files in case of added new types - only one file (sidebar) changed and added new files for new types.

I can suggest two ways to handle:

• client javascript to render sidebar
• SSI on webserver. Just like this https://www.nginx.com/resources/wiki/start/topics/examples/dynamic_ssi/