Feature Request/Discussion: Multipaged slate support

[E1337Kat]

Sometimes our APIs can become quite... large. Slate is beautiful, but when large enough the page starts having issues loading or keeping up with scrolling. The seemingly simple solution to this is to allow the documentation to be broken off between multiple pages and even nesting folders. Some work has already been done towards this effort, but no major changes have come out of it.

I know that issue #345 exists, but I think a new ticket to properly outline how this feature should look and work would be a good idea. In #994 I gave a go at trying to get this feature to work. I should have made this ticket first to give it a go here. Since I put in that PR, I actually had made some further changes which touched upon some of @lord's comments in #994. The branch with these changes are here.

---

Anyways, this is a discussion to decide how to go about solving 4 key issues:

• How should Resources and Requests be organized?
• How can we reconcile the ToC so that the sidebar links correctly
• How can search work across different pages with minimal effort
• All this should be accomplished with minimal effort on the end user.

---

• I would propose the solution for the first point be that we have each resource in it's own folder and all the requests for a given resource be listed on that page. I had planned on each request being a page and the folder would be the resource, but in practice this does not work out well.
• For the ToC, I do not have too much to add. I know some changes would need to be added to the toc_data.rb file, but I am unsure how much needs to be changed. In my current attempt to work at this, I have an issue where the ToC has all of the resources listed, but the "dropdown" doesn't work correctly.
• Search is a difficult one. As @lord mentioned in Add support for viewing resources on multiple pages (backwards compatible) #994, either some sort of server side code is needed to allow the search to work across pages, or the search implementation needs to be swapped out completely. Server side is difficult as many people likely host docs on Github, or they are static pages in the public html folder for a server. This is second to ToC working.
• Currently with Slate, you add your file, type in some commands, and done. We should aim to keep this as simple as possible. Adding some end user work for the feature is ok, but not as much work as my first implementation counts on. By making the changes fully backwards compatible, then only people specifically interested in using the feature will have to make any additional changes.

---

I look forward to any and all thoughts regarding this. 🙂

[musikele]

How can search work across different pages with minimal effort

On jeykll I created, at compile time, a posts.json containing all titles, post texts, and all relevant data. Search consisted in a textual search inside this document. Can we do the same with multiple pages?

[E1337Kat]

Hmmm. That might could, or something similar. The issue arises again if the documentation becomes too large, this file could too become a bit large maybe. I suppose the file would need to include the link to the appropriate page and any keywords associated with that page. There is a way with Middleman to have a global data file that can be accessed by all templates being generated, so that might could work.

I'll delve into the current search and look into other search algorithms to see what I can find. Avoiding backend work is fairly important to me, I think

[lord]

Some thoughts:

• I've seen some open source projects use Algolia's free tier to do search, a possible solution is to add in optional hooks for that if people want search across multiple pages?
• Some people have wanted a version switcher similar to that of read the docs (see the green v:latest in the bottom left corner), which would be much simpler to implement than actually putting things into the table of contents. And if we allow any arbitrary pages to appear in the dropdown, it could be used for anything, including breaking up super long pages into logical chunks.
• If the problem is purely load/scroll performance of large documentation pages, I'm sure there are some workarounds we could put in to improve this, while preserving nice single page docs. Stripe's docs (which we originally copied the design from way back when) have gotten very, very long, but they have some dynamic loading stuff that seems to keep things in check.

[E1337Kat]

So I have been working on a project heavily based on slate, but decoupling slate from middleman so that it could be it's own standalone ruby app. The only issues that I run into with it is that search only uses the current page (just like current slate implementation) and an issue where the toc headers do not style correctly when using multiple pages (that I should be able to solve).

In decoupling I think I might have managed to make set up even easier with a simple 5 steps:

1. "copy and paste templates into the template folder"
2. "check that config/data.yml has wanted options"
3. "bundle install"
4. "rails s thin"
5. done.

This is not backwards compatible without some work on the end users part, but is doable.

While this is a silly approach if you have only one page documentation, it works well for multipaged documentation, and is a great starting point for fixing search for multiple pages issue, maybe. This might also make a good playground to look at the version switcher. Further, I think that perhaps Stripe's docs with a single page is more likely to be implemented in such a way. Using a full rails app, you could ajaxily load new pages as you scroll. Admittedly, I'm not sure how well you could manage using just javascript.

If you wanna check the repro out, it is at https://github.com/E1337Kat/slate_server (I need to write the readme so that people know how to use it... This has been personal work so far, but I plan to implement it for my job so our docs can finally look nice.)

[E1337Kat]

I finally figured out how to make the ToC work with multiple pages when using the slate server build. I believe I should be able to make it work for the middleman built docs as well, but I have not tested this yet. I think I could use similar design principles for the search as well.

The basic solution is that when building the ToC, you use the new way of building it from the data files. When the builder gets to the current page, it hands over to the old way of building the ToC that lord created. This way the current page can utilize the beautiful ToC css, and you still get to show all the other resources. The only bug seems to be when someone reloads the page when not at the top of the page and having not clicked on a request to use.

I am gonna read up on some dynamic loading stuff in rails and see if I can figure it out. with the middleman process, I would think the dynamic loading would need to be JS based.... which does not sound so fun.