Static site-ify the main web site

[matiasilva]

Project/idea summary

Turn the main web site into a website generated by a static site using a technology like Hugo or Jekyll. This would involve creating a few custom layouts.

Requirements

• is easy to edit and access
• I can launch a dev server in one command

Motivation

The website is currently templated by Apache SSI. A technology that is old and unfamiliar to today's population. From personal experience, there is greater familiarity with static site generators like Gatsby, Hugo, Jekyll, and so on. If we take a look at this link, we can conclude that there is a much larger search volume for the above technologies in comparison to Apache SSI. Granted, we discard the results before 2008 but still a portion of these searches led to the learning of these platforms. Our site is static content only, so this seems like a no-brainer.

What does this solve?

• remove barrier to entry caused by Apache SSI (I don't want to install Apache just to view our website)

Those familiar with static site generators, will very quickly pick up skills needed to work with another, after all the concepts are pretty much the same and templating as well.

While it is true that most websites are viewed via web servers, relying on Apache SSI means that we cannot use any other web server. It means that newer, faster, more performant and increasingly more dominant and popular web servers like nginx cannot be used. Take a look at https://hackr.io/blog/nginx-vs-apache and https://news.netcraft.com/archives/2019/09/27/september-2019-web-server-survey.html if you're curious. We see that Apache at its peak in 2011 was used by ~65%. Today, the market share is less than half. Furthermore, for viewing a built version on my laptop, I would prefer to use tools that come with it already, like opening an index.html or using a simple python web server.

• remove barrier to entry caused by using lots of plain HTML

People aren't used to writing plain HTML on the scale that we do. Literally everything is HTML. Many things could benefit from templating, defining macros and so on. At present, I believe it is a barrier because it results in a cluttered and very long file.

• get rid of the confusing include system and use standard technologies like REST APIs

The crux of this issue comes down to improving the accessibility of the SRCF to newcomers. I believe that an approach that does proper templating and combined with a static site generator would make maintaining the editing the site more sustainable in the future. I don't want to replace SSI because it is old and falling out of use, instead I believe this approach would have the following benefits:

• hot reloading and easy development (comes with an SSG)
• allows us to write in markdown for pages like about
• build once only (SSI is fast, but if we were to using Sass in the future, we would need a SSG to compile it)
• more familiarity among potential contributors, lower barrier to entry
• allows for more advanced templating by passing in arguments to includes, using conditionals, filters and tests (as observed in the jinja language)
• our website becomes webserver-agnostic and be built easily locally, in a container or using an actions runner
• there is a big community build around SSGs, including many plugins and add-ons that we could one day explore to enrich our website
• SSGs are a framework essentially so there is order to the structure of folders and files, I believe we presently lack this

Alternatives considered

A full blown web app is pointless, it's static content. Keeping with Apache SSI just means accumulating more crud and driving us further into the pit of unsustainability.

[dalcde]

We might want to have a template that can be used across many different sites, e.g. the control panel, blog, documentation, Goose, etc. Currently there is a lot of copying and reimplementation of the templates. It would be helpful to do this in a centralized way.

If we were to do this with bootstrap, we may want to wait for bootstrap 5 to be released, or at least be in beta.

[CMTC]

Requirements

* I can launch a dev server in one command

This does not strike me as a feature a static site generator should include. The whole point, as I see it, is that authoring is done in something close to plaintext and the result gets slotted into an HTML template. There should be nothing unexpected in the output. In any case, what is a webserver doing in a tool for converting between document languages and putting things into templates?

Motivation

The website is currently templated by Apache SSI.

To what extent? How simple are they? Do we just include things, or have we got complicated decision trees?

A technology that is largely outdated and unfamiliar to today's population. There is much greater familiarity with static site generators like Gatsby, Hugo, Jekyll, and so on. If we take a look at this link, we can conclude that there is a much larger search volume for the above technologies in comparison to Apache SSI.

If they are so familiar to people, why do they keep having to look them up?

What does this solve?
* remove barrier to entry caused by Apache SSI (I don't want to install Apache just to view our website)

I don't see why elements of the template (which one is surely not involving oneself with when writing content to fill it) are of concern, and so I see little need for the SSIs to be dereferenced by the content author before the page goes live.

In any case, most of the pages on the SRCF website already exist. Confusion about templating SSIs would surely only be of concern if one were writing a new page, in which case one could always decide to avoid them. Editing an existing page, or creating one similar to one that already exists is already a simple task; just ignore the non-content parts. Even in a situation in which there is an SSG, it will presumably be necessary to indicate somewhere in each page's source the template that it is intended to use, which could be of comparable complexity to an SSI include.

Those familiar with static site generators, will very quickly pick up skills needed to work with another, after all the concepts are pretty much the same and templating as well.

SSIs are similarly supported by several webservers, they aren't just an apache thing.

relying on Apache SSI means that we cannot use any other web server.

See above.

Furthermore, for viewing a built version on my laptop, I would prefer to use tools that come with it already, like opening an index.html or using a simple python web server.

Why do you need to see the thing in the template before it goes live? The point of templates is to free the author of such concerns.

* remove barrier to entry caused by using lots of plain HTML

!

Many things could benefit from templating, defining macros and so on.

So aren't more SSIs similarly an answer? Or by defining macros do you mean to say we need something more expressive?

* get rid of the confusing include system and use standard technologies like REST APIs

What has this to do with static sites? Including files to avoid retyping/copying seems quite simple to me, and it's a common pattern.

[matiasilva]

Requirements

* I can launch a dev server in one command

This does not strike me as a feature a static site generator should include. The whole point, as I see it, is that authoring is done in something close to plaintext and the result gets slotted into an HTML template. There should be nothing unexpected in the output. In any case, what is a webserver doing in a tool for converting between document languages and putting things into templates?

The web server is to build the projects (put all the templates together) and combine that with hot reloading so I can easily see what my change looks like as it refreshes the part that I have just changed. If we're using Sass files, it will also need compiling. The dev server makes it easier to see your changes live.

Motivation

The website is currently templated by Apache SSI.

To what extent? How simple are they? Do we just include things, or have we got complicated decision trees?

Presently, just including different bits, the header, the footer, the MOTD. There are however, many things that would benefit from greater templating and are highly repetitive, such as the cards on the front page.

A technology that is largely outdated and unfamiliar to today's population. There is much greater familiarity with static site generators like Gatsby, Hugo, Jekyll, and so on. If we take a look at this link, we can conclude that there is a much larger search volume for the above technologies in comparison to Apache SSI.

If they are so familiar to people, why do they keep having to look them up?

Newcomers who hear about this from others might want to know what it is. Experienced developers need to look up documentation. You don't type the name of what you're trying to learn once in google and never search it again. It's pretty common that whatever software issue people have, people google their problem, and the solution is on stackoverflow or whatever. Same thing. Google searches are indicative of something's popularity.

What does this solve?

• remove barrier to entry caused by Apache SSI (I don't want to install Apache just to view our website)

I don't see why elements of the template (which one is surely not involving oneself with when writing content to fill it) are of concern, and so I see little need for the SSIs to be dereferenced by the content author before the page goes live.

The page is not valid HTML without it. You can't open the index page as the SSI include syntax is incompatible. You'll be missing the footer, the header and bits of the website, if it even works.

In any case, most of the pages on the SRCF website already exist. Confusion about templating SSIs would surely only be of concern if one were writing a new page, in which case one could always decide to avoid them. Editing an existing page, or creating one similar to one that already exists is already a simple task; just ignore the non-content parts. Even in a situation in which there is an SSG, it will presumably be necessary to indicate somewhere in each page's source the template that it is intended to use, which could be of comparable complexity to an SSI include.

We can't avoid them as we have to use them for the header and the footer at the least. It is also the only way of including any dynamic content, like the number of VMs we have and details about them, the MOTD, and such.

I believe you're trying to ask why an SSG would be beneficial. In addition to allowing us to develop different templates, it allows us to write in markdown. Even the way of specifying the template, via YAML frontmatter, is easier to understand and more descriptive than an SSI include.

Those familiar with static site generators, will very quickly pick up skills needed to work with another, after all the concepts are pretty much the same and templating as well.

SSIs are similarly supported by several webservers, they aren't just an apache thing.

relying on Apache SSI means that we cannot use any other web server.

See above.

But the issue is there are far more people familiar with static site generators than Apache SSI, or even the concept of a server side include. They aren't just an Apache thing, but some work is needed to change them to another web server's format. SSG generated HTML is just HTML.

Furthermore, for viewing a built version on my laptop, I would prefer to use tools that come with it already, like opening an index.html or using a simple python web server.

Why do you need to see the thing in the template before it goes live? The point of templates is to free the author of such concerns.

Because I want to verify that the styling is correct. It is common practice to test locally before releasing or sharing code. A similar thing would apply here.

* remove barrier to entry caused by using lots of plain HTML

Not sure what the exclamation point is here. Copying and pasting identical HTML and just changing the content of a title tag generates large and unwieldy HTML documents. The point is to reduce the document to its bare essentials and make it as easy as possible to navigate and understand. Furthermore (though not my point), writing actual HTML is increasingly rare these days with the dominance of front-end frameworks like React, Angular, and Vue. That doesn't forego familiarity with the basis of HTML though, just that if you're writing mountains of it manually, there's probably a better way.

Many things could benefit from templating, defining macros and so on.

So aren't more SSIs similarly an answer? Or by defining macros do you mean to say we need something more expressive?

Why invest further into a technology that (in my opinion) can be replaced by a superior and webserver-agnostic solution? 1. I'm not sure if what I have in mind can be done with SSI 2. I have to spend extra time to find a way to make it work, or to hack it together -- but why invest more time and effort when doing such is standard with SSGs?

* get rid of the confusing include system and use standard technologies like REST APIs

What has this to do with static sites? Including files to avoid retyping/copying seems quite simple to me, and it's a common pattern.

For dynamic content such as the number of VMs (which is already served via a REST API), or the MOTD.

[doismellburning]

As enjoyable as debates on the merits of various web technologies can be, is this not one for whoever 'owns' the website? (Publicity?)

[CMTC]

As enjoyable as debates on the merits of various web technologies can be, is this not one for whoever 'owns' the website? (Publicity?)

I agree. I note that Mr Silva was elected by the society's members to be the Junior Treasurer, rather than the Publicity officer. Despite this he seems to be intent on making what appears to me in many cases to be changes for their own sake to parts of the society that I don't believe are really part of his remit. Perhaps if he should like to make such changes, he should stand at the next AGM for a different role better suited to his platform.

[tguruswamy]

This is only a proposal, keep this friendly please.
This is a choice heavily driven by personal preference, gaining consensus for any particular solution may be difficult.

Some considerations, which do not necessarily rule out a change:

• A large part of SRCF tooling (particularly user and soc websites) is currently specific to Apache; therefore it is unlikely that we would switch from Apache wholesale on the webserver.
• Switching to markdown (or anything else) is not likely to lead to more people contribute if the website is still a complex hierarchy. From personal experience, ensuring that the organisation of the site is easily understood at a glance is more impactful than the specific choice of language.
• Design of APIs for the "backend" is mostly orthogonal to design of the frontend.

I personally do agree the website could be more contributor accessible, and so this is a valid project in principle. However moving from undocumented SSI to undocumented SSG is not necessarily an improvement, so let's proceed carefully. As always it comes down to documentation -- even the most apparently obtuse technology is usable if it comes with really good docs. (in before wv)

[matiasilva]

@tguruswamy I agree entirely.

I mention other web servers not with the intent to suggest we might move, but just to allude to the general trend that technology is evolving, and using SSI (while, yes it does exist for other web servers) is mostly an Apache thing.

Documentation is definitely a must to welcome any contributor, I have personally not developed any docs for editing the website because I feel like we should first do something about it. I feel "stuck" with SSI, I have always felt that there are better solutions to what we want. Speaking of contributors, I think that a good portion come because they are interested in developing their own technical skills and it might prove useful to use technologies that are prominent in today's hobby and commercial world.

The point about markdown was just to simplify editing for not-as-technically-inclined committee members. For the first time in a while, the committee is non sysadmin this year and has non-technical elements. I think it is clearer to have just text in a clean markdown file than to litter it with <p> tags and <h2>s and such. At least for long bits of text, it could be a distraction.

As for the APIs, modern SSGs have plugins that allow you to fetch data on build. We currently insert foreign data by running cron jobs on the server that run python scripts to print data to a file. I feel that the closer we can integrate with a SSG framework, the closer we can get to completely taking away our reliance on Apache/a server for building, thus allowing contributors to easily build our website.

As for the last comment, any member of the SRCF is allowed to have their say on this forum. I am both our treasurer as well as a volunteer asset to the team. Inevitably, a non-technical committee will lack the knowledge to implement proper tools, so as someone who is more technically inclined, I'm giving my 2 cents here. I have also added more to my original post to focus on the merits of this technology rather than its apparent novelty.

[matiasilva]

Thanks for the feedback everyone. I'm going to go ahead with this one. I'm going to use Hugo primarily for its speed and advanced templating features.

There are a few blockers remaining and certain functionality of the website will not be available until we manage to solve them:

• content that is fetched by writing to a file and grabbing local data via cron jobs
• PHP pages that access database files -> REST APIs

Care needs to be taken to preserve all URLs.