hexo generate slow with 400 posts

[iMilnb]

$ hexo version
hexo: 3.1.1
os: NetBSD 7.0_STABLE netbsd x64
http_parser: 2.5.0
node: 4.1.1
v8: 4.5.103.33
uv: 1.7.4
zlib: 1.2.3
ares: 1.10.1-DEV
modules: 46
openssl: 1.0.2d
$ hexo clean
INFO  Deleted database.
INFO  Deleted public folder.
$ hexo g
[...]
INFO  1071 files generated in 3.68 min
# without cleaning
$ hexo g
INFO  Files loaded in 4.14 s
INFO  0 files generated in 4 min

Is this a normal generation time for 400 posts? seems awfully slow to me. Every change I make on a layout or a css takes ages to be generated. hexo g -w is barely unusable.

FWIW:

highlight:
  enable: true
  line_number: true
  auto_detect: false
  tab_replace:

Similar results with enable: false.

Theme is light.

[leesei]

Did you tried hexo server?

[iMilnb]

I did, it is indeed more usable and faster, but only for pre-visualization / debugging purposes

[leesei]

I see. I was trying to suggest you to use it during development, so that generate is used sparingly.
What about not cleaning before every generate? It should be able to pick up changes in posts (a theme change will need to be cleaned though).

[iMilnb]

Sure, I don't run clean everytime, just for testing, but TBH, even hexo g without a previous clean takes ages...

$ hexo g
INFO  Files loaded in 4.14 s
INFO  0 files generated in 4 min

[iMilnb]

For the sake of it, I've tried to hexo g on a tmpfs mounted directory, and the result is pretty much the same. Meaning it's not about filesystem access but really processing time.

[Xuanwo]

I don't think it's normal.
As in my blog: https://travis-ci.org/Xuanwo/xuanwo.github.io/builds/111696580 ,

INFO  Files loaded in 4.46 s
INFO  718 files generated in 22 s

The curve of time/files is strange from 22/718(about 0.03) to 240/1071(about 0.224).

Maybe you can try set line_number to false to disable highlight.js totally.

[iMilnb]

Well:

highlight:
  enable: false
  line_number: false
  auto_detect: false
  tab_replace:

$ hexo g 
INFO  Files loaded in 4.35 s
[...]
INFO  36 files generated in 4.08 min

Doesn't seem to run faster. Any hint on how to help debug this?

[leesei]

Using genstubs to generate 500 posts: ./genstubs.js -N500 -P20

Symlinking source to https://github.com/leesei/hexo-vanilla-site

hexo clean
hexo g --debug

00:10:36.697 DEBUG Hexo version: 3.1.1
...
00:10:38.561 INFO  Files loaded in 1.7 s
...
00:10:42.014 INFO  723 files generated in 3.45 s

(Disclaimer: I'm on SSD, but since you've tried with tmpfs, it shouldn't matter)
Here's the full log: hexo-stubs.txt

Please check where most of the time is spent on your run.

[iMilnb]

--debug seems to indicate the rendering is the slowest process, ~0.3s by post. Is there any way to dig deeper on the rendering process to know which part is the slowest?

[leesei]

@tl3shi I cloned your repo
It took 16mins to run hexo g --debug, however if I changed the theme from maupassant to landscape, it took 16 seconds.
maupassant used 0.6s to render a post vs 0.1s to that of landsacape. Maybe you can reflect this to the author.

@iMilnb what theme are you using? Can you give landscape a try and report.

[iMilnb]

theme: landscape

INFO  1074 files generated in 15 s

Boom! that's it, it took only 15 seconds to render the 400 posts with landscape

[leesei]

Alas, though this is not a Hexo problem, it seems easy to write an inefficient theme.

[iMilnb]

Sad thing is that I used the light theme which has been forked by many themes and so is spreading around a lot; so the question remains, how to dig the debug deeper to understand what makes the rendering so slow in comparison to landscape. Meanwhile I'll open an issue at https://github.com/hexojs/hexo-theme-light

[iMilnb]

Ok I found that it's the tagcloud widget that slows the pages generation. Without it I get:

INFO  840 files generated in 3.92 s

Even faster than landscape.

Edit

The tag widget also slows down page generation. FWIW we're talking about 276 tags.

[leesei]

Themes are supposed to make use of fragment_cache (cached render result) to improve performance.
I think theme-light didn't use it.

Templates Optimization| Hexo
Fragment cache by tommy351 · Pull Request #637 · hexojs/hexo

[iMilnb]

Perfect! here's the fix for light, in layout.ejs, replace:

<aside id="sidebar" class="alignright"><%- partial('_partial/sidebar') %></aside>

with

<aside id="sidebar" class="alignright"><%- partial('_partial/sidebar', {}, {cache: true}) %></aside>

And then:

INFO  1072 files generated in 6.44 s

Thanks a lot @leesei !

[leesei]

Thanks for your testing and helping us to get to fragment_cache.
Care to submit PR to theme-light?

[iMilnb]

Will definitely do

[iMilnb]

@leesei PR merged hexojs/hexo-theme-light#58 (comment)
thanks for your help!

[rahil471]

Can we only generate/deploy single file?
Eg:

• I run hexo deploy/generate once. (This renders everything)
• Now I'm only changing the _posts (creating a new article or editing it).
• Now instead of having me to do hexo generate again it would be great if I could just place that new/edited article.

[mnlbox]

Hi guys,
I have same issue with 10000 posts. Can you make me some suggestion in related issue:
#2579

[chilly]

git clone -b source git@github.com:chilly/chilly.github.io.git
git submodules update
hexo generate

hexo generate is too slow. only 600+ posts. And cost more then 10 mins.

[qixiaobo]

git clone -b source git@github.com:chilly/chilly.github.io.git
git submodules update
hexo generate

hexo generate is too slow. only 600+ posts. And cost more then 10 mins.

Me 2