Static Site Generator Speed Test (Nikola, Pelican, Hexo, Octopress)

Setup

Every site generator was set up in an identical clean environment, using Ubuntu 15.04, x64, as a 512 MB DigitalOcean VM with a 20 GB SSD drive. The machine was updated, an user account with passwordless sudo was created, and build-essential was installed. Tests were run by an automated installer and timer, written in Bash and C, respectively (custom; source code is available). Pre-compiled wheels for lxml and Pillow were used for Nikola testing, because lxml cannot be compiled with less than 1.5 GB of RAM; they were built with pip wheel lxml pillow outside of the testing environment (on a local VM). The machine was reimaged after every test. Lists of installed Python/Ruby/Node packages are available in the GitHub repo (see below).

Input

Every site generator was given the same set of 179 log files from #nikola on freenode. The raw logs contain 1209507 bytes (1.1 MiB) of plain text. The logs were processed into post files, which fit the format of each engine (reST or Markdown), containing mandatory metadata, an introductory paragraph and a code block (using :: for reST, four spaces for Markdown). One file had to be altered, because they contained the {{ sequence, which was misinterpreted as internal templating by Hexo and Octopress — it was replaced by a harmless ~~ sequence for all four generators.

The generators used default config, with one exception: highlighting was disabled for Hexo. The highlighting would cause an unfair advantage (other generators did not automatically highlight the code boxes), and led to very high build times (see table 4 in comparison spreadsheet).

Build

Sites were built a total of 110 times, in 10 cycles of 11 builds each. The first build of a cycle was a fresh build, the remaining 10 were rebuilds. Sites and cache files were removed after each cycle.

Results

Because Nikola and Hexo use incremental rebuilds, the results were compared in two groups: 11 and 10 runs.

Questions and answers