Over on the Crunch.io dev blog, I wrote about how we used Travis-CI to automate the building of our various static sites. I thought I’d do the same for my personal blog, though with one twist: using R and the blogdown package on top of Hugo.
This blogdown GitHub issue pointed to Yihui’s guide to building bookdown projects on Travis, as well as a project that had adapted that flow for
blogdown. Both were helpful, and with my prior experience, I was able to build on those examples to come up with a simple workflow. It was shockingly straightforward.
This post outlines the steps I took to automate the Hugo/blogdown build on Travis-CI. It starts from the point of having already installed blogdown locally, created the Hugo project, played around with themes and customization, and having something serving locally that’s worth deploying. It assumes that you have a GitHub account and have used Travis before (surely you use continuous integration on all of your projects, right ;). If you’re not at that point yet, the internet has lots of great resources to help you get started.
There are three main steps: setting up your repository on GitHub and Travis-CI, adding the build script, and setting up your GitHub API token access to connect it all.
Setup on GitHub, git, and Travis-CI
First, we need to create a new repository on GitHub. Then, in your local directory where you’ve started your blogdown project, initialize a git repository and link it with your new repo on GitHub:
git init git remote add origin email@example.com:nealrichardson/nealrichardson.github.io.git
of course, substituting in your user/organization name and repository name. (GitHub will tell you the exact code to run after you create the repository there.)
Next, go to Travis and enable Travis to run on your new GitHub repository. You’ll have to click the “sync” button in order for the new project to show in the list if you just created it.
About git branches: User page vs. project page
For a typical repository on GitHub, the usual way GitHub Pages are supported is on a
gh-pages branch, though there are some options. However, if this is your user or organization page, as is the case for this blog, the static site must be on the
master branch. That means my markdown and Hugo customizations need to be on a different branch. I opted to put them on a
src branch, and will use Travis to build from that and push the generated site to
Given that I already had started the project locally, and I’d done
git init, I created the
src branch before committing my initial blog code:
git checkout -b src git add . git commit -m "Initial commit"
and then checked out
master and pushed an empty README.md file up there, just so the branch would exist on GitHub. Travis will need the branch to exist in order to start pushing to it.
Add some files
The .travis.yml file is where the action happens—it’s what tells Travis what to do. The examples I found for using Travis to build bookdown or blogdown sites followed the model of putting “build” and “deploy” logic into Bash scripts, which the .travis.yml file makes executable and then calls. I found this unnecessarily indirect. Travis treats the YAML file like a script anyway, so you can just enter the shell commands you want to run there. That way, you can look in one file to see everything that is happening.
My .travis.yml is grouped into a few sections. To start, it sets the language/container type to “R” and indicates that packages should be cached for faster repeat building. Second, this block
branches: only: - src
tells Travis only to build when I push to my
src branch. I want to be free to make other branches in my repository and not have them trigger a build of my site—that could lead to very unexpected changes! Moreover, on success, Travis will be pushing to the
master branch, and I don’t want that push to attempt a re-build of the site.
Next, the YAML file directs the installation of the blogdown package from GitHub using the
r_github_packages block. It’s not yet on CRAN, so GitHub is the way, and while you can specify that it be installed via the DESCRIPTION file using
devtools’s special “Remotes” field, I’ve had success installing GitHub packages on Travis this way in the past, and it worked here too.
I grouped the main action in .travis.yml into a
before_script block and a
script block, but that’s purely for aesthetics—it’s all just a series of commands to be executed. The “before” block sets my GitHub user name and email—the
git commit of the built site will need them—installs Hugo via blogdown,1 and then adds my Hugo theme. You could use the
blogdown::install_theme function, but all you need it to do is checkout the theme repository to your “themes” directory, so I just do that directly.
At that point, building the site is just
R -e 'blogdown::build_site()'. That’s it. As I said in the beginning: shockingly simple.
The last step is to commit the built site to a different branch, in this case “master”, and push that. Even though the Travis job has already cloned your repository from GitHub to start the build, it has only cloned the current branch, so
git checkout master won’t work. To change branches, we have to do a fresh
git clone. Note that the
clone command involves a
GH_TOKEN environment variable. That’s your API token—we’ll discuss that below.
One other quirk I added to the YAML file was to write a .travis.yml file into the destination branch for the built site, reiterating the “only build on ‘src’ branch” command. This is because two lines above that, I
rm -rf’d everything, so there was no .travis.yml file present on
master branch telling Travis not to try to build from
master. (Update 7/4/2017: if you have a domain name pointing at your site and you
rm -rf everything, you’ll also need to rewrite the “CNAME” file too.)
git push, I have the output redirected to
/dev/null, i.e. it should run without printing anything. I saw this suggested before when I was setting up my sites at work because without a quiet push, the Travis log would reveal the GitHub API token that was encrypted in the YAML file (see below). It turns out that this was a security vulnerability that Travis has since fixed, but I’ve left the silencing in there for extra comfort.
Because we’re going to use an R language build on Travis, the job will expect that we’re testing an R package, so we need a DESCRIPTION file. In practice, it is useful for installing packages your R code will need, and ostensibly for installing
blogdown itself. But since I installed blogdown in the .travis.yml file, and I’m just starting the blog and don’t have any meaningful R code to run on it, there aren’t any other dependencies to install. So, my DESCRIPTION is pretty empty. But it exists.
For Hugo projects, you want to .gitignore the “themes” directory because they aren’t really part of your project; you add your customization of templates and stylesheets on top of a theme. I also like to ignore the “public” directory where the build static site gets written—the build on Travis will overwrite it anyway.
By that logic, you may also want to ignore the
static/post directory where blogdown writes generated images and other artifacts from the .Rmd code, and
*.html from the
content/post directory, which is the built version of the .Rmd files. I tried ignoring those artifacts, and the site built fine on Travis without them, but I don’t have enough experience with blogdown to recommend it as a best practice. If your R code is computationally intensive or relies on third-party services that may be down when your site gets built on Travis, and if having the cached versions checked in prevents running the R code again on Travis, maybe it’s good to keep them.
Add GitHub API token
As mentioned above, you need to generate a GitHub API token so that your job on Travis is authorized to push. When you do, you will given a bunch of checkboxes to define the authorization scope for the token, but you only need to select the
Token in hand (or clipboard), encrypt it and add it to your YAML file using Travis’s Ruby library. To install that library,
gem install travis
You only need to install this the first time you deal with adding encrypted variables to Travis. In using the utility (instructions here), note that you’re not just encrypting the API token, you’re encrypting the pair of
NAME=tokenstring. As mentioned above, I’m using
GH_TOKEN as the environment variable name, so, in the project directory, run:
travis encrypt GH_TOKEN=token --add
token is the string copied from the GitHub page when the token was generated. The
--add flag will add the result to the .travis.yml file for you.
While clearly this step has to happen before you can proceed with your build, I put it last in this discussion because some of the steps outlined above have to happen first. You can’t run
travis encrypt --add until you’ve enabled Travis on your repository (on travis-ci.org), and you have to have done
git remote add origin so that your local repository points to the same project that Travis is expecting.
Once you’ve done all this, commit and
git push your blogdown code and Travis config. You can now
go to the Travis site and watch the build run forget about it and trust that your site is being built for you—that’s the point of automating it!2
blogdown is a relatively new tool, I expected that using it would show some rough edges, and that trying to build it on Travis would be challenging. It took me ten attempts to figure out the build automation for our Hugo blog at Crunch. I’d hoped I’d learned a few things about how to use Travis for static sites and could do better this time, but adding R to the mix was a complication.
In the end, it was a breeze. The only part I had to tinker with was installing blogdown itself. The complications I feared—having to install go and Hugo on top of a container configured to run R—were seamlessly handled by blogdown.
And finally, to demonstrate that this post is R Markdown and not regular markdown, here’s Figure 1 from the example site that
par(mar = c(0, 1, 0, 1)) pie( c(280, 60, 20), c('Sky', 'Sunny side of pyramid', 'Shady side of pyramid'), col = c('#0292D8', '#F7EA39', '#C4B632'), init.angle = -50, border = NA )
I had to fix a specific version of Hugo to install after what was hopefully just a transient failure to install on Travis. It was previously working with the default “latest” version, but there was a Hugo release a couple of days ago, so maybe that caused some disturbance. You shouldn’t have to hard-code a version.↩
If the build fails, as in when you’re setting it up or adding a new dependency, Travis will notify you by email. No news is good news.↩