enpiar

Building a Blogdown Site with Travis-CI

It's easy to automate the publishing of your R Markdown site to GitHub Pages. Free yourself to focus on content, and let Travis handle the rest.

Over on the Crunch.io dev blog, I wrote about how we used Travis-CI to automate the building of our various static sites. I thought I’d do the same for my personal blog, though with one twist: using R and the blogdown package on top of Hugo.

This blogdown GitHub issue pointed to Yihui’s guide to building bookdown projects on Travis, as well as a project that had adapted that flow for blogdown. Both were helpful, and with my prior experience, I was able to build on those examples to come up with a simple workflow. It was shockingly straightforward.

This post outlines the steps I took to automate the Hugo/blogdown build on Travis-CI. It starts from the point of having already installed blogdown locally, created the Hugo project, played around with themes and customization, and having something serving locally that’s worth deploying. It assumes that you have a GitHub account and have used Travis before (surely you use continuous integration on all of your projects, right ;). If you’re not at that point yet, the internet has lots of great resources to help you get started.

There are three main steps: setting up your repository on GitHub and Travis-CI, adding the build script, and setting up your GitHub API token access to connect it all.

Setup on GitHub, git, and Travis-CI

First, we need to create a new repository on GitHub. Then, in your local directory where you’ve started your blogdown project, initialize a git repository and link it with your new repo on GitHub:

git init
git remote add origin git@github.com:nealrichardson/nealrichardson.github.io.git

of course, substituting in your user/organization name and repository name. (GitHub will tell you the exact code to run after you create the repository there.)

Next, go to Travis and enable Travis to run on your new GitHub repository. You’ll have to click the “sync” button in order for the new project to show in the list if you just created it.

About git branches: User page vs. project page

For a typical repository on GitHub, the usual way GitHub Pages are supported is on a gh-pages branch, though there are some options. However, if this is your user or organization page, as is the case for this blog, the static site must be on the master branch. That means my markdown and Hugo customizations need to be on a different branch. I opted to put them on a src branch, and will use Travis to build from that and push the generated site to master.

Given that I already had started the project locally, and I’d done git init, I created the src branch before committing my initial blog code:

git checkout -b src
git add .
git commit -m "Initial commit"

and then checked out master and pushed an empty README.md file up there, just so the branch would exist on GitHub. Travis will need the branch to exist in order to start pushing to it.

Add some files

.travis.yml

The .travis.yml file is where the action happens—it’s what tells Travis what to do. The examples I found for using Travis to build bookdown or blogdown sites followed the model of putting “build” and “deploy” logic into Bash scripts, which the .travis.yml file makes executable and then calls. I found this unnecessarily indirect. Travis treats the YAML file like a script anyway, so you can just enter the shell commands you want to run there. That way, you can look in one file to see everything that is happening.

My .travis.yml is grouped into a few sections. To start, it sets the language/container type to “R” and indicates that packages should be cached for faster repeat building. Second, this block

branches:
  only:
  - src

tells Travis only to build when I push to my src branch. I want to be free to make other branches in my repository and not have them trigger a build of my site—that could lead to very unexpected changes! Moreover, on success, Travis will be pushing to the master branch, and I don’t want that push to attempt a re-build of the site.

Next, the YAML file directs the installation of the blogdown package from GitHub using the r_github_packages block. It’s not yet on CRAN, so GitHub is the way, and while you can specify that it be installed via the DESCRIPTION file using devtools’s special “Remotes” field, I’ve had success installing GitHub packages on Travis this way in the past, and it worked here too.

I grouped the main action in .travis.yml into a before_script block and a script block, but that’s purely for aesthetics—it’s all just a series of commands to be executed. The “before” block sets my GitHub user name and email—the git commit of the built site will need them—installs Hugo via blogdown,1 and then adds my Hugo theme. You could use the blogdown::install_theme function, but all you need it to do is checkout the theme repository to your “themes” directory, so I just do that directly.

At that point, building the site is just R -e 'blogdown::build_site()'. That’s it. As I said in the beginning: shockingly simple.

The last step is to commit the built site to a different branch, in this case “master”, and push that. Even though the Travis job has already cloned your repository from GitHub to start the build, it has only cloned the current branch, so git checkout master won’t work. To change branches, we have to do a fresh git clone. Note that the clone command involves a GH_TOKEN environment variable. That’s your API token—we’ll discuss that below.

One other quirk I added to the YAML file was to write a .travis.yml file into the destination branch for the built site, reiterating the “only build on ‘src’ branch” command. This is because two lines above that, I rm -rf’d everything, so there was no .travis.yml file present on master branch telling Travis not to try to build from master. (Update 7/4/2017: if you have a domain name pointing at your site and you rm -rf everything, you’ll also need to rewrite the “CNAME” file too.)

On the git push, I have the output redirected to /dev/null, i.e. it should run without printing anything. I saw this suggested before when I was setting up my sites at work because without a quiet push, the Travis log would reveal the GitHub API token that was encrypted in the YAML file (see below). It turns out that this was a security vulnerability that Travis has since fixed, but I’ve left the silencing in there for extra comfort.

DESCRIPTION

Because we’re going to use an R language build on Travis, the job will expect that we’re testing an R package, so we need a DESCRIPTION file. In practice, it is useful for installing packages your R code will need, and ostensibly for installing blogdown itself. But since I installed blogdown in the .travis.yml file, and I’m just starting the blog and don’t have any meaningful R code to run on it, there aren’t any other dependencies to install. So, my DESCRIPTION is pretty empty. But it exists.

.gitignore

For Hugo projects, you want to .gitignore the “themes” directory because they aren’t really part of your project; you add your customization of templates and stylesheets on top of a theme. I also like to ignore the “public” directory where the build static site gets written—the build on Travis will overwrite it anyway.

By that logic, you may also want to ignore the static/post directory where blogdown writes generated images and other artifacts from the .Rmd code, and *.html from the content/post directory, which is the built version of the .Rmd files. I tried ignoring those artifacts, and the site built fine on Travis without them, but I don’t have enough experience with blogdown to recommend it as a best practice. If your R code is computationally intensive or relies on third-party services that may be down when your site gets built on Travis, and if having the cached versions checked in prevents running the R code again on Travis, maybe it’s good to keep them.

Add GitHub API token

As mentioned above, you need to generate a GitHub API token so that your job on Travis is authorized to push. When you do, you will given a bunch of checkboxes to define the authorization scope for the token, but you only need to select the public_repo scope.

Token in hand (or clipboard), encrypt it and add it to your YAML file using Travis’s Ruby library. To install that library,

gem install travis

You only need to install this the first time you deal with adding encrypted variables to Travis. In using the utility (instructions here), note that you’re not just encrypting the API token, you’re encrypting the pair of NAME=tokenstring. As mentioned above, I’m using GH_TOKEN as the environment variable name, so, in the project directory, run:

travis encrypt GH_TOKEN=token --add

where token is the string copied from the GitHub page when the token was generated. The --add flag will add the result to the .travis.yml file for you.

While clearly this step has to happen before you can proceed with your build, I put it last in this discussion because some of the steps outlined above have to happen first. You can’t run travis encrypt --add until you’ve enabled Travis on your repository (on travis-ci.org), and you have to have done git remote add origin so that your local repository points to the same project that Travis is expecting.

Once you’ve done all this, commit and git push your blogdown code and Travis config. You can now go to the Travis site and watch the build run forget about it and trust that your site is being built for you—that’s the point of automating it!2

Conclusion

Given that blogdown is a relatively new tool, I expected that using it would show some rough edges, and that trying to build it on Travis would be challenging. It took me ten attempts to figure out the build automation for our Hugo blog at Crunch. I’d hoped I’d learned a few things about how to use Travis for static sites and could do better this time, but adding R to the mix was a complication.

In the end, it was a breeze. The only part I had to tinker with was installing blogdown itself. The complications I feared—having to install go and Hugo on top of a container configured to run R—were seamlessly handled by blogdown.

And finally, to demonstrate that this post is R Markdown and not regular markdown, here’s Figure 1 from the example site that blogdown generates.

par(mar = c(0, 1, 0, 1))
pie(
  c(280, 60, 20),
  c('Sky', 'Sunny side of pyramid', 'Shady side of pyramid'),
  col = c('#0292D8', '#F7EA39', '#C4B632'),
  init.angle = -50, border = NA
)
The first and last pie chart to appear on this blog

Figure 1: The first and last pie chart to appear on this blog


  1. I had to fix a specific version of Hugo to install after what was hopefully just a transient failure to install on Travis. It was previously working with the default “latest” version, but there was a Hugo release a couple of days ago, so maybe that caused some disturbance. You shouldn’t have to hard-code a version.

  2. If the build fails, as in when you’re setting it up or adding a new dependency, Travis will notify you by email. No news is good news.

Published in general and tagged R, automation, meta, travis-ci and website