Redesigning website update workflow
January 2018
Today I made a significant change to the workflow I use to update both this website, as well as the website for my re-enactment group. For context, the websites are hosted on a VPS, running Nginx as its web-server. Since I am using jekyll to build the websites, I make all changes, as well as basic testing on my workstation. Once the desired update is ready, I use Jekyll to generate the static HTML files.
Previously I’d simply run the build locally, and then securely copy the files across to the VPS, placing them in the relevant directories, and ensuring that they have the correct permissions. For change management, the files are in local git repos, which gets pushed to a GitLab instance I’m hosting in my homelab. This process was fairly hands-on, and is technical enough that most of the stakeholders for the re-enactment website would not be able to make any changes.
Enter the upgrade, Firstly I migrated the private GitLab instance to github. I partially chose this because I was required to make a GitHub account for a training course I completed late last year (and ended up not using it anyway). The other reason is that it allows me to use an admin panel for the re-enactment website if this is ever required.
Once this was completed, I then set up a Continuous Integration (CI) server to get the changes via git, and then build and publish the website. I ended up choosing Buildbot as the CI software. This was to ensure the CI was as lean as possible and still allow enough flexibility to build the website, matching my previous workflow.
The configuration is fairly straightforward for anyone that is familiar with python and has read the documentation. Below are some excerpts of the config that powers this website.
####### CHANGESOURCES
# the 'change_source' setting tells the buildmaster how it should find out
# about source code changes. Here we point to the buildbot version of a python hello-world project.
c['change_source'] = []
c['change_source'].append(changes.GitPoller(
repourl='git://github.com/ThorgrimA/alexgardner.id.au.git',
branches=['master', 'preview'],
project='alexgardner.id.au',
pollAtLaunch=True,
pollinterval=600))
####### SCHEDULERS
# Configure the Schedulers, which decide how to react to incoming changes. In this
# case, just kick off a 'runtests' build
c['schedulers'] = []
c['schedulers'].append(schedulers.SingleBranchScheduler(
name="alexgardner.id.au",
change_filter=util.ChangeFilter(project='alexgardner.id.au', branch='master'),
builderNames=["alexgardner.id.au"]))
####### BUILDERS
# The 'builders' list defines the Builders, which tell Buildbot how to perform a build:
# what steps, and which workers can execute them. Note that any particular build will
# only take place on one worker.
c['builders'] = []
##############################################################################################
# Alexgardner Build steps
f_alexgardner = util.BuildFactory()
f_alexgardner.addStep(steps.Git(
name="Git Pull", haltOnFailure=True,
repourl='git://github.com/ThorgrimA/alexgardner.id.au', branch='master',
mode='full', method='fresh', submodules=True, progress=True))
f_alexgardner.addStep(steps.ShellCommand(
name="Bundle Install", haltOnFailure=True,
command=["/home/buildbot/.rvm/wrappers/ruby-2.2.5/bundle", "install"]))
f_alexgardner.addStep(steps.ShellCommand(
name="Bundle Exec Jekyll Build", haltOnFailure=True,
command=["/home/buildbot/.rvm/wrappers/ruby-2.2.5/bundle", "exec", "jekyll", "build"]))
f_alexgardner.addStep(steps.ShellCommand(
name="Transfer Files", haltOnFailure=True,
command=["/home/buildbot/jekyll-builds/transfer-alexgardner.sh"]))
f_alexgardner.addStep(steps.RemoveDirectory(name="Cleaning Up", dir="build"))
c['builders'].append(
util.BuilderConfig(name="alexgardner.id.au",
workernames=["worker-1"],
factory=f_alexgardner))
Buildbot does have a basic web GUI as well, and like most CI software details the steps configured for a build.
Overall I’m quite happy with the change, as it greatly simplifies updating this website/blog. An added benefit is that it will allow my re-enactment group to update their website without needing to know the details of the technology that builds the website.