The How and Why Automating Dependency Updates

Organizations already automate running builds, executing tests, and performing deployments to free developers from tedium and improve reliability. The next step is to use automation to improve projects. Tools (bots) can submit pull requests that fix typos, optimize images, and more. I’ve had a great positive experience using a bot to perform the tedious task of checking for dependency updates and submitting pull requests when they’re found, and so far, my tool of choice has been Renovate.

Before diving into the “how” it’s important to cover the “why” – why update dependencies at all? The answer is because it provides business value. Outdated dependencies carry security risks, and security vulnerabilities are obviously a threat to the business. Another reason is for maintainability; the net effort to do frequently, very small upgrades is less than doing one huge upgrade . For example, if a project is using a dependency named Foo at version 1.0, and suddenly there is a business need to upgrade to Foo 1.5 (perhaps due a security vulnerability), that 1.0 to 1.5 jump is pretty significant and carries risk. On the other hand, if Foo had been kept up to date, upgrading Foo from 1.4 to 1.5 is much lower risk. Finally, another reason is for bug fixes and performance improvements. Keeping dependencies up to date can provide business value that a project may not even notice it was missing out on.

Checking for and performing dependency updates is exactly the kind of tedious task that developers don’t like doing and aren’t good at it. To check for updates, developers have to check project web sites or run tools such as npm outdated. When an update is discovered, the developer has to make the change then submit a pull request. Hopefully, at this point, Continuous Integration (CI) takes over, running tests against the PR determine if it’s safe to merge.

Renovate, and other tools like it such as Dependabot (provided by GitHub), will do the checking, updating of dependency declaration files (such as package.json, build.gradle, pom.xml, etc), and creation of pull requests automatically. They even include release notes in the pull request description so developers can review the changes before merging.

The next step is to automatically merge these pull requests when CI executed tests pass. This step requires a comprehensive tests suite and a significant trust in it – not all projects and organizations may be comfortable with it, and it may not be a good idea to do for all dependencies. Sometimes, a developer’s eyes are still the best review.

One of the greatest features of Renovate is that it is entirely free and open source software that can be self-hosted. Renovate is very friendly to contributors, so any bugs get fixed quickly and features are routinely added; I’ve made a couple of contributions to it: one, two. And if self-hosting isn’t your thing, there is a free Renovate GitHub application and Renovate bot for GitLab.com. The organization where I currently work self-hosts everything, including a GitLab instance. Self-hosting Renovate for this private GitLab instance was straightforward:

I set up Renovate by creating a new GitLab project (for example, renovate-pipeline) as follows:

stages:
  - lint
  - run

.template:
  image: renovate/renovate
  before_script:
    - export GITHUB_COM_TOKEN=$(cat "${GITHUB_COM_TOKEN_FILE}")
    - if [[ -z "${GITHUB_COM_TOKEN}" ]]; then  echo "variable not set"; exit 1; fi
    - export RENOVATE_TOKEN=$(cat "${RENOVATE_TOKEN_FILE}")
    - if [[ -z "${RENOVATE_TOKEN}" ]]; then  echo "variable not set"; exit 1; fi

renovate-config-validator:
  extends: .template
  stage: lint
  script:
    - renovate-config-validator

renovate-run:
  extends: .template
  resource_group: renovate-run
  stage: run
  services:
    - name: docker:dind
      alias: docker
  variables:
    DOCKER_HOST: "tcp://docker:2375"
    DOCKER_TLS_CERTDIR: ""
  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: pull-push
    paths:
      - .renovate
  script:
    - renovate --base-dir .renovate
  only:
    - schedules

  • The project repository contains a renovate configuration file called config.json, like this:
const gitlabApiUrl = process.env.CI_API_V4_URL;
if(! gitlabApiUrl){
    console.log("The CI_API_V4_URL environment variable must be set")
    process.exit(1);
}

module.exports = {
    platform: 'gitlab',
    endpoint: gitlabApiUrl,
    gitAuthor: "Renovate Service <email@example.com>",
    labels: ["renovate"],
    //logLevel: 'debug',
    printConfig: true,
    onboardingConfig: { extends: ["config:base"] },
    autodiscover: true
};

That’s it – with this setup, Renovate will start submitting merge requests (MRs) to GitLab projects it can access.

For reference, here’s what such an MR looks like:

screenshot of a Renovate created GitLab Merge Request

Renovate’s configuration is highly customization allowing global defaults to be specified in config.js or in each individual project repository’s renovate.json. And unless that global configuration is overridden to the contrary, Renovate will only open one onboarding MR to each project it can access, and until and unless that onboarding MR is merged, Renovate won’t submit any more MRs. This approach allows organizations to gradually adopt Renovate as each project desires to do so.

CC BY-SA 4.0 The How and Why Automating Dependency Updates by Craig Andrews is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.