Organizations already automate running builds, executing tests, and performing deployments to free developers from tedium and improve reliability. The next step is to use automation to improve projects. Tools (bots) can submit pull requests that fix typos, optimize images, and more. I’ve had a great positive experience using a bot to perform the tedious task of checking for dependency updates and submitting pull requests when they’re found, and so far, my tool of choice has been Renovate.
Before diving into the “how” it’s important to cover the “why” – why update dependencies at all? The answer is because it provides business value. Outdated dependencies carry security risks, and security vulnerabilities are obviously a threat to the business. Another reason is for maintainability; the net effort to do frequently, very small upgrades is less than doing one huge upgrade . For example, if a project is using a dependency named Foo at version 1.0, and suddenly there is a business need to upgrade to Foo 1.5 (perhaps due a security vulnerability), that 1.0 to 1.5 jump is pretty significant and carries risk. On the other hand, if Foo had been kept up to date, upgrading Foo from 1.4 to 1.5 is much lower risk. Finally, another reason is for bug fixes and performance improvements. Keeping dependencies up to date can provide business value that a project may not even notice it was missing out on.
Checking for and performing dependency updates is exactly the kind of tedious task that developers don’t like doing and aren’t good at it. To check for updates, developers have to check project web sites or run tools such as npm outdated. When an update is discovered, the developer has to make the change then submit a pull request. Hopefully, at this point, Continuous Integration (CI) takes over, running tests against the PR determine if it’s safe to merge.
Renovate, and other tools like it such as Dependabot (provided by GitHub), will do the checking, updating of dependency declaration files (such as package.json, build.gradle, pom.xml, etc), and creation of pull requests automatically. They even include release notes in the pull request description so developers can review the changes before merging.
The next step is to automatically merge these pull requests when CI executed tests pass. This step requires a comprehensive tests suite and a significant trust in it – not all projects and organizations may be comfortable with it, and it may not be a good idea to do for all dependencies. Sometimes, a developer’s eyes are still the best review.
One of the greatest features of Renovate is that it is entirely free and open source software that can be self-hosted. Renovate is very friendly to contributors, so any bugs get fixed quickly and features are routinely added; I’ve made a couple of contributions to it: one, two. And if self-hosting isn’t your thing, there is a free Renovate GitHub application and Renovate bot for GitLab.com. The organization where I currently work self-hosts everything, including a GitLab instance. Self-hosting Renovate for this private GitLab instance was straightforward:
I set up Renovate by creating a new GitLab project (for example, renovate-pipeline) as follows:
- In the project, set some custom CD/CD variables:
- “GITHUB_COM_TOKEN” must be set a GitHub API token used to get release notes. See instructions in Renovate’s documentation.
- “RENOVATE_TOKEN” must be set to a GitLab personal access token for the user that Renovate runs as; it accesses project repositories and creates pull requests using these credentials. See details in Renovate’s documentation.
- The project repository contains a GitLab CI
.gitlab-ci.ymlpipeline configuration file, like this:
stages:
- lint
- run
.template:
image: renovate/renovate
before_script:
- export GITHUB_COM_TOKEN=$(cat "${GITHUB_COM_TOKEN_FILE}")
- if [[ -z "${GITHUB_COM_TOKEN}" ]]; then echo "variable not set"; exit 1; fi
- export RENOVATE_TOKEN=$(cat "${RENOVATE_TOKEN_FILE}")
- if [[ -z "${RENOVATE_TOKEN}" ]]; then echo "variable not set"; exit 1; fi
renovate-config-validator:
extends: .template
stage: lint
script:
- renovate-config-validator
renovate-run:
extends: .template
resource_group: renovate-run
stage: run
services:
- name: docker:dind
alias: docker
# explicitly disable tls to avoid docker startup interruption as of docker 20.10.9
# there is so security concern because docker is only accessed from within the same system;
# there is no external network communication to worry about.
# See https://github.com/testcontainers/testcontainers-java/pull/4573
command: ["--tls=false"]
variables:
DOCKER_HOST: "tcp://docker:2375"
DOCKER_TLS_CERTDIR: ""
cache:
key: "$CI_COMMIT_REF_NAME"
policy: pull-push
paths:
- .renovate
script:
- renovate --base-dir .renovate
only:
- schedules
- The project repository contains a renovate configuration file called
config.json, like this:
const gitlabApiUrl = process.env.CI_API_V4_URL;
if(! gitlabApiUrl){
console.log("The CI_API_V4_URL environment variable must be set")
process.exit(1);
}
module.exports = {
platform: 'gitlab',
endpoint: gitlabApiUrl,
gitAuthor: "Renovate Service <email@example.com>",
labels: ["renovate"],
//logLevel: 'debug',
printConfig: true,
onboardingConfig: { extends: ["config:base"] },
autodiscover: true
};
- Finally, in the project’s “CI/CD” “Schedule” schedule the pipeline to run periodically (for example, every 4 hours).
That’s it – with this setup, Renovate will start submitting merge requests (MRs) to GitLab projects it can access.
For reference, here’s what such an MR looks like:
Renovate’s configuration is highly customization allowing global defaults to be specified in config.js or in each individual project repository’s renovate.json. And unless that global configuration is overridden to the contrary, Renovate will only open one onboarding MR to each project it can access, and until and unless that onboarding MR is merged, Renovate won’t submit any more MRs. This approach allows organizations to gradually adopt Renovate as each project desires to do so.
