I’m a big fan of linters. They detect problems earlier (also known as “shifting to the left”), and the earlier problem detection is, the more efficient remediation is. Therefore, I want to lint as much as possible. Lately, I’ve been working a lot with GitLab CI YAML which oftentimes has shell script embedded in it which naturally would benefit from linting. So I figured out how to do so using a combination of yq and shellcheck as demonstrated by this GitLab CI job:
shellcheck:
# See: https://candrews.integralblue.com/2022/02/shellcheck-scripts-embedded-in-gitlab-ci-yaml/
image: alpine:3.22.1@sha256:4bcff63911fcb4448bd4fdacec207030997caf25e9bea4045fa6c8c44de311d1
before_script:
- |
# shellcheck shell=sh
apk update
apk add --no-cache git yq shellcheck
script:
- |
# shellcheck shell=sh
git ls-files --exclude='*.sh' --ignored -c -z | xargs -0r shellcheck -P SCRIPTDIR -x
newline="$(printf '\nq')"
newline=${newline%q}
git ls-files --exclude='*.gitlab-ci.yml' --ignored -c | while IFS= read -r file; do
documentCount=$(yq eval-all '[.] | length' "${file}")
documentIndex=-1
while [ "$documentIndex" -lt "$documentCount" ]; do
true $((documentIndex=documentIndex+1))
yq eval 'select(documentIndex == '${documentIndex}') | .[] | select(tag=="!!map") | (.before_script,.script,.after_script) | select(. != null ) | path | "select(documentIndex == '${documentIndex}') | .[\"" + join("\"].[\"") + "\"]"' "${file}" | while IFS= read -r selector; do
set +e
script=$(yq eval "${selector} | join(\"${newline}\")" "${file}")
status=$?
set -e
if [ $status -ne 0 ]; then
>&2 printf "\nError getting the contents of the selector %s in the file %s:\n\nThe YAML may be malformed." "${selector}" "${file}"
exit 1
fi
if ! printf '%s' "${script}" | shellcheck -x -; then
>&2 printf "\nError in %s in the script specified in %s:\n%s\n" "${file}" "${selector}" "${script}"
exit 1
fi
done
done
done
This GitLab CI job will run shellcheck on:
*.shfiles- files with a shell shebang
- script embedded in GitLab CI YAML in
script,before_script, andafter_script(which are which are all the places where shell script can appear in GitLab CI YAML). GitLab CI YAML files are expected to follow the naming convention*.gitlab-ci.yml.
This job will not handle !reference tags in GitLab CI YAML, so on the off chance that that feature is used in a before_script, after_script, or script, the results won’t be correct.
Happy linting!
