I’m a big fan of linters. They detect problems earlier (also known as “shifting to the left”), and the earlier problem detection is, the more efficient remediation is. Therefore, I want to lint as much as possible. Lately, I’ve been working a lot with GitLab CI YAML which oftentimes has shell script embedded in it which naturally would benefit from linting. So I figured out how to do so using a combination of yq and shellcheck as demonstrated by this GitLab CI job:
shellcheck:
image: alpine:3.19.1
script:
- |
# shellcheck shell=sh
# Check *.sh
git ls-files --exclude='*.sh' --ignored -c -z | xargs -0r shellcheck -P SCRIPTDIR -x
# Check files with a shell shebang
git ls-files -c | while IFS= read -r file; do
if head -n1 "${file}" |grep -q "^#\\! \?/.\+\(ba|d|k\)\?sh" ; then
shellcheck -P SCRIPTDIR -x "${file}"
fi
done
# Check script embedded in GitLab CI YAML named *.gitlab-ci.yml
newline="$(printf '\nq')"
newline=${newline%q}
git ls-files --exclude='*.gitlab-ci.yml' --ignored -c | while IFS= read -r file; do
documentCount=$(yq eval-all '[.] | length' "${file}")
documentIndex=-1
while [ "$documentIndex" -lt "$documentCount" ]; do
true $((documentIndex=documentIndex+1))
yq eval 'select(documentIndex == '${documentIndex}') | .[] | select(tag=="!!map") | (.before_script,.script,.after_script) | select(. != null ) | path | "select(documentIndex == '${documentIndex}') | .[\"" + join("\"].[\"") + "\"]"' "${file}" | while IFS= read -r selector; do
set +e
script=$(yq eval "${selector} | join(\"${newline}\")" "${file}")
status=$?
set -e
if [ $status -ne 0 ]; then
>&2 printf "\nError getting the contents of the selector %s in the file %s:\n\nThe YAML may be malformed." "${selector}" "${file}"
exit 1
fi
if ! printf '%s' "${script}" | shellcheck -x -; then
>&2 printf "\nError in %s in the script specified in %s:\n%s\n" "${file}" "${selector}" "${script}"
exit 1
fi
done
done
done
This GitLab CI job will run shellcheck on:
*.shfiles- files with a shell shebang
- script embedded in GitLab CI YAML in
script,before_script, andafter_script(which are which are all the places where shell script can appear in GitLab CI YAML). GitLab CI YAML files are expected to follow the naming convention*.gitlab-ci.yml.
This job will not handle !reference tags in GitLab CI YAML, so on the off chance that that feature is used in a before_script, after_script, or script, the results won’t be correct.
Happy linting!
