I’m a big fan of linters. They detect problems earlier (also known as “shifting to the left”), and the earlier problem detection is, the more efficient remediation is. Therefore, I want to lint as much as possible. Lately, I’ve been working a lot with GitLab CI YAML which oftentimes has shell script embedded in it which naturally would benefit from linting. So I figured out how to do so using a combination of yq and shellcheck as demonstrated by this GitLab CI job:
shellcheck:
image: alpine:3.15.0
script:
- |
# shellcheck shell=sh
# Check *.sh
git ls-files --exclude='*.sh' --ignored -c -z | xargs -0r shellcheck -x
# Check files with a shell shebang
git ls-files -c | while IFS= read -r file; do
if head -n1 "${file}" |grep -q "^#\\! \?/.\+\(ba|d|k\)\?sh" ; then
shellcheck -x "${file}"
fi
done
# Check script embedded in GitLab CI YAML named *.gitlab-ci.yml
newline="$(printf '\nq')"
newline=${newline%q}
git ls-files --exclude='*.gitlab-ci.yml' --ignored -c | while IFS= read -r file; do
yq eval '.[] | select(tag=="!!map") | (.before_script,.script,.after_script) | select(. != null ) | path | ".[\"" + join("\"].[\"") + "\"]"' "${file}" | while IFS= read -r selector; do
script=$(yq eval "${selector} | join(\"${newline}\")" "${file}")
if ! printf '%s' "${script}" | shellcheck -; then
>&2 printf "\nError in %s in the script specified in %s:\n%s\n" "${file}" "${selector}" "${script}"
exit 1
fi
done
done
This GitLab CI job will run shellcheck on:
*.shfiles- files with a shell shebang
- script embedded in GitLab CI YAML in
script,before_script, andafter_script(which are which are all the places where shell script can appear in GitLab CI YAML). GitLab CI YAML files are expected to follow the naming convention*.gitlab-ci.yml.
This job will not handle !reference tags in GitLab CI YAML, so on the off chance that that feature is used in a before_script, after_script, or script, the results won’t be correct.
Happy linting!
Shellcheck Scripts Embedded in GitLab CI YAML by Craig Andrews is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Hi Craig Andrews, is there any commands available to implement linting on shell-script present in yaml file using shellcheck and yq, rather than this CI job.
The commands to implement linting on shell script present in GitLab CI yaml files using shellcheck and yq are in the job, here they are:
newline="$(printf '\nq')" newline=${newline%q} git ls-files --exclude='*.gitlab-ci.yml' --ignored -c | while IFS= read -r file; do yq eval '.[] | select(tag=="!!map") | (.before_script,.script,.after_script) | select(. != null ) | path | ".[\"" + join("\"].[\"") + "\"]"' "${file}" | while IFS= read -r selector; do script=$(yq eval "${selector} | join(\"${newline}\")" "${file}") if ! printf '%s' "${script}" | shellcheck -; then >&2 printf "\nError in %s in the script specified in %s:\n%s\n" "${file}" "${selector}" "${script}" exit 1 fi done done