Shellcheck Scripts Embedded in GitLab CI YAML

I’m a big fan of linters. They detect problems earlier (also known as “shifting to the left”), and the earlier problem detection is, the more efficient remediation is. Therefore, I want to lint as much as possible. Lately, I’ve been working a lot with GitLab CI YAML which oftentimes has shell script embedded in it which naturally would benefit from linting. So I figured out how to do so using a combination of yq and shellcheck as demonstrated by this GitLab CI job:

shellcheck:
  image: alpine:3.15.0
  script:
    - |
      # shellcheck shell=sh
      # Check *.sh
      git ls-files --exclude='*.sh' --ignored -c -z | xargs -0r shellcheck -x
      # Check files with a shell shebang
      git ls-files -c | while IFS= read -r file; do
        if head -n1 "${file}" |grep -q "^#\\! \?/.\+\(ba|d|k\)\?sh" ; then
          shellcheck -x "${file}"
        fi
      done
      # Check script embedded in GitLab CI YAML named *.gitlab-ci.yml
      newline="$(printf '\nq')"
      newline=${newline%q}
      git ls-files --exclude='*.gitlab-ci.yml' --ignored -c | while IFS= read -r file; do
        yq eval '.[] | select(tag=="!!map") | (.before_script,.script,.after_script) | select(. != null ) | path | ".[\"" + join("\"].[\"") + "\"]"' "${file}" | while IFS= read -r selector; do
          script=$(yq eval "${selector} | join(\"${newline}\")" "${file}")
          if ! printf '%s' "${script}" | shellcheck -; then
              >&2 printf "\nError in %s in the script specified in %s:\n%s\n" "${file}" "${selector}" "${script}"
              exit 1
          fi
        done
      done

This GitLab CI job will run shellcheck on:

  • *.sh files
  • files with a shell shebang
  • script embedded in GitLab CI YAML in script, before_script, and after_script (which are which are all the places where shell script can appear in GitLab CI YAML). GitLab CI YAML files are expected to follow the naming convention *.gitlab-ci.yml.

This job will not handle !reference tags in GitLab CI YAML, so on the off chance that that feature is used in a before_script, after_script, or script, the results won’t be correct.

Happy linting!

CC BY-SA 4.0 Shellcheck Scripts Embedded in GitLab CI YAML by Craig Andrews is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

2 thoughts on “Shellcheck Scripts Embedded in GitLab CI YAML

  1. Hi Craig Andrews, is there any commands available to implement linting on shell-script present in yaml file using shellcheck and yq, rather than this CI job.

    1. The commands to implement linting on shell script present in GitLab CI yaml files using shellcheck and yq are in the job, here they are:

      newline="$(printf '\nq')"
      newline=${newline%q}
      git ls-files --exclude='*.gitlab-ci.yml' --ignored -c | while IFS= read -r file; do
        yq eval '.[] | select(tag=="!!map") | (.before_script,.script,.after_script) | select(. != null ) | path | ".[\"" + join("\"].[\"") + "\"]"' "${file}" | while IFS= read -r selector; do
          script=$(yq eval "${selector} | join(\"${newline}\")" "${file}")
          if ! printf '%s' "${script}" | shellcheck -; then
        >&2 printf "\nError in %s in the script specified in %s:\n%s\n" "${file}" "${selector}" "${script}"
        exit 1
          fi
        done
      done
      

Leave a Reply to Craig Andrews Cancel reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.