CSCI 241 Systems Programming Labs

Part 3. Shell script hygiene (25 points)

Shell scripting is a powerful tool that is used in many areas of software development, including build scripts (that build the software), test scripts (that test the software), and conformance checking scripts (that check things like formatting conventions). An example of a conformance checking script might be one that is run on every git commit to check that the files being changed have no errors or warnings according to some tool.

You’re going to write a conformance checking script that checks if all of the shell scripts in a directory pass shellcheck.

To find all of the shell scripts in a directory, you will use the find command. Conceptually, we want to loop over all of the files that have a particular extension and do something with them. For example, if we want to print the paths of every text file in a directory whose path is stored in the dir variable, we’d like to be able to do something like this:

for file in $(find "${dir}" -name '*.txt'); do
  echo "${file}"
done

Unfortunately, this doesn’t work! If we run shellcheck on this we get a warning

for file in $(find "${dir}" -name '*.txt'); do
            ^-- SC2044 (warning): For loops over find output are fragile. Use find -exec or a while read loop.

The problem here is that file names can contain spaces (and even newline characters!). The shellcheck wiki page for error SC2044 gives example code to make this work. It would look something like

while IFS= read -r -d '' file; do 
  echo "${file}"
done < <(find "${dir}" -name '*.txt' -print0)

Note carefully the space after the = and the space between the two < characters!

What this arcane construction is doing is it’s first running

find "${dir}" -name '*.txt' -print0

which is the same as the previous find command except that rather than printing the matching paths separated by a newline (which, as mentioned, is a valid character in a file name), it separates the output with a 0 byte. (Not the character 0 which is a byte with integer value 48, but the byte with value 0.)

The output of the find command becomes the standard input for the

while IFS= read -r -d '' file; do ... done

loop. The loop is going to read from stdin (from the read command) and split up the input by 0 bytes (which, conveniently, is what the -print0 argument to find produced).

Each time through the body of the loop, the file variable will be set the path of one of the files that find found.

Your task

Write a shell script called shellcheckall which takes zero or one parameters. The parameter, if given, should be a path to a directory. If no parameters are given, it should act on the current directory. If two or more parameters are given, output usage information to stderr, and exit with return value 1. If the supplied parameter is not a directory, output an error message (on stderr) and exit with return value 1.

The script should search the given directory (and any directories inside of it) to find all of the files with the extension .sh. It should run shellcheck on each file and count how many scripts pass out of the total number of shell scripts. If any of the scripts fail shellcheck, then when your script exits, it should exit with return value 1. See the examples below, including the return values.

Make sure you handle files with spaces in the name. Make sure the script works correctly when run on an empty directory and one that contains no shell scripts (see the examples below).

How many shell scripts in the entire linux-6.4 pass shellcheck? Write your answer in your README.

Write a for-loop that runs shellcheckall on each directory in linux-6.4. It should print out the name of the directory, a colon, a space, and then the output from shellcheckall. Your loop should probably start like this.

for dir in linux-6.4/*/; do

The / after the * means the glob will only match directories. You’ll want to use echo -n to print text without a trailing newline. See the final example below.

Put your for-loop and the output in your README.

$ ./shellcheckall too many args
Usage: ./shellcheckall [dir]
$ echo $?
1

$ ./shellcheckall linux-6.4/tools/perf
6 of 74 shell scripts passed shellcheck
$ echo $?
1

$ ./shellcheckall empty-directory
0 of 0 shell scripts passed shellcheck
$ echo $?
0

$ for dir in linux-6.4/*/; do ...; done
linux-6.4/Documentation/: 1 of 9 shell scripts passed shellcheck
linux-6.4/LICENSES/: 0 of 0 shell scripts passed shellcheck
linux-6.4/arch/: 15 of 39 shell scripts passed shellcheck
...