Chapter 11. Command Substitution

Command substitution reassigns the output of a command [1] or even multiple commands; it literally plugs the command output into another context. [2]

The classic form of command substitution uses backquotes (`...`). Commands within backquotes (backticks) generate command line text.

1 script_name=`basename $0` 2 echo "The name of this script is $script_name."

1 rm `cat filename` # "filename" contains a list of files to delete. 2 # 3 # S. C. points out that "arg list too long" error might result. 4 # Better is xargs rm -- < filename 5 # ( -- covers those cases where "filename" begins with a "-" ) 6  7 textfile_listing=`ls *.txt` 8 # Variable contains names of all *.txt files in current working directory. 9 echo $textfile_listing 10  11 textfile_listing2=$(ls *.txt) # The alternative form of command substitution. 12 echo $textfile_listing2 13 # Same result. 14  15 # A possible problem with putting a list of files into a single string 16 # is that a newline may creep in. 17 # 18 # A safer way to assign a list of files to a parameter is with an array. 19 # shopt -s nullglob # If no match, filename expands to nothing. 20 # textfile_listing=( *.txt ) 21 # 22 # Thanks, S.C.

Command substitution invokes a subshell.

Command substitution may result in word splitting. 1 COMMAND `echo a b` # 2 args: a and b 2  3 COMMAND "`echo a b`" # 1 arg: "a b" 4  5 COMMAND `echo` # no arg 6  7 COMMAND "`echo`" # one empty arg 8  9  10 # Thanks, S.C. Even when there is no word splitting, command substitution can remove trailing newlines. 1 # cd "`pwd`" # This should always work. 2 # However... 3  4 mkdir 'dir with trailing newline 5 ' 6  7 cd 'dir with trailing newline 8 ' 9  10 cd "`pwd`" # Error message: 11 # bash: cd: /tmp/file with trailing newline: No such file or directory 12  13 cd "$PWD" # Works fine. 14  15  16  17  18  19 old_tty_setting=$(stty -g) # Save old terminal setting. 20 echo "Hit a key " 21 stty -icanon -echo # Disable "canonical" mode for terminal. 22  # Also, disable *local* echo. 23 key=$(dd bs=1 count=1 2> /dev/null) # Using 'dd' to get a keypress. 24 stty "$old_tty_setting" # Restore old setting. 25 echo "You hit ${#key} key." # ${#variable} = number of characters in $variable 26 # 27 # Hit any key except RETURN, and the output is "You hit 1 key." 28 # Hit RETURN, and it's "You hit 0 key." 29 # The newline gets eaten in the command substitution. 30  31 Thanks, S.C.

Using echo to output an unquoted variable set with command substitution removes trailing newlines characters from the output of the reassigned command(s). This can cause unpleasant surprises. 1 dir_listing=`ls -l` 2 echo $dir_listing # unquoted 3  4 # Expecting a nicely ordered directory listing. 5  6 # However, what you get is: 7 # total 3 -rw-rw-r-- 1 bozo bozo 30 May 13 17:15 1.txt -rw-rw-r-- 1 bozo 8 # bozo 51 May 15 20:57 t2.sh -rwxr-xr-x 1 bozo bozo 217 Mar 5 21:13 wi.sh 9  10 # The newlines disappeared. 11  12  13 echo "$dir_listing" # quoted 14 # -rw-rw-r-- 1 bozo 30 May 13 17:15 1.txt 15 # -rw-rw-r-- 1 bozo 51 May 15 20:57 t2.sh 16 # -rwxr-xr-x 1 bozo 217 Mar 5 21:13 wi.sh

Command substitution even permits setting a variable to the contents of a file, using either redirection or the cat command.

1 variable1=`<file1` # Set "variable1" to contents of "file1". 2 variable2=`cat file2` # Set "variable2" to contents of "file2". 3  # This, however, forks a new process, 4  #+ so the line of code executes slower than the above version. 5  6 # Note: 7 # The variables may contain embedded whitespace, 8 #+ or even (horrors), control characters.

1 # Excerpts from system file, /etc/rc.d/rc.sysinit 2 #+ (on a Red Hat Linux installation) 3  4  5 if [ -f /fsckoptions ]; then 6  fsckoptions=`cat /fsckoptions` 7 ... 8 fi 9 # 10 # 11 if [ -e "/proc/ide/${disk[$device]}/media" ] ; then 12  hdmedia=`cat /proc/ide/${disk[$device]}/media` 13 ... 14 fi 15 # 16 # 17 if [ ! -n "`uname -r | grep -- "-"`" ]; then 18  ktag="`cat /proc/version`" 19 ... 20 fi 21 # 22 # 23 if [ $usb = "1" ]; then 24  sleep 5 25  mouseoutput=`cat /proc/bus/usb/devices 2>/dev/null|grep -E "^I.*Cls=03.*Prot=02"` 26  kbdoutput=`cat /proc/bus/usb/devices 2>/dev/null|grep -E "^I.*Cls=03.*Prot=01"` 27 ... 28 fi

Do not set a variable to the contents of a long text file unless you have a very good reason for doing so. Do not set a variable to the contents of a binary file, even as a joke. Example 11-1. Stupid script tricks 1 #!/bin/bash 2 # stupid-script-tricks.sh: Don't try this at home, folks. 3 # From "Stupid Script Tricks," Volume I. 4  5  6 dangerous_variable=`cat /boot/vmlinuz` # The compressed Linux kernel itself. 7  8 echo "string-length of \$dangerous_variable = ${#dangerous_variable}" 9 # string-length of $dangerous_variable = 794151 10 # (Does not give same count as 'wc -c /boot/vmlinuz'.) 11  12 # echo "$dangerous_variable" 13 # Don't try this! It would hang the script. 14  15  16 # The document author is aware of no useful applications for 17 #+ setting a variable to the contents of a binary file. 18  19 exit 0 Notice that a buffer overrun does not occur. This is one instance where an interpreted language, such as Bash, provides more protection from programmer mistakes than a compiled language.

Command substitution permits setting a variable to the output of a loop. The key to this is grabbing the output of an echo command within the loop.

   1 #!/bin/bash
   2 # csubloop.sh: Setting a variable to the output of a loop.
   3 
   4 variable1=`for i in 1 2 3 4 5
   5 do
   6   echo -n "$i"                 #  The 'echo' command is critical
   7 done`                          #+ to command substitution here.
   8 
   9 echo "variable1 = $variable1"  # variable1 = 12345
  10 
  11 
  12 i=0
  13 variable2=`while [ "$i" -lt 10 ]
  14 do
  15   echo -n "$i"                 # Again, the necessary 'echo'.
  16   let "i += 1"                 # Increment.
  17 done`
  18 
  19 echo "variable2 = $variable2"  # variable2 = 0123456789
  20 
  21 #  Demonstrates that it's possible to embed a loop
  22 #+ within a variable declaration.
  23 
  24 exit 0

The $(...) form has superseded backticks for command substitution. 1 output=$(sed -n /"$1"/p $file) # From "grp.sh" example. 2  3 # Setting a variable to the contents of a text file. 4 File_contents1=$(cat $file1) 5 File_contents2=$(<$file2) # Bash permits this also. The $(...) form of command substitution treats a double backslash in a different way than `...`. bash$ echo `echo \\` bash$ echo $(echo \\) \ The $(...) form of command substitution permits nesting. [3] 1 word_count=$( wc -w $(ls -l | awk '{print $9}') ) Or, for something a bit more elaborate . . . Example 11-3. Finding anagrams 1 #!/bin/bash 2 # agram2.sh 3 # Example of nested command substitution. 4  5 # Uses "anagram" utility 6 #+ that is part of the author's "yawl" word list package. 7 # http://ibiblio.org/pub/Linux/libs/yawl-0.3.2.tar.gz 8 # http://personal.riverusers.com/~thegrendel/yawl-0.3.2.tar.gz 9  10 E_NOARGS=66 11 E_BADARG=67 12 MINLEN=7 13  14 if [ -z "$1" ] 15 then 16  echo "Usage $0 LETTERSET" 17  exit $E_NOARGS # Script needs a command-line argument. 18 elif [ ${#1} -lt $MINLEN ] 19 then 20  echo "Argument must have at least $MINLEN letters." 21  exit $E_BADARG 22 fi 23  24  25  26 FILTER='.......' # Must have at least 7 letters. 27 # 1234567 28 Anagrams=( $(echo $(anagram $1 | grep $FILTER) ) ) 29 # $( $( nested command sub. ) ) 30 # ( array assignment ) 31  32 echo 33 echo "${#Anagrams[*]} 7+ letter anagrams found" 34 echo 35 echo ${Anagrams[0]} # First anagram. 36 echo ${Anagrams[1]} # Second anagram. 37  # Etc. 38  39 # echo "${Anagrams[*]}" # To list all the anagrams in a single line . . . 40  41 # Look ahead to the "Arrays" chapter for enlightenment on 42 #+ what's going on here. 43  44 # See also the agram.sh script for an example of anagram finding. 45  46 exit $?

Examples of command substitution in shell scripts:

1. Example 10-7

2. Example 10-26

3. Example 9-30

4. Example 15-3

5. Example 15-20

6. Example 15-16

7. Example 15-50

8. Example 10-13

9. Example 10-10

10. Example 15-30

11. Example 19-8

12. Example A-17

13. Example 27-2

14. Example 15-43

15. Example 15-44

16. Example 15-45