Chapter 1: Hello World!

—
## Hello World: Our First Script

1 #!/usr/bin/env bash
2 echo "Hello World!"

Above script is stored in file welcome.sh
The shebang line /usr/bin/env bash tells which interpreter to use (in this case, bash) and also informs commands like file that welcome.sh is a Bash script.

Linux command file welcome.sh will show that it’s a bash script and an ASCII text executable. If we do not include the shebang line, the file command will not recognize that the file is a shell script.

It’s best practice to write the shebang line in a way that uses the shells environment to locate the bash executable. Not all systems have bash installed at the same place (/bin/bash for example). Using /usr/bin/env bash, tells whatever bash executable appears in current user’s $PATH variable.

Executing the script

—
Now that we have our first script, lets see how to execute it:

• Execute with bash welcome.sh. No need to set the execute permission while executing this way.
• To execute the script directly (i.e. ./welcome.sh ), we need to set the execute permission, chmod a+x welcome.sh

umask

—

When a new file or directory is created, the permissions of that new object are determined by the default permissions for the file or directory. umask controls what permissions are not given to newly created file or directory. It does not affect the permissions of existing objects, only newly created objects.

• The default Unix permission set for newly created directories is 777
• The default permissions for newly created files is 666

Without the umask, all new directories would be created with full 777
permissions, and all new files would be created with full 666
permissions. The umask blocks certain permissions from being given to newly created file system objects.

Every bit set in the umask “masks”, or “takes away”, that permission from the default. “Mask” does not mean “subtract”, in the arithmetic sense.

Below is the octal representation of permissions:

• 4 for Read
• 2 for Write
• 1 for Execute

Shell command umask 022 sets to —-w–w- permissions to be removed from the default permissions. Therefore, setting umask to this value will result in a file having 644 (rw-r—-r—) permission and directory having 755 (rwxr-xr-x) permission.

Using the chmod command without specifying whether you want to change User, Group, or Other permissions causes chmod to use your umask to decide what sets of permissions to change. If you want everyone (user, group, others) to have execute permissions, use chmod a+x file_name.sh.

Adding file to $PATH

—

In the $PATH environment variable, include the location where your script is and you can execute it by just specifying the name (welcome.sh). Else, use ./welcome.sh

Positional Parameters

—

• $1, $2, $3....${10}: Upto 9, no need to use braces (we can if we want to). From 10 onwards, use braces to indicate it’s number ten and not 1 followed by 0.
• $0 is NOT positional and represents the script name.
• $# represents the number of positional parameters that have been supplied.
• $* will list all parameters as a list, we can iterate through the same.

Chapter 2: Variables

—

Quotations

—

• Single Quotes: Bash will treat as literal text, it will not attempt any substitutions, replacements etc.
• Double Quotes: When you wrap up text in double quotes, bash will still interpret substitutions, expansions, evaluations,variables, and so on, but it will be reasonable about not trying to do that when it doesn’t it make sense to.

Setting Variables

—
Syntax to set variables: var_name=value

• Make sure there is NO space around the assignment operator.
• Use lowercase for variable names, to distinguish them from environment variables.
• If the value has spaces in it, enclose it in double quotes.
• Variable names are case sensitive.
• Variable name in bash must begin with a letter or an underscore, and can be followed by any letter or number, or more underscores.
• Variables can be changed, re-assigned during the course of the script.

Specifying Default Values

—

If users do not provide the required inputs, we can provide default values to variables:

1 # 2nd positional parameter is password
2 # If user doesn't provide the value, then Password1 is de\
3 fault
4 user_name="$1"
5 user_password="${2:-Password1}"

Reading Variables

—
Syntax to read variables: "$var_name"

Best practice is to quote the variables - this is to protect elements like spaces within our variables.

If you expand a variable that doesn’t exist, Bash won’t issue any warnings and just replace the variable with an empty string. If you want Bash to throw an error whenever an unset variable is expanded, use set -u

To Brace or Not to Brace?

—
If we need to push other characters against our variable name, use braces "${var_name}"
If you need to distinguish the variable name from characters around it, you can use curly braces, { } - they are optional, but serve to protect the variable to be expanded from characters immediately following it which could be interpreted as part of the name.

In the below example, since the variable name was not delimited, the _ character was considered part of the variable name - the shell tried to expand the non-existent $a_, which does not exist and therefore nothing was returned. Wrapping the variable with curly braces solves this problem:

1 a="hello"
2 echo $a_
3 # will print a blank line
4 echo ${a}_
5 # will print hello_

We will use braces for most parameter expansions, because if we left them off, the shell would just interpret the part after the parameter name, as characters and show them instead of using them to do what we intend.

Built-in Variables

—

• command -V echo will show that echo is a built-in, which will take precedence over other commands.
• enable -n echo will run command version (/bin/echo) of echo instead. I can re-enable it with enable echo
• enable -n will show the builtins that are disabled
• echo string will output the string with a newline. This is the builtin version. I can mention builtin echo string to specify that I want to run builtin version, or command echo string for command version.
• Builtins use a different documentation that regular man pages. There’s a builtin called help that shows supporting information about builtins. Example, help echo
• help will show a list of all builtins.

Environment Variables

—

Variable assignments as done above are only visible in the current shell - their values don’t affect a program called by a script (i.e. a forked process)

If you want a way for variables to be visible and usable by any of the programs you call on the script or command line, you need to use environment variables using the export builtin command.

1 $ myvar="John"
2 $ export myvar
3 $ declare -p myvar
4 declare -x myvar="John"

• The -x in the output tells that the variable has been exported to the environment.
• You can use declare -x to get a list of all environment variables and their values.
• We can also use env and will need to sort its output to read it.

You should only export variables when you actually need to use their values in subprocesses - you risk overwriting an important environment variable used by a program without intending to, which can be hard to debug.

declare Command

—

declare is a shell built-in, and is used to declare shell variables and functions, set their attributes and display their values.

1 declare -- today='August 9th'

The -- in above example tells that there’s no special attribute associated with the variable

• declare -r myvar=value will make the variable read only and cannot be changed later on.

Below options allow us to turn the characters in the value to lower or uppercase. Even when the value of the variable is changed later on, it will be transformed to lower or uppercase. These help us normalize user inputs for consistence.

• declare -l myvar=value will turn the characters in the value to lowercase.
• declare -u myvar=value will turn the characters in the value to uppercase.

declare -p will show all variables set in the current session.

Chapter 3: Prompting for input

—

read can be used to prompt for user input:

1 read -p "Prompt: " var_name
2 
3 # Use -s to silence screen input for passwords, example
4 read -sp "Enter password: " user_password

If we do not supply the variable name to the read command, the value will be stored in $REPLY

Another way to read is using heredoc notation, which we will see later in the book:

1 read var1 var2 <<< "Hello World"
2 echo $var1
3 echo $var2

If we use the -n option followed by an integer, we can specify the number of characters to accept before continuing. For example, if you want to read only 1 character, use read -n1 - there’s no need for the user to hit enter after providing the input.

We can also accept user input in an array with the read -a flag. Note that indexing is 0-based:

1 read -a array_var name age email
2 # ${array[@]} will loop through all variables
3 echo "Your name, age, email are: ${array_var[@]}"
4 
5 # Iterate through indexes 0 to 1
6 echo "Your name and age are: ${array_var:0:1}"
7 
8 # Access value at index 2
9 echo "Your email is ${array_var[2]}"

Chapter 4: Heredoc

—

When writing shell scripts you may be in a situation where you need to pass a multiline block of text or code to an interactive command. A Here document (Heredoc) is a type of re-direction that allows you to pass multiple lines of input to a command. It uses a token word or delimiter to specify where a document for a command’s standard input should finish. Below is the syntax:

1 [command] <<[delimiter] 
2 Here Document
3 delimiter

Below is an example of a heredoc:

1 cat <<EOF
2 Current working directory: $PWD
3 You are logged in as $(whoami)
4 EOF

• Any string can be used as a delimiter, EOF and END are most common.
• If delimiter is unquoted, shell will substitute variables, commands and special characters before passing the here-document lines to the command.If you want to expand variables or do command substitution inside a here-document, you can leave out the single quotes around the delimiter.
• If you really want to indent your here-documents, you can include a hyphen between << and the delimiter, i.e. <<-[delimiter] the tabs (but not spaces) at the front of your input lines will be ignored. Use this when your heredoc is inside an if statement for example.

Since bash does not support multiline comments, we can use heredoc to overcome this. If you are not redirecting heredoc to any command, the interpreter will simply read the block of code and will not execute anything:

1 <<- COMMENT
2 	This is a comment line 1
3 	This is a comment line 2
4 	This is a comment line 3
5 COMMENT

Heredoc output can be sent to a file or be piped to another command.

Chapter 5: Comparing Values

—

Bash has a built-in command called test, which is also represented by single brackets, [ ].

For example, [ -d ~ ] will test if my home directory is a directory. It will return an exit status of 0 (success) or non-zero number (fail). We can use the value of $? to read the value of return status.

Comparing Numerical Values

—

To test numbers, use eq, -ne, -lt, -le, -gt, -ge.

1 if [ $# -eq 0 ]; then
2 	read -p "Enter a username: " user_name
3 else
4 	user_name="$1"
5 fi

Comparing Strings

—

Use == and != to compare strings. We must use a single space before and after these operators.

 1 "$string1" == "$string2"
 2 "$string1" != "$string2"
 3 
 4 # Example
 5 if [ "$password" != "$password_check" ]; then
 6 	echo "Passwords do not match"
 7 	# Exit to ensure nothing else is executed: 1 is Error, 0\
 8  is for success
 9 	exit 1
10 fi

Extended Test

—

Extended test uses double brackets, [[ ]], and allows us to use more than one expression within a test, for creating a little bit more complex logic.

For example, below, we are checking if my home directory is a directory and whether the bash binary exists:

1 [[ -d ~ && -a /bin/bash ]]; echo $?

Similarly, we can use || for OR operation, which executes the next command only if the previous command fails.

Extended test also allows us to use regular expression. Below we are testing if “cat” starts with c: [[ "cat" =~ c.* ]]; echo $?

It is recommended to use extended tests as a best practice.

Chapter 6: Working with numbers

—

• Bash can do integer math, not decimal or fractional.
• Supports 6 operators, + - * / % **
• $((...)) - Is an Arithmetic Expansion, and returns the result of the mathematical operation. This does NOT modify the existing variable.
• ((...)) - Is an Arithmetic Evaluation and performs calculations and changes the values of existing variables. They do not provide any return value, but an exit status instead.
• When a variable is referenced inside the double parentheses, whether for arithmetic expansion or arithmetic evaluation, it does not need to be prefaced with a dollar sign.

1 echo $((8 + 8))
2 # returns 16
3 a=3
4 ((a+=3))
5 echo $a # will return 6
6 ((a++))
7 echo $a # will return 7

We can use declare -i b=3 to tell Bash that the variable is an integer and not treat it as a string. It’s a good idea to use declare when we know the variables we use will be integers.

1 a=3
2 a=$a+2
3 # will print 3+2 as bash is treating the variable as a st\
4 ring
5 echo $a 
6 declare -i b=3 # declare b as an integer
7 b=$b+2
8 # will return 5
9 echo $b 

echo $RANDOM will return a pseudo-random value between 0 and 32,767. Use $(( 1 + $RANDOM%20)) to get a random number between 1 and 20.

Chapter 7: Functions

—

Declaring Functions

—

1 function_name () {
2 
3 	# code block
4 }
5 
6 # Calling the function
7 function_name

Function Parameters

—

We may send data to the function in a similar way to passing command line arguments to a script. We supply the arguments directly after the function name. Within the function they are accessible as $1, $2.. etc

 1 #!/usr/bin/env bash
 2 
 3 prompt_user () {
 4 
 5 	message={1:-"Hello World"}
 6 	echo "$message"
 7 }
 8 
 9 # No argument provided - default value for message will b\
10 e used
11 prompt_user
12 
13 # Argument provided
14 prompt_user "How are you?"

Return Values

—

Unlike programming languages, Bash does not allow functions to return values - it does however allow us to set a return status. Similar to how a program or command exits with an exit status which indicates whether it succeeded or not. We use the keyword return to indicate the return status:

• Typically a return status of 0 indicates that everything went successfully.
• A non zero value indicates an error occurred.

Chapter 8: Loops

—

for Loop

—

There are 2 variations of for loop in Bash:

Below is a type of for loop which is characterized by counting. The range is specified by a beginning and end:

1 for user in $*; do
2 	echo $user
3 done

A second variation involves a three-parameter loop control expression; consisting of an initializer (EXP1), a loop-test or condition (EXP2), and a counting expression/step (EXP3).

1 for (( INITIALIZER;CONDITION;STEP ))
2 do
3 	commands
4 done

while Loop

—

1 while [ "$password" != "$password_check" ]; do
2 	echo "Passwords do not match"
3 done

Chapter 9: Conditional Logic

—

if - else Statement

—

Below is the syntax:

1 if [[ condition ]]; then
2 	code
3 elif [[ condition ]]; then
4 	code
5 else
6 	code
7 fi

• There must be a space between the opening and closing brackets and the condition you write. Otherwise, the shell will complain of error.
• There must be space before and after the conditional operator (=, ==, <= etc). Otherwise, you’ll see an error like "unary operator expected".

Useful Unary Operators

—

Below are useful unary operators we can use with if statements:

Avoiding if-else Statements

—

1 # Using if-else
2 if [ "$EDITOR" = "" ]; then
3 	EDITOR=nano
4 fi
5 
6 # Not using if-else
7 [ -z "$EDITOR" ] && EDITOR=nano

Chapter 10: Expansions and Substitutions

—

Tilde Expansion

—

• ~ represents $HOME environment variable
• ~- represents $OLDPWD, and will show the previous directory you were in (if you had recently changed directories).

Brace Expansion

—

It is used to generate strings at the command line or in a shell script. For example, touch file{0..12} will create file1, file2,.., file12.

Parameter Expansion

—

We’ve seen this while reading variables - parameter expansion lets us recall stored values:

1 a="Hello World"
2 echo ${a}
3 echo $a

Pattern Substitution

—

It is a special case of parameter expansion.

1 greeting="hello there"
2 # will replace there with everybody
3 echo ${greeting/there/everybody}

Command Substitution

—

Allows us to use the output of a command within another command. For example,
echo "$(uname -r)".

Chapter 11: Formatting output with printf

—

printf is a bash built-in and does not add a newline by default.

1 printf "The results are %d and %d\n" $((2+2)) $((3/1))

We can enclose variables within the printf statement:

1 myvar=Linux
2 printf "$myvar is fun to work with.\n"

When you enclose your arguments with single quotes the variable and command will be treated as plain text. You have to enclose the arguments with double quotes if you want the variable and command to be expanded.

We can assign printf output to a variable in 2 ways:

1 # Method 1
2 release=$(printf "$(uname -r)")
3 printf release
4 
5 # Method 2
6 printf -v release "$(uname -r)"
7 printf release

The -v option causes the output to be assigned to the variable var rather than being printed to the standard output.

Width Modifiers

—

1 printf "%20s" Ronaldo
2 #             Ronaldo

Since Ronaldo is 7 characters, and the specified width is 20, it will add spaces to justify the width.

We can use a - to justify the alignment:

1 printf "%-10s" Ronaldo
2 #Ronaldo

For integers, we can replace the space with zeros by adding a 0 flag modifier:

1 printf "%010d" 7
2 #0000000007

Pricision Modifiers

—

It is used to decide the number of characters to be printed, use the . followed by number of characters:

1 printf "%.7s \n" "Ronaldo has joined Manchester United"
2 #Ronaldo

We can also use * to specify the precision:

1 printf "%.*s \n" 7 "Ronaldo has joined Manchester United"
2 #Ronaldo

Chapter 12: Arrays

—

Bash supports 2 kinds of arrays:

• Indexed Arrays: Set or Read values by referring to their position (using an index). Indexing is 0-based.
• Associative Arrays (Bash v4 & above): These are key:value pairs
• We cannot create nested arrays in Bash.

Defining Arrays

—

1 # Defining arrays: 2 ways
2 snacks=("apple" "banana" "orange")
3 declare -a stationery=("paper" "stapler")

Retrieving Values

—

1 # Retrieve Values - this is parameter expansion
2 echo ${snacks[2]}

Setting Values

—

1 # Set values by index number
2 snacks[5]="grapes"

Appending to an array

—

1 # Add to end of array
2 snacks+=("mango")

Length of an array

—

${#array_variable[@]} will retrieve the length

Looping through an array

—

1 for item in "${array_var[@]}"
2 do
3     # code
4 done

Associative Arrays

—

1 # Create an associative array
2 declare -A office
3 
4 # Set values
5 office["buillding name"]="HQ West"
6 
7 # Accessing values
8 echo ${office["building name"]}

Chapter 13: Case Statement

—

Case statement is generally used to simplify complex conditionals when you have multiple different choices, and is preferred over nested if-else statements.

Syntax

—

 1 case EXPRESSION in
 2     
 3     PATTERN_1)
 4         # code block
 5         ;;
 6     PATTERN_2)
 7         # code block
 8         ;;
 9     
10     PATTERN_N)
11         # code block
12         ;;
13     *)
14         # code block
15         ;;
16 esac

• The ) operator terminates a pattern list.
• Each clause must be terminated with ;;
• It is a common practice to use the wildcard (*) as a final pattern to define the default case. This pattern will always match.
• If no pattern is matched, the return status is zero. Otherwise, the return status is the exit status of the executed commands.

Example

—

 1 while true; do
 2 	clear
 3 	echo "Chose 1, 2 or 3"
 4 	echo "1: See logged in users"
 5 	echo "2: Date in 90 days"
 6 	echo "3: Quit"
 7 	read -sn1
 8 	case "$REPLY" in
 9 		1) who;;
10 		2) date --date="90 days";;
11 		3) exit 0;;
12 		*) echo "Incorrect option selected. Please select again"
13 	esac
14 	read -n1 -p "Enter any key to continue"
15 done

Chapter 14: Extracting Substrings

—

Using Bash Substring Expansion

—

• Syntax: ${str_var:offset:length}
• Start extracting from the offset, up to the specified length.
• The offset is used to specify the position from where to start the extraction of a string. The length is used to specify the range of characters to be extracted, excluding the offset
• Note: Indexing is 0-based

1 # Example
2 u="Hello World"
3 # Start from index 0 and extract upto length 5
4 echo ${u:0:5} # output: Hello

Using IFS

—

IFS (Internal Field Separator), is an environment variable that determines how Bash recognizes word boundaries while splitting a sequence of character strings. Default values are space, tab and newline.

In the below example, we have set the IFS to an underscore, which is what will be used to split the numbers variable with underscore as the delimiter. Once split, we can access the words using $1, $2 and so on.

1 # Example 1
2 
3 numbers=one_two_three_four_five
4 IFS="_"
5 
6 # Setting positional parameters $1, $2 and so on
7 set $numbers
8 
9 echo $2 #output will be two

 1 # Example 2: using space as IFS
 2 
 3 greeting="Hello World"
 4 IFS=" "
 5 
 6 # Save the split string in an array
 7 read -a array_var <<< "$greeting"
 8 
 9 echo "There are ${#array_var[@]} words in the string"
10 
11 for word in ${array_var[@]}:
12 	do
13 			printf "$word\n"
14 done

Using cut

—

• -d option allows us to specify the delimiter
• -f sets the field to be extracted

1 numbers="one_two_three_four_five"
2 substring=$(echo $numbers | cut -d "_" -f 3)
3 echo $substring # will output three

Chapter 15: Debugging Scripts

—

• If we want to look at the verbose output from our script and the detailed information about the way the script is evaluated line by line, we can use the -v option, bash -v script_name.sh
• The -x option, which displays the commands as they are executed, is more commonly used. Traces of each command plus its arguments are printed to standard output after the commands have been expanded but before they are executed.