spO0q 🐒🎃Basics
Starting somewhere... Be careful, some command might require bash 4 (the vast majority of snippets are for bash 3, though).
Bash is an acronym
Bash stands for "Bourne Again Shell", which refers to "born again shell". Indeed, Bash has replaced the old Bourne shell initially released in 1979.
Bash is a "Unix-versal" scripting language
Bash is a crazy good language you can learn to automate various tasks whether it's for your personal use, for server maintenance, or in devOps actions. Bash scripting may consist of writing a series of command lines in a plain text file.
Note that you can type these commands on your terminal as well. You can use existing commands such as grep, sed, find but also additional commands from third-party libraries you might have installed.
However, it's often more convenient to group all instructions in files you can git and transfer from one machine to another or run in multiple servers. Bash scripts are usually extension-less files you can execute this way:
bash myscript
Alternatively, you can make your script executable and run it:
chmod +x myscript && ./myscript
Bash is not sh!
While Bash can run most sh scripts (the syntaxes are quite similar), sh cannot handle all Bash commands.
The following command lines will probably work but they are a bit misleading:
bash mysh.sh
sh myscript
Don't forget the Shebang
It's the #! at the top of bash files. "Bang" is for the exclamation point "!":
#!/usr/bin/env bash
The above line is not decorative at all! It links the script to the absolute path of the Bash interpreter. /usr/bin/env is usually a smart choice as it grabs the executable set in the user's $PATH variable.
Writing comments
Use a #:
# comment
Variables
Variables are easy to declare, use, and manipulate in Bash.
Built-in variables (not an exhaustive list)
echo "$1 $2" # positional arguments passed one by one, e.g., bash myscript arg1 arg2
echo "$#" # number of arguments passed
echo "$@" # all arguments passed
echo "You are here: $PWD" # $PWD is the current path
Assigning new variables
You assign variables with the = sign and no space around:
MyVar="My string"
MyArray=(all in one)
Using variables
You can use declared variables with the $ sign:
echo $Variable
echo "This is my var: $Variable" # double quotes are required for interpolation
echo "${MyArray[@]:0:3}" # "all in one", because it prints 3 elements starting from the first (0)
# Be careful the following does not display a value
echo ${#MyArray[2]} # "3" as there are 3 chars in the third element of the array, which is the word "one"
Modifying variables
You can modify declared variables with various signs enclosed with {}:
MyVar="My variable"
# Length
echo ${#MyVar}
# Substitution
echo ${MyVar//a/O} # "My vOriOble"
# Expansion
echo ${!BASH@} # all variable names than begin with "BASH"
echo ${!BASH*} # all variable names than begin with "BASH"
# Removals
MyFile="list.txt"
echo ${MyFile%.*} # the filename without extension
echo ${MyFile##*.} # the file's extension
# Default value
echo ${AnotherOne:-"Another one"} # displays "Another one" even if AnotherOne is not defined
Dictionaries
declare -A Movies
Movies[title]="The Dark Knight Rises"
Movies[bestActress]="Anne Hathaway"
Movies[director]="Christopher Nolan"
printf
printf is the C-like way to print preformatted text. It provides way more advanced features than a simple echo. It takes formats and parameters as arguments and prints the formatted text.
For example, you can specify the type and some formatting option in one line:
printf "%s\n" "Learn it" "Zip it" "Bash it" # it prints 3 lines
printf "%.*f\n" 2 3.1415926 # prints 3.14
Loop, loop, loop
For
for i in "${MyArray[@]}"; do
echo "$i"
done
It works with strings too:
MyString="First Second Third"
for n in $MyString
do
echo "$n line"
done
While
while [ true ]
do
echo "We loop"
break
done
Handy ranges
echo {1..7} # 1 2 3 4 5 6 7
echo {a..g} # a b c d e f g
Until
until [ $counter == 111 ]
do
echo "Number is: $((counter++))"
done
Get user inputs
echo "6x7?"
read answer
echo answer # hopefully 42 but can be anything the user wants
Conditional statements
if [[ CONDITION1 || CONDITION2 ]]
then
# code
elif [[ CONDITION3 && CONDITION4 ]]
then
# code
else
# code
fi
Alternatively, you might use a case statement:
case $COLOR in
Red)
echo -n "red"
;;
Blue | Green)
echo -n "blue or green"
;;
Yellow | "Sunny" | "Gold" | "Fire")
echo -n "Yellow"
;;
*)
echo -n "What?"
;;
esac
Errors & exit strategies
It's best if you can raise errors to prevent any misuse of your scripts.
Exit immediately if a command failed
#!/usr/bin/env bash
set -e
Exit N
You can exit the script and specify an exit code:
#!/usr/bin/env bash
if [ CONDITION ]; then
exit 0 # 0 1 2 3 ... N
fi
0 indicates a success whereas {1,2,3, N} are errors.
Test errors
# $? is the exit status of last command
if [[ $? -ne 0 ]] ; then
echo "You failed!"
exit 1
else
echo "You are the best!"
fi
Debug bash scripts
#!/usr/bin/env bash
set -x # set -n can also be used to check syntax errors
ARGS=("$@") # store all script args in a var
You can also execute the script with the -x option in the terminal:
bash -x myscript
Filesystem & directories
You can use all basic filesystem commands such as cp, rm, ls, mv or mkdir in your bash scripts.
To check if a file is executable before anything:
if [ ! -x "$PWD/myscript" ]; then
echo "File is not executable!"
exit 1
fi
Use -f for files, -d for directories, -e for existence, etc. There are tons of options, you can even test symbolic links with -L and compare files by date with the -nt (newer than) and -ot (older than) options.
A bit more advanced usages
Here are some tips I consider a little more complicated than the basic ones.
Leverage the benefits of the bash syntax
Bash is pretty convenient to build output fast, especially using brace expansions:
echo {1..10}{0,5}h # bash 3
echo {10..120..10}km # requires bash 4
Define functions
function test ()
{
echo "$1 $2" # displays positional arguments passed to the function ("All" "In")
return 0 # if you return a value, it !must be numerical!
}
test "All" "In"
Scopes
You don't have to declare variables in Bash. If you call a variable that does not exist, you don't get any error but an empty value.
Environment variables
Env vars are defined at the system level. They exist everywhere, in all processes.
To list them, use printenv:
printenv USER # if you don't specify a variable, all variables will be displayed
Shell variables
The following assignment defines a shell variable:
TEST="My test"
You can use it directly everywhere within the script, including in functions. However, they're not passed to child processes unless you use the export command to transmit such information.
The local keyword
The local keyword can only be used within functions:
function test ()
{
local MyVar="My string" # local variable
}
test
You cannot export a local variable.
Best practices
As a general rule, it's better to use local variables and functions. Global variables should be used meaningfully. It prevents unwanted overriding and confusions.
Loop through the results of a command
for r in $(ls $PWD)
do
# task using the result of the command => r
echo "$r"
done
Use the output of a function
function hello() {
echo "Hello Ethan"
}
echo "This is the result of the function : $(hello)"
Store the result of a command in a variable
Users=$(cat users.txt)
Capture yes/no inputs
read -p "Continue? [y/n]: " -n 1 -r
echo # extra line
if [[ $REPLY =~ ^[Yy]$ ]]
then
echo "ok"
elif [[ $REPLY =~ ^[Nn]$ ]]
then
echo "Too bad!"
else
echo "Sorry, I did not understand your answer :("
fi
Capture user's selection
The following bash code prompts a selection to the user and capture the anwser:
select w in "zsh" "bash" "powershell"
do
echo "You prefer $w"
break
done
Aliasing and overriding commands
You can create special aliases:
alias lz='ls -alh'
You can also override existing commands simply by redefining them.
Chaining commands
wget -O /tmp/logo.jpg 'https://dev-to-uploads.s3.amazonaws.com/uploads/logos/resized_logo_UQww2soKuUsjaOGNB38o.png' && echo "echo only if the first hand is true"
wget -O /tmp/logo2.jpg 'https//example.com/logo.jpg' || echo "echo only if the first hand is wrong"
wget -O tmp/logo3.jpg 'https//example.com/logo.jpg' ; echo "echo whatever happens"
The hard mode
Here some advanced concepts in Bash you might find useful.
Execute last executed command
sudo !!
Redirections, stderr, stdout
COMMAND > out.txt # write in out.txt
COMMAND >> out.txt # append in out.txt
COMMAND 2> test.log # stderr to test.log
COMMAND 2>&1 # stderr to stdout
COMMAND &>/dev/null # stdout and stderr to (null)
Replace COMMAND with your command.
It's a Trap!
The trap command is an advanced approach of errors. The key to understand it is to forget about exceptions and similar concepts you might know in other languages.
The idea is to execute something only in specific conditions:
#!/usr/bin/env bash
trap 'catch' EXIT
catch() {
echo "Exit game!"
}
echo "ok"
echo "---"
exit 0
The message "Exit game!" displays only because of the exit 0 at the end.
More pragmatically, it's not uncommon to see that kind of usage:
#!/usr/bin/env bash
trap "rm /tmp/scan.txt" EXIT
Execute scripts inside a Bash script
Use the source command:
#!/usr/bin/env bash
source ./otherScript
Subshells
A subshell is a separate instance of the command processor. A shell script can itself launch subprocesses. These subshells let the script do parallel processing, in effect executing multiple subtasks simultaneously.
In other words, commands enclosed inside parenthesis execute in a subshell. It partly explains the use of the export command that passes environment variables on to sub-processes.
You sometimes read the term "forks" instead, but it's the same idea.
pipefail
You might use the following as a more advanced error handling:
#!/usr/bin/env bash
set -eu # u is to exit if using undefined variables
set -o pipefail
The above set commands not only enable "error mode" but also prevent any error in pipeline from occurring silently, which is the default behavior if you don't set the -o pipefail option.
My favorite tricks in the terminal
The following commands might be more Unix-based than pure bash scripting, but it's not completely off topic 🤞🏻.
Use bash
You might have install third-party solutions for terminal prompt or use zsh. Just type:
bash
If you press enter, you will now use the bash prompt. Then you might type man bash to display the help.
Use \
You can use \ at the end of long command lines to improve readability and not lose your focus.
Use &
nohup bash script &
The above allows executing bash scripts in the background and catch the hangup signal.
Misc
Some tricks I use frequently:
List sub-directories only in the current folder
SUBDIRS=$(ls -d */)
for sub in $SUBDIRS
do
echo $sub
done
Quick rename a file
mv /project/file.txt{,.bak} # rename file.txt.bak
Execute local bash script on a remote server
ssh REMOTE_HOST 'bash -s' < myScript
Find heavy files (larger than 10MB)
find . -size +10M -print
Create multiple files quickly
You can type:
touch {script1,script2,script3,script4,script5,script6,script7}
But that's still a bit tedious, so use this:
touch script{1..7}
Bash for hackers
Hackers would bash for sure. While Python is amazingly convenient for pen-testing, bash can also help automate many tasks. It's not uncommon for professionals to create custom bash scripts to speed up analysis.
Hello world: scanning script
Instead of repeating the same command lines over and over, you can group them into specific files. Most security binaries are built like that.
You might use the read command to make your script a bit more friendly to use or just pass arguments. Let's name our script scanning
#!/usr/bin/env bash
echo "Welcome aboard"
echo "What's the targeted IP?"
read TargetIp
if [[ ! $TargetIp ]]
then echo "Missing targeted IP!"
exit 1
fi
echo "What port do you want to scan?"
read PortNumber
if [[ ! $PortNumber ]]
then echo "Missing port number!"
exit 1
fi
nmap $TargetIp -p $PortNumber >> scan.txt # append results in scan.txt
Then we can run bash scanning. Of course, there are existing tools to ease the use of nmap and it's a very incomplete example, but you see the idea.
Aliasing
Attackers would likely use aliases when using advanced commands in the terminal:
alias whatsMyIp="echo $(ifconfig -a | grep broadcast | awk '{print $2}')"
whatsMyIp # IP local
Pen-testing distribution may have better autocompletion features, though, but you don't always operate in such optimized environment, so aliases can save a lot of time and energy.
Hacking bash
Hackers can hack bash like any other language. Just read this post by hacktricks.
