123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297 |
- # BASH reference
- Bash reference manual:
- www.gnu.org/software/bash/manual/bashref.html
- Advanced Bash-Scripting Guide:
- http://www.tldp.org/LDP/abs/html/
- man bash # manual page for bash
- What Bash is good for
- * File and directory management
- * Systems management (build scripts)
- * Combining other scripts and commands
- * Rapid prototyping of more advanced scripts
- * Very simple output processing, plotting
- Some common tasks in Bash
- * file writing
- * for-loops
- * running an application
- * pipes
- * writing functions
- * file globbing, testing file types
- * copying and renaming files, creating and moving to directories, creating
- directory paths, removing files and directories
- * directory tree traversal
- * packing directory trees`
- # shellcheck - static analysis tool for shell scripts
- $ doas apt-get install shellcheck
- $ shellcheck file.sh
- Reference:
- https://www.shellcheck.net/
- https://www.shellcheck.net/wiki/Home
- # shfmt - tool for formatting, parsing, and interpreting shell script
- $ doas apt-get install shfmt
- $ shfmt file.sh # print the formatted script to the terminal
- $ shfmt -indent 4 file.sh # number of spaces that we want to use for indentation
- $ shfmt -diff file.sh # changes between the unformatted and the formatted script
- $ shfmt -list *.sh # list all the files that need formatting
- $ shfmt -write file.sh # print the formatted script to the file itself
- Bash example 1: hello world
- #!/bin/bash
- echo "Hello World"
- Two options to run this script:
- 1. Type the commands directly in the bash shell (only feasible for small scripts)
- 2. Save the code as helloworld.sh and run with:
- chmod a+x helloworld.sh # Make script executable
- ./helloworld.sh
- Bash example 2: hello world
- #!/bin/bash
- x="World" # no space between them, if space is present its an error
- echo "Hello ${x}!" # ${x} - variable substitution
- Bash has a number of built in commands, type help or help | less to see all.
- Bash variables:
- Variables in bash are untyped
- >>> x=5
- Generally treated as character arrays
- >>> x=5
- >>> x=$x+1 # concatenates the text
- >>> echo $x
- 5+1
- Use the let command for simple arithmetic and other operations:
- >>> x=5
- >>> let "x+=1"
- >>> echo $x
- 6
- Varibles can be explicitly declared to integer or array:
- >>> declare -i i # -i is argument and i is name of the variable, i is an integer
- >>> declare -a A # A is an array
- >>> r=10
- >>> declare -r r # r is read only, means the value of r cannot be changed, its constant
- The echo command is used for writing:
- >>> s=42
- >>> echo "The answer is $s"
- and variables can be inserted in the text using (variable interpolation)
- Frequently seen variables:
- Command line arguments:
- $0 # Name of script
- $1 # First command line argument
- $2 # Second command line argument
- # ...
- All the command line arguments: $@
- Number of command line arguments: $#
- The exit status of the last executed command:
- $? # returns 0 if the last command was success
- Example:
- Write a script that takes a command as an argument, runs it and checks if it was
- succesfull.
- # run_and_test.sh
- $@
- if ["$?" == "0"]; then
- echo "Hurray, everything went fine."
- else
- echo "Oops, there was an error."
- fi
- To execute the above script: >>> ./run_and_test.sh ls Programming/*.c
- Comparison of two integers use a syntax different from comparison of two strings:
- if [$i -eq 10]; then # integer comparison
- fi
- if ["$name" == "10"]; then # string comparison
- fi
- Unless you have declared a variable to be an integer, assume that all variables are
- strings and use double quotes (strings) when comparing variables in an if test.
- Executing a command, storing the result as a variable can be done in two ways:
- time=$(date)
- time=`date`
- Convenient debugging tool: -x
- Each source code line is printed prior to its execution if you add -x as option to
- /bin/sh or /bin/bash
- Either in the header: #!/bin/bash -x
- or on the command line: >>> bash -x hw.sh
- Very convenient during debugging
- Combining bash commands:
- * The power of Unix lies in combining simple commands into powerful operations
- * Standard bash commands and unix applications normally do one small task
- * Text is used for input and output - easy to send output from one command as input
- to another
- Two standard ways to combine commands:
- The pipe, sends the output of one command as input to the next:
- >>> ls -l | grep 3331
- Will list all files having 3331 as part of the name
- Send files with size to sort -rn (reverse numerical sort) to get a list of files
- sorted after their sizes:
- >>> ls -s | sort -rn # s - size
- Make a new application: sort all files in a directory tree assignments, with the
- largest files appearing first, and equip the output with paging functionality:
- >>> du -a assignments | sort -rn | less
- Bash redirects:
- Redirects are used to pass output to either a file or stream.
- echo "Hei verden" > myfile.txt # Save (stdout) output to file
- echo "Hei verden" >> myfile.txt # Append (stdout) output to file
- wc -w < myfile.txt # Use file content as (stdin) command input
- ls > files && grep 2017 < files is equivalent to ls | grep 2017
- cat INF3331-$username | ./test is equivalent to ./test < INF3331-$username
- The in-/outputs of a shell process: stdin, stdout and stderr
- A process takes standard input (STDIN) and returns
- * standard output (STDOUT)
- * standard error (STDERR)
- * return code - 0 on success, a different number otherwise
- ~~~~~~~~~~~~~
- STDIN | |---------> STDOUT
- --------->| Process |---------> STDERR
- | |---------> Return Code
- ~~~~~~~~~~~~~
- Redirecting process streams: (-v : verbose, explain what is being done)
- rm -v *.txt # stdout and stderr are displayed on the terminal
- rm -v *.txt 1> out.txt # redirect stdout to a file, same as > (document which files had been deleted)
- rm -v *.txt 2> err.txt # redirect stderr to a file
- rm -v *.txt &> outerr.txt # redirect stdout and stderr to file
- You can print to stderr with: echo "Wrong arguments" >&2 # redirect output as stderr
- Redirects and pipes can be combined:
- ./compile 2>&1 | less # view both stdout and stderr in less
- Example scripts:
- case="testbox"
- cmt="WinslowRice"
- if [ $# -gt 0 ]; then # if user provides an argument else use default
- cmt=$1
- fi
- infile="ellipsoidtest.i"
- outfile="outtext.txt"
- ./pulse app -cmt $cmt -cname $case < $infile | tee $outfile
- dirname=$(pwd)/$(date)
- mkdir $dirname && cd $dirname
- # mktemp - create a temporary file or directory
- # check directory exists
- if [ -d $dirname ]
- then
- echo "directory exists"
- exit # exit the script
- fi
- mkdir $dirname && cd $dirname
- # if statement in bash
- if [ -d $dirname ];
- then
- exit
- fi
- if test -d $dirname; then
- exit
- fi
- # short form of if statement
- [ -d $dirname ] && exit
- test -d $dirname && exit
- # multi-line commands (multi-line text is directed to filetext.txt)
- cat > filetext.txt << EOF
- multi-line text
- can now be inserted here,
- and variable substitution such as
- $variable is
- supported.
- EOF
- # parsing command-line options
- while [ $# -gt 0 ]
- do
- option=$1; # load command-line arg into option
- shift; # shift the command-line arg
- # some commands
- done
- # alternative to case statement is if statement
- if [ "$option" == "-m" ]; then
- m=$1; shift;
- elif [ "$option" == "-b" ]; then
- b=$1; shift;
- else
- echo exit;
- fi
- # add a check for successful execution
- # the shell variable $? is 0 if command was successful, otherwise $? != 0
- if [ "$?" != "0" ]; then
- echo "application failed"; exit 1 # exit n sets $? to n
- fi
- For loops:
- ./run.sh test1.i test2.i test3.i test4.i
- or
- ./run.sh *.i
- or
- for arg in $@; do
- # scripts
- done
- A for-loop over command line arguments.
- for loops for file management:
- files=`ls *.tmp`
- for file in $files
- do
- echo removing $file
- rm -f $file
- done
- # counters
- declare -i counter # declare an integer counter
- counter=0
- ((counter++)) # arithmetic expressions must appear inside (( ))
- echo $counter # yields 1
- # for loop with counter
- declare -i n; n=1
- for arg in $@; do
- echo "command-line argument number $n is <$arg>"
- ((n++))
- done
- C-style for-loops:
- declare -i i
- for ((i=0; i<$n: i++)); do
- echo $c
- done
- # find command
- find ~/scripting/src/ -name 'oscillator*' -print # find file 'oscillator*' in ~/scripting/src/
- find $HOME \( -name '*.ps' -o -name '*.eps' \) -prin # {} place holder for file name
- find ~/Analytics -name file.txt -exec rm {} \; -print
- find $HOME -name '*' -type f -size +2000 -exec ls -s {} \; # find all files larger than 2000 blocks a 512 bytes (= 1Mb)
- find $HOME -name '*' -type f -size +2000 -exec ls -s {} \; -exec rm -f {} \; # remove all these files
- find $HOME -name '*' -type f -size +2000 -exec ls -s {} \; -ok rm -f {} \; # ask the user for permission to remove
- # pdf to csv:
- pdftotext -layout file.pdf -| sed '$d' | tail -n5 | sed -r 's/ +/,/g; s/ //g'
|