bashref.txt 8.7 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297
  1. # BASH reference
  2. Bash reference manual:
  3. www.gnu.org/software/bash/manual/bashref.html
  4. Advanced Bash-Scripting Guide:
  5. http://www.tldp.org/LDP/abs/html/
  6. man bash # manual page for bash
  7. What Bash is good for
  8. * File and directory management
  9. * Systems management (build scripts)
  10. * Combining other scripts and commands
  11. * Rapid prototyping of more advanced scripts
  12. * Very simple output processing, plotting
  13. Some common tasks in Bash
  14. * file writing
  15. * for-loops
  16. * running an application
  17. * pipes
  18. * writing functions
  19. * file globbing, testing file types
  20. * copying and renaming files, creating and moving to directories, creating
  21. directory paths, removing files and directories
  22. * directory tree traversal
  23. * packing directory trees`
  24. # shellcheck - static analysis tool for shell scripts
  25. $ doas apt-get install shellcheck
  26. $ shellcheck file.sh
  27. Reference:
  28. https://www.shellcheck.net/
  29. https://www.shellcheck.net/wiki/Home
  30. # shfmt - tool for formatting, parsing, and interpreting shell script
  31. $ doas apt-get install shfmt
  32. $ shfmt file.sh # print the formatted script to the terminal
  33. $ shfmt -indent 4 file.sh # number of spaces that we want to use for indentation
  34. $ shfmt -diff file.sh # changes between the unformatted and the formatted script
  35. $ shfmt -list *.sh # list all the files that need formatting
  36. $ shfmt -write file.sh # print the formatted script to the file itself
  37. Bash example 1: hello world
  38. #!/bin/bash
  39. echo "Hello World"
  40. Two options to run this script:
  41. 1. Type the commands directly in the bash shell (only feasible for small scripts)
  42. 2. Save the code as helloworld.sh and run with:
  43. chmod a+x helloworld.sh # Make script executable
  44. ./helloworld.sh
  45. Bash example 2: hello world
  46. #!/bin/bash
  47. x="World" # no space between them, if space is present its an error
  48. echo "Hello ${x}!" # ${x} - variable substitution
  49. Bash has a number of built in commands, type help or help | less to see all.
  50. Bash variables:
  51. Variables in bash are untyped
  52. >>> x=5
  53. Generally treated as character arrays
  54. >>> x=5
  55. >>> x=$x+1 # concatenates the text
  56. >>> echo $x
  57. 5+1
  58. Use the let command for simple arithmetic and other operations:
  59. >>> x=5
  60. >>> let "x+=1"
  61. >>> echo $x
  62. 6
  63. Varibles can be explicitly declared to integer or array:
  64. >>> declare -i i # -i is argument and i is name of the variable, i is an integer
  65. >>> declare -a A # A is an array
  66. >>> r=10
  67. >>> declare -r r # r is read only, means the value of r cannot be changed, its constant
  68. The echo command is used for writing:
  69. >>> s=42
  70. >>> echo "The answer is $s"
  71. and variables can be inserted in the text using (variable interpolation)
  72. Frequently seen variables:
  73. Command line arguments:
  74. $0 # Name of script
  75. $1 # First command line argument
  76. $2 # Second command line argument
  77. # ...
  78. All the command line arguments: $@
  79. Number of command line arguments: $#
  80. The exit status of the last executed command:
  81. $? # returns 0 if the last command was success
  82. Example:
  83. Write a script that takes a command as an argument, runs it and checks if it was
  84. succesfull.
  85. # run_and_test.sh
  86. $@
  87. if ["$?" == "0"]; then
  88. echo "Hurray, everything went fine."
  89. else
  90. echo "Oops, there was an error."
  91. fi
  92. To execute the above script: >>> ./run_and_test.sh ls Programming/*.c
  93. Comparison of two integers use a syntax different from comparison of two strings:
  94. if [$i -eq 10]; then # integer comparison
  95. fi
  96. if ["$name" == "10"]; then # string comparison
  97. fi
  98. Unless you have declared a variable to be an integer, assume that all variables are
  99. strings and use double quotes (strings) when comparing variables in an if test.
  100. Executing a command, storing the result as a variable can be done in two ways:
  101. time=$(date)
  102. time=`date`
  103. Convenient debugging tool: -x
  104. Each source code line is printed prior to its execution if you add -x as option to
  105. /bin/sh or /bin/bash
  106. Either in the header: #!/bin/bash -x
  107. or on the command line: >>> bash -x hw.sh
  108. Very convenient during debugging
  109. Combining bash commands:
  110. * The power of Unix lies in combining simple commands into powerful operations
  111. * Standard bash commands and unix applications normally do one small task
  112. * Text is used for input and output - easy to send output from one command as input
  113. to another
  114. Two standard ways to combine commands:
  115. The pipe, sends the output of one command as input to the next:
  116. >>> ls -l | grep 3331
  117. Will list all files having 3331 as part of the name
  118. Send files with size to sort -rn (reverse numerical sort) to get a list of files
  119. sorted after their sizes:
  120. >>> ls -s | sort -rn # s - size
  121. Make a new application: sort all files in a directory tree assignments, with the
  122. largest files appearing first, and equip the output with paging functionality:
  123. >>> du -a assignments | sort -rn | less
  124. Bash redirects:
  125. Redirects are used to pass output to either a file or stream.
  126. echo "Hei verden" > myfile.txt # Save (stdout) output to file
  127. echo "Hei verden" >> myfile.txt # Append (stdout) output to file
  128. wc -w < myfile.txt # Use file content as (stdin) command input
  129. ls > files && grep 2017 < files is equivalent to ls | grep 2017
  130. cat INF3331-$username | ./test is equivalent to ./test < INF3331-$username
  131. The in-/outputs of a shell process: stdin, stdout and stderr
  132. A process takes standard input (STDIN) and returns
  133. * standard output (STDOUT)
  134. * standard error (STDERR)
  135. * return code - 0 on success, a different number otherwise
  136. ~~~~~~~~~~~~~
  137. STDIN | |---------> STDOUT
  138. --------->| Process |---------> STDERR
  139. | |---------> Return Code
  140. ~~~~~~~~~~~~~
  141. Redirecting process streams: (-v : verbose, explain what is being done)
  142. rm -v *.txt # stdout and stderr are displayed on the terminal
  143. rm -v *.txt 1> out.txt # redirect stdout to a file, same as > (document which files had been deleted)
  144. rm -v *.txt 2> err.txt # redirect stderr to a file
  145. rm -v *.txt &> outerr.txt # redirect stdout and stderr to file
  146. You can print to stderr with: echo "Wrong arguments" >&2 # redirect output as stderr
  147. Redirects and pipes can be combined:
  148. ./compile 2>&1 | less # view both stdout and stderr in less
  149. Example scripts:
  150. case="testbox"
  151. cmt="WinslowRice"
  152. if [ $# -gt 0 ]; then # if user provides an argument else use default
  153. cmt=$1
  154. fi
  155. infile="ellipsoidtest.i"
  156. outfile="outtext.txt"
  157. ./pulse app -cmt $cmt -cname $case < $infile | tee $outfile
  158. dirname=$(pwd)/$(date)
  159. mkdir $dirname && cd $dirname
  160. # mktemp - create a temporary file or directory
  161. # check directory exists
  162. if [ -d $dirname ]
  163. then
  164. echo "directory exists"
  165. exit # exit the script
  166. fi
  167. mkdir $dirname && cd $dirname
  168. # if statement in bash
  169. if [ -d $dirname ];
  170. then
  171. exit
  172. fi
  173. if test -d $dirname; then
  174. exit
  175. fi
  176. # short form of if statement
  177. [ -d $dirname ] && exit
  178. test -d $dirname && exit
  179. # multi-line commands (multi-line text is directed to filetext.txt)
  180. cat > filetext.txt << EOF
  181. multi-line text
  182. can now be inserted here,
  183. and variable substitution such as
  184. $variable is
  185. supported.
  186. EOF
  187. # parsing command-line options
  188. while [ $# -gt 0 ]
  189. do
  190. option=$1; # load command-line arg into option
  191. shift; # shift the command-line arg
  192. # some commands
  193. done
  194. # alternative to case statement is if statement
  195. if [ "$option" == "-m" ]; then
  196. m=$1; shift;
  197. elif [ "$option" == "-b" ]; then
  198. b=$1; shift;
  199. else
  200. echo exit;
  201. fi
  202. # add a check for successful execution
  203. # the shell variable $? is 0 if command was successful, otherwise $? != 0
  204. if [ "$?" != "0" ]; then
  205. echo "application failed"; exit 1 # exit n sets $? to n
  206. fi
  207. For loops:
  208. ./run.sh test1.i test2.i test3.i test4.i
  209. or
  210. ./run.sh *.i
  211. or
  212. for arg in $@; do
  213. # scripts
  214. done
  215. A for-loop over command line arguments.
  216. for loops for file management:
  217. files=`ls *.tmp`
  218. for file in $files
  219. do
  220. echo removing $file
  221. rm -f $file
  222. done
  223. # counters
  224. declare -i counter # declare an integer counter
  225. counter=0
  226. ((counter++)) # arithmetic expressions must appear inside (( ))
  227. echo $counter # yields 1
  228. # for loop with counter
  229. declare -i n; n=1
  230. for arg in $@; do
  231. echo "command-line argument number $n is <$arg>"
  232. ((n++))
  233. done
  234. C-style for-loops:
  235. declare -i i
  236. for ((i=0; i<$n: i++)); do
  237. echo $c
  238. done
  239. # find command
  240. find ~/scripting/src/ -name 'oscillator*' -print # find file 'oscillator*' in ~/scripting/src/
  241. find $HOME \( -name '*.ps' -o -name '*.eps' \) -prin # {} place holder for file name
  242. find ~/Analytics -name file.txt -exec rm {} \; -print
  243. find $HOME -name '*' -type f -size +2000 -exec ls -s {} \; # find all files larger than 2000 blocks a 512 bytes (= 1Mb)
  244. find $HOME -name '*' -type f -size +2000 -exec ls -s {} \; -exec rm -f {} \; # remove all these files
  245. find $HOME -name '*' -type f -size +2000 -exec ls -s {} \; -ok rm -f {} \; # ask the user for permission to remove
  246. # pdf to csv:
  247. pdftotext -layout file.pdf -| sed '$d' | tail -n5 | sed -r 's/ +/,/g; s/ //g'