reference/commandref.txt

# Never-do's in UNIX (The Unix programming environment - Mark Burgess page: 11)
* Don't ever call a program or an important file 'core'. Many scripts go around
  deleting files called 'core' because, when a program crashes, UNIX dumps the
  entire kernel image to a file called 'core' and these files use up a lot of
  disk space. If you call a file 'core' it might get deleted!
* Don't call test programs test. There is a UNIX command which is already
  called test and chances are that when you try to run your program you will
  start the UNIX command instead. This can cause a lot of confusion because the
  UNIX command doesn't seem to do very much at all!

w.xx.y-zzz
w - kernel version, xx - major revision, y - minor revision, zzz - patch number

i386  - 32bit
amd64 - 64bit

Note:
adding rw init=/bin/bash to boot parameters will open a full root session for
anyone who happens to push the power button on your PC. If you think you might
need this kind of access, I would advise you to create a secure BIOS or GRUB
password to protect yourself.

Note:
always tar up a directory, not an individual files

echo $LANG
echo $MAIL # To output the location of your inbox
echo $PATH
echo $SHELL
echo $TERM
printenv # to display environment variables

name=newton
echo $name   # $ symbol is used before a variable in echo command
newton

ulimit # get and set user limits

# command and argument
cal 2020  # command: cal, argument: 2020
* Passing an argument to the command alters its behavior.
* Some commands may require obligatory arguments and will return an error
  message if they are missing. For example, the command to copy a file needs
  two arguments: what file to copy, and where to copy it.
* All commands can be modified using options that are specific to each command.
  cal -j  # use option -j to display Julian calendar

leave 2005 # to remind yourself to leave at 08:05 pm

mkdir Directory; cd Directory; touch 1.txt 2.txt 3.c
echo *.txt
1.txt 2.txt
echo *
1.txt 2.txt 3.c
rm -rf Directory

mkdir -p dir1/dir2/dir3  # will create directories and its parent directories
rm -rf dir1 # recursively deletes the contents of dir1 and its sub-directories

# verify electronic mail addresses
vrfy user@example.edu  # verify that the email address user@example.edu valid

# type
type -a pwd

IBM Developer
https://developer.ibm.com/technologies/
Learn Linux, 101:A road-map for LPIC-1
https://developer.ibm.com/tutorials/l-lpicl-map/
http://www.research.ibm.com/haifa/ponderthis/challenges/October2019.html
http://www.brendangregg.com/index.html  # perf tutorials

# boot time check in Linux
systemd-analyze
systemd-analyze blame
sudo systemctl disable NetworkManager-wait-online.service
sudo systemctl enable NetworkManager-wait-online.service

A repository is a collection of files that has information about various
software, their versions and some other details like the checksum. Each Ubuntu
version has its own official set of four repositories:

    Main - Canonical-supported free and open-source software
    Universe - Community-maintained free and open-source software
    Restricted - Proprietary drivers for devices
    Multiverse - Software restricted by copyright or legal issues

/bin - Binaries, programs used in booting the system.
/usr/bin - User binaries, standard programs available to users.
/usr/local/bin - Local binaries, programs specific to an installation.

https://www.ubuntu.com/about/release-cycle
https://www.kernel.org/doc/man-pages/
https://clearlinux.org/documentation/clear-linux

# free and df
free -h # RAM memory
df -h   # Hard Drive memory

# Generate a random password
gpw () { openssl rand -base64 48 | cut -c1-${1}; }
date +%s | sha256sum | base64 | head -c 32 ; echo
< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c${1:-32};echo;
openssl rand -base64 32
tr -cd `[:alnum:]` < /dev/urandom | fold -w30 | head -n1
< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c6
date | md5sum
gpg --gen-random --armor 1 14
pwgen 14 1
Ref: https://www.howtogeek.com/howto/30184/10-ways-to-generate-a-random-
     password-from-the-command-line/

journalctl -b | grep 'DMI:'
dell supports LVFS https://fwupd.org/lvfs/ for some, but not all devices.

Note: If you use systemd, read the journalctl man page. It may seem heavy-going
at first, but it makes searching the system logs so much more effective.

sudo lsblk # list block devices

swapon --summary # to find information about your swap space
cat /proc/swaps

set -o # to display all shell options
set -o noclobber # - (dash) for enabling
set +o noclobber # + (plus) for disabling

apt-show-versions # to list installed packages
aptitude search ~E # list essential packages

## id
$ id username # print real and effective user and group IDs

## installed packages
$ apt list --installed | less
$ apt list --installed | grep -i apache
$ dpkg -l
$ dpkg -l | grep -i apache
$ dpkg -S $(which <command>)

/var/log/apt/
sudo less -N history.log

/var/lib/dpkg/available # available packages

/etc/init.d # Configuration of System V init under Debian GNU/Linux
/etc/systemd # Configuration of Systemd
/etc/default # some default files

In systemd, unit files get packaged into
/usr/lib/systemd/system, but if you want
to replace the default with your own, you
can put them in /etc/systemd/system and
whatever is there will take precedence
over the defaults.

To limit the amount of CPU a process gets by dropping in a new unit
configuration file to /etc/systemd/system and adding:
[Service]
CpuShares=200

# To login in text mode
vi /etc/default/grub
replace "quiet splash" -> "text"
sudo update-grub

# To start XMonad
startx /usr/bin/xmonad-session
# To start dwm
startx /usr/local/bin/dwm
# To start without cursor
startx -- -nocursor

# To start qutebrowser
cd qutebrowser
.venv/bin/python3 -m qutebrowser
:set zoom.default 67%
:config-source   # to recompile
ctrl+T # new tab
shift+j # move to next tab
shift+k # move to previous tab
D # close the current tab

# man pages
apropos partition
man ascii
$ ascii # to display ascii characters
man hier
man systemd.unit
man systemd.service

# ctrl
ctrl+A - begining of shell command
ctrl+E - end of shell command
ctrl+H - delete a letter
ctrl+W - delete a word
ctrl+U - erase line before cursor
ctrl+K - erase line after cursor
ctrl+Y - reverse of ctrl+W, ctrl+U and ctrl+K
ctrl+B - move backward by letter
ctrl+F - move forward by letter
ctrl+P - scrolls through previous command
ctrl+N - scrolls through next command
ctrl+R - searches for commands you've already typed

# alt
alt+B - move backward by word
alt+F - move forward by word
alt+U - uppercase a word
alt+L - lowercase a word
alt+C - capitalizes letter where cursor is and moves to end of word

# Special key strokes
ctrl-U      erase line before cursor
ctrl-H      erase a character before cursor
ctrl-D      terminate input (exit shell if you are using shell)
ctrl-C      terminate a running program
ctrl-Z      temporarily stop program by moving it to the background job
ctrl-S      halt output to screen
ctrl-Q      reactivate output to screen
ctrl-Alt-Del    reboot/halt the system, see inittab(5)
Left-Alt-Key    meta-key for Emacs and the similar UI
Up-arrow    start command history search under bash
ctrl-R      start incremental command history search under bash
Tab     complete input of the filename to the command line under bash
ctrl-V Tab  input Tab without expansion to the command line under bash

# shutdown
shutdown -h now
shutdown -r now

# Groups
groups           # to list the groups
groups username  # to list the groups to which the user belongs
Note: Please bear in mind that the user ID of root is always 0 -every user
      with a user ID of 0 has root privileges!

# sudo
sudo -u username program # in sudoers you can permit the user to only sudo to
                           that user and run that one program

# Documents
/usr/share/doc

# whereis - to find the binary files
whereis python3

# alias
alias
alias ec='echo'
unalias ec    # to unalias the command
note: enter it in .bashrc file (source ~/.bashrc)

# information regarding the OS installed
cat /etc/*release
cat /etc/issue.net # to view Linux distribution
lsb_release -a

# Bash
source ~/.bashrc # to compile changes in .bashrc file
(to execute your .bashrc with source ~/.bashrc)
~/.bash_history # bash history file
~/.local/share/Trash  #files in trash folder
cp -r directory/ foo/  # copy recursively
du -Sh # list all files size in directory
du -sh # list only the file size of directory
dpigs -H --lines=3000 | less -N
ip addr list wlan0
hostname -I # show ip address
last - show a listing of last logged in users
!! - alias to previous command

# Adding/Deleting Users
adduser username
deluser username

## su command
The su command is used to switch to a different user's account. You can use
this command with the username of another user to log in with that user's
session.
$ su username
To exit anytime from the other user's session just type exit and hit enter.

# file
file <filename> # displays the file type
file -i <filename> # to get the mimetype

# You can obtain detailed information on the hardware using ls commands such as
  lspci  - list all PCI devices
  lsblk  - list block devices
  lscpu  - display information about the CPU architecture
  lsscsi - display SCSI devices information

# hardware identification
For the PCI-like devices (AGP, PCI-Express, CardBus, ExpressCard, etc.), lspci
(probably with "-nn" option) is a good start for the hardware identification.

Alternatively, you can identify the hardware by reading contents of
"/proc/bus/pci/devices" or browsing directory tree under "/sys/bus/pci"

pciutils - Linux PCI utilities: lspci
usbutils - Linux USB utilities: lsusb
pcmciautils - PCMCIA utilities for Linux: pccardctl
scsitools - collection of tools for SCSI hardware management: lsscsi
procinfo - system information obtained from "/proc": lsdev
lshw - information about hardware configuration: lshw
discover - hardware identification system: discover

# system and hardware time
The following sets system and hardware time to MM/DD hh:mm, CCYY.
# date MMDDhhmmCCYY
# hwclock --utc --systohc
# hwclock --show

If the hardware (BIOS) time is set to UTC, change the setting to "UTC=yes" in
the "/etc/default/rcs".

The following reconfigure the timezone used by the Debian system.
# dpkg-reconfigure tzdata

# NTP
If you wish to update system time via network, consider to use the NTP service
with the packages such as ntp, ntpdate, and chrony.
Under systemd, use systemd-timesyncd for the network time synchronization
instead.
ntptrace in the ntp package can trace a chain of NTP servers back to the
primary source

## keyboard
sudo dpkg-reconfigure keyboard-configuration

The keyboard type on the first screen will likely be correct ("Generic") then
when you hit OK (Enter) the other screens will prompt you for language etc. Use
arrow keys to scroll up and down between options and "Enter" for OK.

# systemd is the most popular init process for bootstrapping user spaces and
  controlling multiple system process.
  sudo systemctl status

# Dmesg allows you to figure out errors and warnings in the kernel's latest
  messages.
  dmesg | less

# You can also look at all Linux system logs in the /var/log/messages file,
  which is where you'll find errors related to specific issues. It's worthwhile
  to monitor the messages via the tail command in real time when you make
  modifications to  your hardware, such as mounting an extra disk or adding an
  Ethernet network interface.
  tail -f /var/log/messages

# Commands to figure out networking functions in the Linux server include
  ip addr, traceroute, nslookup, dig, and ping, among others.
  sudo ip addr show

To mount cd to /home/user/lan :
sudo mount /dev/cdrom /home/user/lan

# chmod
chmod u=rwx,g=rwx,o=rwx <filename>
chmod 777 <filename>
4=read, 2=write, 1=execute, 0=no permission
7=4+2+1   5=4+0+1    4=4+0+0

chmod 600 <filename>   # make an existing file "<filename>" to be non-readable
                         and non-writable by the other people
                         (non-executable for all)
chmod 644 <filename>   # make an existing file "<filename>" to be readable but
                         non-writable by the other people
                         (non-executable for all)
chmod 755 <filename>   # make an existing file "<filename>" to be readable but
                         non-writable by the other people (executable for all)

u user     + to add a permission                      r read
g group    - to remove a permission                   w write
o other    = to assign a permission explicitly        x execute (for files),

chmod u=rw file
sets the permissions on the file to give the user read and write permission on
file. No other permissions are altered.

chmod u+x,g+w,o-r file
alters the permissions on the file to give the user execute permission, to
give members of the users group write permission, and prevent any users not
in the group from reading it.

chmod u+w, go-x dir
gives the user write permission in the dir, and prevents all other users
having access to that directory (by using cd. They can still list its
contents using ls.)

chmod a+w file  # to add write access for everybody
chmod a-w file  # to remove write access
chmod a=r file  # set permissions on the file to allow
                  only read access for all users

chmod u+w, go+r-w file      # adds write access for the file owner and removes
                              write access and adds read access for the group
                              and other classes

chmod -R go-rwx /home/user  # if user wants to protect all the files under
                              his home directory from everyone else
                              [recursive option (-R)]

chmod -R +r .               # grants read access to all users for the current
                              directory and every file under it

chmod go= file              # prevents any access to the file by anyone other
                              than its owner

chmod go+rX *               # the X access type grants execute access to the
                              specified access classes only when execute access
                              is already set for some access class. A typical
                              use for this access type is to grant group or
                              other read and execute access to all the
                              directories and executable files within a
                              subtree while granting only read access to all
                              other types of files (example: datafile.txt)

chmod o=g *                 # makes the other access the same as the current
                              group access for each file/directory

# umask
027 is better from security perspective, even better is to use 077 for root.
With 027 mask, when root creates a file, it can be read and executed by users
who belongs to the group of the file. With 077 mask, only root can read write
and execute files.

Why is 077 better?
because by making files belonging to user readable by root only, it avoids some
common system administrator mistakes. because it's harder for an attacker to
run privilege escalation only poorly written shell scripts/binaries if they
can't even read it.

# chown
chown user file/Directory       # Change the owner of fiel/Directory to "user"
chown user:staff file/Directory # Likewise but also change its group to "staff"
chown -hR user file/Directory   # Change the owner of file/Dire and subfiles
Note: refer man chown (Examples section)

# chgrp
chgrp user file/Directory   # Change the group of file/Directory to "user"
chgrp -hR user file/Directory # Change the group of file/Dire and subfiles
Note: refer man chgrp (Examples section)

# irssi - https://freenode.net/kb/answer/registration
https://irssi.org/documentation/startup/
https://wiki.debian.org/IRC
https://fedoraproject.org/wiki/How_to_use_IRC#Meeting_Protocol
Help regarding registration - /msg NickServ help register
Register your IRC nick - /msg NickServ REGISTER password youremail@example.com
Logging in - /connect chat.freenode.net 6667/6697 account_name:password
/help                     # show all commands
/help command             # shows command usage
/network list             # list network
/connect Freenode         # connect network Freenode
/join #debian             # join channel debian
/part                     # leave channel
/nick name                # changes the name
/whois name               # shows basic whois info for a name
/msg NickServ info name   # more info of /whois name
/away message             # leaves a message when you are away frm channel
/back                     # use /back when you return
/quit message             # leaves a message when you quit IRC channel
long pastes: wgetpaste or dpaste.com
For posting multi-line texts into the channel - https://paste.ubuntu.com
To post screenshots use https://imgur.com/
!pastebinit to paste directly from command line
Make sure you give us the URL for your paste

# wireless connection - wlx00873239946f
sudo ifconfig wlan0 up/down     # bring wireless network up/down
iw dev                          # gives wifi adapter device name
iw wlan0 link                   # Connected/Not Connected
                                  (wlan0 - device name from previous command)

sudo iw wlan0 scan / sudo iwlist wlan0 scan
nmcli dev
nmcli dev wifi
sudo nmcli dev wifi connect name password *****
(replace name with network name)
nmcli con show = nmcli -p connection
(list all ethernet and wireless connections)

# nmcli
nmcli connection show      # list all network connections
nmcli device show          # list device information including hardware address
nmcli general status       # show overall status of Network Manager
nmcli general hostname     # get and change system hostname
nmcli general permissions  # show the permissions
nmcli general logging      # get and change Network Manager logging level
                             and domains
nmcli networking on/off    # enable or disable networking control by
                             Network Manager

nmcli networking connectivity  # get network connectivity state
none = the host is not connected to any network
portal = the host is behind a captive portal and cannot reach the full Internet
limited = the host is connected to a network, but it has no access to Internet
full = the host is connected to a network and has full access to the Internet
unknown = the connectivity status cannot be found out

nmcli radio all/wifi/wwan
nmcli radio wifi on/off
nmcli radio wwan on/off
nmcli radio all on/off
nmcli monitor # observe Network Manager activity
nmcli connection show/up/down/add/edit/modify/delete/monitor/reload/load
nmcli dev status/show/set/connect/reapply/disconnect/delete/monitor/wifi/lldp
nmcli -p -f general,wifi-properties device show wlx00873239946f # dev details
man 7 nmcli-examples -- usage examples of nmcli

# mount/unmount usb
fdisk -l (sudo) #to find name
df -h
/dev/sdb1
sudo mount /dev/sdb1 /media/user
sudo umount /dev/sdb1 /media/user

# undo cannot format on usb storage
- type sudo fdisk -l and not usb drive letter (/dev/sdb1)
- make sure the device is unmount
- type sudo fdisk /dev/sdb
- type d to proceed to delete a partition
- type 1 to select the 1st partition and press enter
- typd d to proceed to delete another partition
    (fdisk should automatically select the second partition)
- type n to make a new partition
- type p to make this partition primary and press enter
- type 1 to make this the first partition and then press enter
- press enter to accept the default first cylinder
- press enter again to accept the default last cylinder
- type w to write the new partition information to the USB key
- type umount /dev/sdb1 (check)
- type sudo mkfs.vfat -F 32 /dev/sdb1

# for mkfs.vfat
sudo apt-get install dosfstools
/sbin/mkfs.vfat

# Basic Network Commands
ping www.slackware.com
traceroute www.slackware.com
host www.slackware.com #map name to ip address
nslookup 64.57.102.34
dig @192.168.1.254 www.slackware.com mx

# text editor
sudo update-alternatives --config editor

# X terminal emulator
sudo update-alternatives --config x-terminal-emulator

# Recording the shell activities (record when you perform any sys admin tasks)
$script #Script started, file is typescript
Do whatever shell commands under script
Press Ctrl-D to exit Script

sudo apt purge `dpkg --get-selections | grep deinstall | cut -f1`
aptitude search '?essential'
# To list all essential packages in debian
dpkg-query -Wf '${Package; -40}${Essential}\n' | grep yes

# send message to everyone who is logged on to the system with wall
echo "We are shutting down in 1 hour" | wall

# viewing binary data
od -t x1 <name>

# To convert *.mkv to *.mp4
ffmpeg -i input.mkv -codec copy output.mp4

# pdftk (read man page of pdftk)
pdftk in.pdf output out.pdf uncompress
LESSOPEN="" less out.pdf

merge multiple pdf files into a single document:
pdftk file1.pdf file2.pdf cat output merged.pdf

combine even and odd pages into one document:
pdftk A=even.pdf B=odd.pdf shuffle A B output combined.pdf

split a pdf file into multiple documents:
pdftk input.pdf cat 1 output page1.pdf
pdftk input.pdf cat 2-end output pages2-end.pdf

rotate the pages of a pdf file:
pdftk input.pdf cat 1east 2east 3-endwest output outfile.pdf
This rotates the first and second pages of 'input.pdf' 90 degrees clockwise and
the remaining pages 90 degrees counterclockwise.

rotate all pages by 180 degrees:
pdftk input.pdf cat 1-endS output outfile.pdf

fill out a pdf form:
pdftk form.pdf fill_form data.fdf output filled.pdf
This fills out the form fields in 'form.pdf' using the data in the 'data.fdf'
file and saves the result in 'filled.pdf'.

# qpdf
to combine two pdf files to form a new pdf file:
qpdf --empty --pages file1.pdf file2.pdf -- combined.pdf

using wildcards instead of listing many source files:
qpdf --empty --pages *.pdf -- allcombined.pdf

to extract pages 1 and 2 from file1.pdf and page 1 from file2.pdf:
qpdf --empty --pages file1.pdf 1-2 file2.pdf 1 -- combined.pdf

to extract pages from a large pdf file:
qpdf --empty --pages largefile.pdf 1-3,7,11,18-21,55 -- summary.pdf

The output file, 'summary.pdf' contains pages 1 to 3, 7, 11, 18 to 21, and 55
from the input file 'largefile.pdf'.

to split a pdf file into separate pdf files each holding a single page:
qpdf summary.pdf page.pdf --split-pages

The file we're splitting is 'summary.pdf', and the output file is given as
'page.pdf'. This is used as the base name. Each new file has a number added to
the base name. The --split-pages option tells qpdf what type of action we're
performing.

use page ranges to select the pages you want to split:
qpdf large.pdf section.pdf --pages large.pdf 1-5,11-14,60,70-100 -- --split-pages

The page ranges are used to specify the pages/ranges we want, but each page is
still stored in a single pdf file.

to extract a page range and stored in a single pdf:
qpdf --empty --pages large.pdf 8-13 -- chapter2.pdf

to rotate a page, we use the --rotate option:
qpdf --rotate=+90:1 summary.pdf rotated1.pdf

The +90 means rotate the page 90 degrees clockwise. You can rotate a page 90,
180, or 270 degrees. You can also specify the rotation in degrees anticlockwise,
by using a negative number, but there's little need to do so. A rotation of -90
is the same as a rotation +270. The number separated from the rotation by a
colon ':' is the number of the page you want to rotate. This could be a list of
page numbers and page ranges, but we're just rotating the first page. To rotate
all pages use a page range of 1-z.

to create an encrypted version of the 'summary.pdf' called 'secret.pdf':
qpdf --encrypt <user password> <owner password> 256 -- summary.pdf secret.pdf

To encrypt a pdf we need to use the --encrypt option and provide both the user
password and the owner password. We also specify the strength of encryption to
use. You'd only need to drop from 256-bit encryption to 128-bit if you want to
support very old pdf file viewers. We suggest to stick with 256-bit encryption.

to decrypt a file:
qpdf --decrypt --password=<owner password> secret.pdf unlocked.pdf

To decrypt a file, use the --decrypt option. Obviously, you must know the owner
password for this to work. We need to use the --password option to identify the
password.

Reference: https://qpdf.readthedocs.io/en/stable/index.html

# scribus

# cowsay
cowsay -l
cowsay -f tux Linux is awesome

# ls
List only the .txt files in the directory: ls *.txt
List by file size: ls -a
Sort by time and date: ls -d
Sort by extension: ls -x
Sort by file size: ls -S
Long format with file size: ls -ls
send it to a file for later viewing: ls -al > mydirectorylist

# mkdir
mkdir sa{1..50}  # 50 directories from sa1 to sa50

# tar files/directories to backup
1. Compress an Entire Directory or a Single file
tar -czvf name-of-archive.tar.gz /path/to/directory-or-file
-c: Create an archive, -z: Compress the archive with gzip,
-v: Display progress in the terminal while creating the archive
-f: Allows you to specify the filename of the archive
2. Compress Multiple Directories or Files at once
tar -czvf archive.tar.gz /home/ubuntu/Downloads /usr/local/stuff
/home/ubuntu/Documents/notes.txt
3. Exclude Directories and Files
tar -czvf archive.tar.gz /home/ubuntu --exclude=/home/ubuntu/Downloads
--exclude=/home/ubuntu/.cache
tar -czvf archive.tar.gz /home/ubuntu --exclude=*.mp4
4. Use bzip2 Compression Instead
tar -cjvf archive.tar.bz2 stuff
5. Extract an Archive
tar -xzvf archive.tar.gz
6. Extract the contents of the archive to a specific directory
tar -xzvf archive.tar.gz -C /tmp

# untar/tar
tar -tf *.tar.gz # to view contents without extracting
tar -tvf *.tar.gz # list more information
tar -xvzf filename.tar.gz # to decompress a tarball

# zip
unzip -l filename.zip # to view contents without unzip
unzip filename.zip # to unzip

# scp
scp ph15m015@172.29.204.101:/home2/mtech/ph15m015/cl.tar.gz .

# hostname
hostname    # list machine name
hostname -i # list 127.0.1.1
hostname -I # list ip address of network

# youtube-dl [http://ytdl-org.github.io/youtube-dl/download.html]

To install it right away for all UNIX users type:
$ sudo curl -L https://yt-dl.org/downloads/latest/youtube-dl -O /usr/local/bin/youtube-dl
$ sudo chmod a+rx /usr/local/bin/youtube-dl

If you do not have curl, you can alternatively use wget:
$ sudo wget https://yt-dl.org/downloads/latest/youtube-dl -O /usr/local/bin/youtube-dl
$ sudo chmod a+rx /usr/local/bin/youtube-dl

:yank # type :yank in qutebrowser to copy the url
xclip -o -selection clipboard > url.txt  # write the url in url.txt
youtube-dl --extract-audio/-x --audio-format mp3 "$(< url.txt)" # mp3 format
youtube-dl -F "$(< url.txt)" # to find best video format
youtube-dl -F --no-playlist "$(< url.txt)" # list available formats of one file
youtube-dl -f no. "$(< url.txt)" # to download the file
youtube-dl -f 22 --no-playlist -f 22 --no-playlist "$(< url.txt)"

# update youtube-dl
sudo youtube-dl --update
# version
youtube-dl --version

# SSH
If you're using a cloud server, you will probably want to allow incoming SSH
connections (port 22) so you can connect to and manage your server. This
section covers how to configure your firewall with various SSH-related rules.

Allow SSH
To allow all incoming SSH connections run this command:
$ sudo ufw allow/deny ssh
An alternative syntax is to specify the port number of the SSH service:
$ sudo ufw allow 22

Allow incoming SSH from specific IP Address or Subnet
To allow incoming SSH connections from a specific IP address or subnet,
specify the source. For example, if you want to allow the entire 15.15.15.0/24
subnet, run this command:
$ sudo ufw allow from 15.15.15.0/24 to any port 22

Allow incoming Rsync from specific IP Address or Subnet Rsync, which runs on
port 873, can used to transfer files from one computer to another.

To allow incoming rsync connections from a specific IP address or subnet,
specify the source IP address and the destination port. For example, if you
want to allow the entire 15.15.15.0/24 subnet to be able to rsync to your
server, run this command:
$ sudo ufw allow from 15.15.15.0/24 to any port 873

Source: https://www.digitalocean.com/community/tutorials/ufw-essentials-common
        -firewall-rules-and-commands
        https://thepcspy.com/read/making-ssh.secure/

# hardware info
hwinfo --help
hwinfo --netcard
hwinfo --gfxcard

# dpkg commands
# to install a downloaded debian package (.deb)
    sudo dpkg -i packagename.deb
# to unpack the package (.deb) without installing it
    sudo dpkg --unpack packagename.deb
# to remove a debian package (.deb)
    sudo dpkg -r packagename
# to purge a debian package (.deb)
    sudo dpkg -P packagename
# to reconfigure/repair an installed debian package
    sudo dpkg-reconfigure packagename
# find out what package the file /etc/logrotate.conf belongs
    dpkg -S /etc/logrotate.conf
# list information about the pkg including all the files it contains
    dpkg -L packagename
# verify the package installation
    dpkg -V packagename
# information about .deb package
    dpkg --info *.deb
# report status of specified package
    dpkg --status packagename
# list all currently installed packages (including those from the repositories)
    dpkg -l
# if you need simpler output
    dpkg --get-selections
# display details of the specified package
    dpkg -s zip

# clamAV
sudo apt install clamav
sudo freshclam # to update the signatures
clamscan --help
clamscan -r --bell -i /home/username/Downloads

# lynis
sudo lynis audit system
sudo lynis audit system --quick
sudo lynis show commands
sudo lynis show help
sudo lynis show profiles
sudo lynis show settings
sudo lynis show version

Reference:
https://cisofy.com/documentation/lynis/get-started/#first-run
https://linuxsecurity.expert/checklists/linux-security-and-system-hardening
https://linuxsecurity.expert/checklists/linux-security-best-practices

Installation:
git clone https://github.com/CISOfy/lynis
Execute:
cd lynis
./lynis audit system
To run as root, change ownership of the related files (or full directory)
chown -R 0:0 lynis
Test and debug information: /var/log/lynis.log
Report data               : /var/log/lynis-report.dat

# ps_mem
sudo pip install --upgrade pip # to upgrade pip to latest version
sudo pip install ps_mem
sudo ps_mem | less # to run ps_mem

# Private Browsing using Tor and I2P

Tor, I2P, etc, are open, distributed networks that bounce your internet
communications around a network of relays run by volunteers all around the
world. This way, anybody watching your internet connection can't figure out the
sites you visit, and the sites you visit can't track your physical location.

Though both of them are supposed to protect your identity online using
cryptographic methods, there are several differences in their respective
approaches to this question of privacy. Tor takes the directory-based approach,
providing a centralised point to manage the overall view of the network, as
well as gather and report statistics, while I2P uses a distributed network
database and peer selection. Tor provides better anonymous access to the open
internet while I2P provides a more tobust and reliable network within the
network layout. Tor also has a larger user base, with more volunteers around
the world.

Also, beware that your ISP can see that you're using Tor or I2P (though they
cannot determine the content of the traffic). To hide this information from
your ISP, you can use a high-quality VPN service to act as an entry point to
your anonymous network. An excellent open source project for this is OpenVPN:
https://openvpn.net

# Tlp
sudo tlp-stat | less  # system info

# Service - /etc/init.d
service --status-all  # to list status of all services
service sshd status   # gives the status of sshd
service sshd stop     # to stop the sshd service
service sshd start    # to start the sshd service
service sshd restart  # to restart the sshd service

# wget
If you ever need to download an entire website, perhaps for off-line viewing,
wget can do the job for you:
$ wget \
 --recursive \ # download the entire website
 --no-clobber \ # don't overwrite any existing files
 --page-requisites \ # get all the elements that compose the page (images, css)
 --html-extension \ # save files with the .html extension
 --convert-links \ # convert links so that they work locally, off-line
 --restrict-file-names=windows \ # modify filename (to work in windows as well)
 --domains website.org \ # don't follow links outside website.org
 --no-parent \ # don't follow links outside the directory tutorials/html/
        www.website.org/tutorials/html/

wget resume download
wget -c url
wget --continue url
wget --continue [options] url
man wget
wget --help

wget address
wget --http-user=USER --http-password=PASSWORD address

# download an entire website
$ wget --random-wait -t -p -e robots=off -U mozilla https://www.website.com

# curl
curl web-address > file-source
example: curl https://address.html > address.html
curl ipinfo.io/ip_addr
curl https://api.ipdata.co/ip_addr?api-key=test
curl https://ipapi.co/ip_addr/json

# trickle
trickle -d 1024 wget -c
   https://archive.org/download/CharlieChaplin/Chaplin_512kb.mp4
will download the Charlie Chaplin movie (available as public domain from the
Internet Archive) with the Wget downloader limiting the download speed to
1024 Kbps.

You can use Wget's --limit-rate option to regulate its download speed.
In the same vein, use the -u switch to regulate the upload speed.

If setting per-application speeds seems too much of a chore, you can set global
speed caps with the trickled command, such as
trickled -d 2048 -u 1024

trickle apt upgrade
will now update the Ubuntu installation while respecting the limits set by the trickled.

# httrack

# tasksel - a user interface for installing tasks
tasksel --list-tasks
tasksel --task-packages dns-server # to list the packages installed
                                     with the DNS server

# Terminal:
sudo update-alternatives --list x-terminal-emulator
sudo update-alternatives --install /usr/bin/x-terminal-emulator
    x-terminal-emulator /usr/bin/stterm 100
sudo update-alternatives --list x-terminal-emulator
sudo update-alternatives --config x-terminal-emulator

# Hardware Info
sudo lshw | less
sudo lshw -C network

# APT commands reference
apt-cache pkgnames
apt-cache search <package-name>
apt-cache --names-only search chm
apt-cache show <package-name>
apt-cache showpkg <package-name>  # show dependencies
apt-cache depends <package-name>
apt-cache rdepends <package-name>
apt-cache depends --installed <package-name>
apt-cache rdepends --installed <package-name>
apt-cache stats
doas apt-get update / doas apt update
doas apt-get upgrade / doas apt upgrade / doas apt-get dist-upgrade
doas apt-get install <package-name> / doas apt install <package-name>
doas apt-get install <package-name1> <package-name2>
doas apt list # to list all available packages
doas apt list | grep <package-name>
doas apt list --installed # to list only the installed packages
doas apt list --upgradable # to list of the upgradable packages before upgrading
apt search <package-name> # to search for a given package in the list of packages
apt show <package-name> # to retrieve information about a given package
doas apt autoremove # to remove the unneeded dependencies
doas apt remove <package-name> # to remove an installed package
doas apt purge <package-name> # to remove an installed package including config files
doas apt update # to update the package index (pull latest changes from APT repo)
doas apt upgrade # to upgrade the installed packages to their latest versions
doas apt upgrade <package-name> # to upgrade <package-name>
doas apt full-upgrade # will remove the installed packages if needed to upgrade
doas apt install <package-name> # install <package-name>
doas apt install <package-name1> <package-name2>
doas apt install /path/to/file.deb # to install local deb.file

Note:
* apt combines the most frequently used commands from the apt-get and apt-cache
  tools with different default values of some options
* apt is designed for interactive use
* prefer using apt-get and apt-cache in shell scripts as they are backward compatible
  between the different versions and have more options and features
* always update the package index before upgrading or installing new packages
* upgrade command doesn't upgrade packages that require removal of installed packages
* the difference between upgrade and full-upgrade is that the later will remove
  the installed packages if that is needed to upgrade the whole system

# apt-get update/upgrade list without changing anything
apt list --upgradable
apt-get --simulate upgrade
apt-get -s upgrade # --simulate, --just-print, --dry-run, --recon, --no-act
doas apt-get -u upgrade --assume-no
doas apt-get -u -V upgrade # to list packages to be upgraded with their versions
doas apt-get --only-upgrade install <package-name>

# apt-show-versions
apt-show-versions # to know for all your installed packages whether they are
                    uptodate or upgradeable
apt-show-versions -u # if you want to have a list of all upgradeable packages
apt-show-versions -a -p libc6 # to get list of all available versions of libc6
apt-show-versions dpkg apt # to get information about several packages
apt-show-versions -r ^texlive
apt-get install `apt-show-versions -u -b | grep testing` # to upgrade all packages in
                                                           testing

-u, --upgradeable # Print only upgradeable packages
-a, --allversions # Print all available versions of the selected packages
-b, --brief # Print only package_name/distribution for upgradeable packages
-v, --verbose # Prints out messages about which package files are parsed
-V, --version # Prints out version of apt-show-versions
-h, --help # Prints out command-line help

# To decrypt file
qpdf -password=password -decrypt /home/username/Documents/Secured.pdf
/home/username/Documents/unsecured.pdf

# xrandr
xrandr --listmonitors    # this command gives the <output name>
                           for the command below
xrandr --output <output name> --brightness "<x>"
## <x> is a number between 1 (full brightness) and 0 (backlight off)

# rclone - a command line program to sync files and directories
# List directories in top level of your pCloud
rclone lsd remote:
# List all the files in your pCloud
rclone ls remote:
# To copy a local directory to remote:backup
rclone copy /home/source remote:backup
# To sync a local directory to a remote
rclone sync /home/source remote:directory_name
rclone sync -P /home/source remote:directory_name # -P progress
rclone sync -P Books/Chess/chess-wikibooks.pdf pcloud:Books/Chess/
rclone sync /home/source remote:directory_name --dry-run
rclone move /home/source remote:directory_name
rclone delete remote:path
# checks the files in the source and dest match
rclone check source:path dest:path
rclone check /home/saran/Books pcloud:Books/
# To create a directory in remote
rclone mkdir remote:directory_name
# prints the total size and number of objects in remote:path
rclone size remote:path
# interactively find duplicate files and delete/rename them
rclone dedupe remote:path
rclone version
rclone version --check
# get quota information from the remote
rclone about remote:
rclone config file
rclone config show
rclone config providers
# list all the remotes in the config file
rclone listremotes
# Mount the remote as file system
rclone mount remote:path/to/files /path/to/local/mount
# Explore a remote with a text based user interface
rclone ncdu remote:
# create new file or change file modification time
rclone touch remote:path
# list the contents of the remote in a tree like fashion
rclone tree remote:path
rclone move remote:GRE remote:Books/GRE/

# IP
sudo ip addr show  # to check an IP address
sudo ip addr del 192.168.50.5/24 dev eth1  # to remove an IP address
sudo ip link set eth1 up  # to enable network interface
sudo ip link set eth1 down # to disable network interface
sudo ip route show  # to check route table
*** https://www.tecmint.com/red-hat-rhcsa-rhce-exam-certification-book/

# systemctl
systemctl status ufw  # to check the status of ufw

# journald
sudo journalctl -b    # log messages right from the system boot

# fdisk  http://www.slackware.com/install/partitions.php
When you start fdisk you need to specify the device to use. By default it will
try to open /dev/hda, but in some cases this is not the correct device to use.
Just specify the device name after typing fdisk on the command line.
For example:

    fdisk /dev/hdb

This will tell fdisk to open the primary slave IDE hard disk. Notice that you
do not specify a partition number on the device name.

An alternative to fdisk is cfdisk, which provides a menu-based setup program
for the partition setup (DOS users comfortable with DOS's fdisk may find this
program easier).
Just run cfdisk at the prompt instead of fdisk.

Here are some key commands you should be familiar with when using fdisk.
p Display the current partition table
m Display the help screen
d Delete a partition
n Add a new partition
t Change the partition's system ID
q Quit fdisk without saving changes
w Write changes to device and quit fdisk

# w3m
sudo apt install w3m w3m-img  # installing w3m
w3m duckduckgo.com            # basic browsing
shift-B                       # take you back a page
shift-T                       # browser tabs
shift-[ and shift-]           # to switch between tabs
Esc + A                       # to bookmark the page
w3m -M <url>                  # Display web content in monochrome terminal
w3m -o auto_image=TRUE        # Display embedded graphics
# Display content from Usenet
w3m -m nntp://news.aioe.org/comp.os.linux.networking
# Upload data for a URL using the POST method
w3m -post - http://example.com/form.php <<<'a=0&b=1'

~/.w3m/bookmark.html  # default bookmark file
~/.w3m/config   # user defined configuration file; orverrides /etc/w3m/config
~/.w3m/cookie   # cookie jar; written on exit, read on launch
~/.w3m/history  # browser history -visited files and URLs
~/.w3m/keymap   # user defined key bindings; overrides default key bindings
~/.w3m/mailcap  # external viewer configuration file
~/.w3m/menu     # user defined menul overrides default menu
~/.w3m/mime.types   # MIME types file
~/.w3m/mouse        # user defined mouse settings
~/.w3m/passwd       # password and username file
~/.w3m/pre_form     # contains predefined values to fill recurrent HTML forms
http://w3m.sourceforge.net

Translate html to text
cat index.html | w3m -dump -T text/html > index.txt

Useful command line options
-s       squeeze multiple blank lines
-cookie
-no-proxy
-post FILE

Moving around
H            show runtime options
K,J          scroll one line
.,,          shift screen by one column
z            center current line
[,]          go to first/last link
TAB          move to next link
C-u,ESC-TAB  move to previous link
C-m          popup link list menu and move cursor to selected link
C-l          popul link list menu and go to selected link
(,)          cancel/redo last cursor movement

Hyperlinks
C-t          open link in new tab
a            save link
c            peek current URL
u            peek link URL
I            view image
;            mark current word as anchor (link navigation will apply)
=            view info of current document
F            render frame
L            show all links and images

Buffers
B            back
v            view HTML source
s            popup buffer selection menu
S            save buffer to file
E            edit current document (local file)
R            reload
+            information about current page

Tabs
T            open new tab
C-q          close current tab
{,}          move to next/previous tab
ESC-t        popup tab selection menu

Bookmarks
ESC-b        view bookmark
ESC-a        add current page to bookmark

Search
/,?          search forward/backward
C-s,C-r      incremental search forward/backward

Marks
C-@          set/unset mark
ESC-n,ESC-p  move to next/preious mark

Misc
o            option setting panel
C-k          view cookie list
C-h          view history
!            execute shell command
C-z          stop loading document
Q            quit without confirmation

# system information viewers
dmidecode - it reports information about your system's hardware as described
            in your system BIOS according to the SMBIOS/DMI standard.
hwinfo    - powerful hardware detection tool come from openSUSE
inxi      - a script to get system information
nmon      - console based application for monitoring various system components

# Apache
To look into the number of Apache processes your Linux system runs:
ps ax | grep apache | grep -v grep | wc -l
The relevant information is kept inside the Apache configuration file:
/etc/apache2/apache2.conf

# Server query
host www.slackware.com
dig www.slackware.com
nslookup www.slackware.com

# Manage processes
When an executable program starts up, it runs a process and this process will
be under the management of your Linux system's process table.

The ps and top commands are very powerful for viewing information about the
running processes. Then there are commands such as nice and renice and they
can be used for raising and lowering the process's priority. You can also move
the processes to run in the background, or back to run in the foreground again.

A command -chrt- can be used to run processes in real-time. You can use the
kill and killall commands to send signals to processes by PID or by name.

The ps command gives a snapshot of currently running processes whereas the
top command offers a screen-oriented, constantly updated listing of the
running commands.

ps # displays the list of processes of the current user at the current shell
ps -u username # specific user's running process
ps -u username u # displays all the running processes with CPU and memory info
ps -e # displays every running process
ps -el # long list of every running process
ps -ef # displays a full format list of all running processes
ps -eF # extra full format list of all running processes

Some processes start up other processes. The hierarchy of processes can be
viewed by using several options witht the ps command.

# shows the process hierarchy with information on process and session IDs.
ps -ejH

# default delay to update is 3 seconds; this command changes it to 5 seconds
top -d 5
# view only the processes related to the specific user
top -u username
# this will refresh the screen ten times before quitting
top -n 10
# run in batch mode
top -b

# exiftool
# to read the entire history of an image
exiftool filename.jpg
# to scrub all the personalizing metadata from the image
exiftool "-all=" filename.jpg
# to read the entire history of an mp3 file
exiftool filename.mp3
exiftool filename.pdf

# ffmpeg
ffmpeg -i filename.mp3/filename.mp4 # to read entire history

# xmonad
--recompile       Recompiles your configuration in ~/.xmonad/xmonad.hs
--restart         Causes the currently running xmonad process to restart
--replace         Replace the current window manager with xmonad
--version         Display version of xmonad
--verbose-version Display detailed version of xmonad

Default keyboard bindings
mod-shift-return         Launch terminal
mod-p                    Launch dmenu
mod-shift-p              Launch gmrun
mod-shift-c              Close the focused window
mod-space                Rotate through the available layout algorithms
mod-shift-space          Reset the layouts on the current workspace to default
mod-n                    Resize viewed windows to the correct size
mod-tab                  Move focus to the next window
mod-shift-tab            Move focus to the previous window
mod-j                    Move focus to the next window
mod-k                    Move focus to the previous window
mod-m                    Move focus to the master window
mod-return               Swap the focused window and the master window
mod-shift-j              Swap the focused window with the next window
mod-shift-k              Swap the focused window with the previous window
mod-h                    Shrink the master area
mod-l                    Expand the master area
mod-t                    Push window back into tiling
mod-comma                Increment the number of windows in the master area
mod-period               Deincrement the number of windows in the master area
mod-shift-q              Quit xmonad
mod-q                    Restart xmonad
mod-shift-slash          Run xmessage with a summary of the default keybindings
mod-[1..9]               Switch to workspace N
mod-shift-[1..9]         Move client to workspace N
mod-{w,e,r}              Switch to physical/Xinerama screens 1, 2, or 3
mod-shift-{w,e,r}        Move client to screen 1, 2, or 3
mod-button1              Set the window to floating mode and move by dragging
mod-button2              Raise the window to the top of the stack
mod-button3              Set the window to floating mode and resize by dragging

Examples
To use xmonad as your window manager add to your ~/.xinitrc file:
exec xmonad

Customization
xmonad is customized in ~/.xmonad/xmonad.hs, and the restarting with mod-q.
You can find many extensions to the core feature set in the xmonad-conrtib
package, available throught your package manager or from xmonad.org

Ref: https://xmonad.org/manpage.html#customization

# System Admin Skills
1. Manage file systems
The single most important file related to file systems is /etc/fstab, which
holds their mount points, their types and various other permissions about
them  - it is the job of the system administrator to alter and maintain the
/etc/fstab file in order to contain the correct information.

https://askubuntu.com/questions/420320/what-permissions-would-change-if-i-
change-umask-to-027-from-022

failed login attempts are logged in /var/log/faillog

https://www.digitalocean.com/community/tutorials/how-to-create-a-sudo-user-on-
ubuntu-quickstart

The magic of SysReq
Linux is usually stable enough to survive misbehaving software, but it is
possible for a program to lock up the whole computer. Before you reach for
the Power or Reset button and risk corrupting your file systems, there is a
better way to get out of it. You can send commands directly to the kernel by
holding down the Alt and SysReq (aka PrtScr) keys and pressing certain letter
keys. As the kernel listens for those directly, they work even if X is
completely locked and accepting no input.

The keys normally used to get out of a lockup are R, to reset the keyboard -
this occasionally fixes the problem on its own. Next, press E to send a TERM
signal to all processes, asking them to shut down cleanly, writing any data
to disk and closing any open files. Next in line is I, which sends a KILL
signal to all remaining processes, forcing them to shut down. Pressing S tells
the kernel to sync, flushing all buffers to disk so that remaining open files
can be closed cleanly. Press U to unmount all file systems and remount them
read-only, to avoid further data corruption. Finally, reboots the system.

So, that is: hold down Alt and SysReq and press R-E-I-S-U-B in turn, preferably
leaving a coubple of seconds between each. There are several, aminly silly,
mnemonics to help remember that sequence, the most appropriate being Reboot
Even if System Utterly Broken, but the easiet way to remember the sequence is
that it is BUSIER backwards. This is not something you should need very often,
but it is well worth remembering when you do.

## cli-magic
w3m http://en.wikipedia.org/wiki/May_06

## Documentation
Debian:
www.debian.org/doc
http://debian-handbook.info
Ubuntu:
https://help.ubuntu.com
https://help.ubuntu.com/community
www.askubuntu.com
Linux Mint:
www.linuxmint.com/documentation.php
OpenSuse:
http://doc.opensuse.org
Fedora:
http://docs.fedoraproject.org
CentOS:
www.centos.org/doc
RedHat:
http://tinyurl.com/rheldocs
Arch:
https://wiki.archlinux.org

## ImageMagick
# Converts from JPEG format into PNG, original intact
convert image.jpg image.png
# can list the supported formats
convert -list format
# displays information about an image in a one-line summary. Add the -verbose
  option to get a very detailed information report.
identify my_image.png
# This gives you a PNG image that is a maximum of 600 pixels wide and 400
  pixels high. It preserves the aspect ratio and won't stretch, compress or
  otherwise distort the image.
convert image.jpg -resize 600*400 image.png
# list color names
convert -list color
# for viewing image sequences(slideshows)
animate -delay 100 *.png

## sxiv
sxiv -f -S 10 * # f - fullscreen, S - slideshow, 10 - 10 seconds, * - files
Enter # to view all images

## Colors
#15242F terminal
#063A76 xmobar

## ispell
ispell filename.txt # to run a spelling check

ispell highlights the first word it doesn't recognize. If the word is
misspelled, ispell usually offers one or more alternatives. Press R and
then the number beside the correct choice. In the screen capture above.
I'd press R and 0 to fix the error.

If, on the other hand, the word is correctly spelled, press A to move to the
next misspelled word. ispell saves your changes, creates a backup of the file
you just checked (with the extension .bak), and shuts down.

You need to tell it a file's format. When starting ispell, add -t for a Tex or
LaTex file, -H for an HTML file, or -n for a groff or troff file. For example,
if you enter
ispell -t myReport.tex, ispell ignores all markup.

If you don't want the backup file that ispell creates after checking a file,
add -x to the command line - for example, ispell -x myFile.txt

You can add a word to a personal word list by pressing I. This saves the word
to a file called .ispell_default in the root of your /home directory.

## system-built
Taskbar: xmobar
http://projects.haskell.org/xmobar/

System tray: stalonetray
http://stalonetray.sourceforge.net/

Application launcher: dmenu
https://tools.suckless.org/dmenu/

Wallpaper setter: feh
https://feh/finalrewind.org/

Time management: calcurse
http://calcurse.org/

Image viewer: sxiv
https://github.com/muennich/sxiv

## Man-pages
Using Git, you can obtain the very latest version of man-pages using the
following command:
$ git clone http://git.kernel.org/pub/scm/docs/man-pages/man-pages

You can see what's recently been pushed to kernel.org via CGit:
http://git.kernel.org/?p=docs/man-pages/man-pages.git

## script - Recording the shell activities
$ script # Script started, file is typescript
Do whatever shell commands under script.
Press Ctrl-D to exit script.
$ vi typescript

## popularity-contest
$ popularity-contest > /var/log/popularity-contest
$ popcon-largest-unused

## apt-mark
To remove package from list of packages considered for removal, mark that
package as installed manually. You can do it using apt-mark:
$ apt-mark manual <package_name>

## finger - displays information about the system users
$ finger # user information lookup program

## clean and lean machine
fslint # a utility to find and clean various forms of lint on a filesystem,
         especially duplicate files and broken symlinks.

debfoster # creates dependency files to enable you to keep track of what you
            have installed.

localepurge # will remove all locale files from your system that you have
              no need for

## Pandoc
pandoc # pandoc is invoked without specifying any input files, it operates as
         a filter, taking input from the terminal and sending its output back
         to the terminal

By default, input is interpreted as pandoc markdown, and output is HTML.
# converting from HTML to markdown (hit Ctrl-D)
pandoc -f html -t markdown
# converting from markdown to LaTeX
pandoc -f markdown -t latex
# convert markdown file to HTML
pandoc test1.md -f markdown -t html -s -o test1.html
# convert markdown file to LaTeX
pandoc test1.md -f markdown -t latex -s -o test1.tex
To create a PDF, you'll need to have LaTeX installed.
pandoc test1.md -s -o test1.pdf
pandoc --help

## update-alternatives command
$ ls -l $(type -p vi)
$ sudo update-alternatives --display vi
$ sudo update-alternatives --config vi

The Debian alternatives system keeps its selection as symlinks in
"/etc/alternatives/". The selection process uses corresponding file
in "/var/lib/dpkg/alternatives/".

## font
# for core X11 font path
xset q
# for fontconfig font default
fc-match
# displays sorted list of best matching fonts, but do not do any pruning
fc-match -a
# displays sorted list of best matching fonts
fc-match -s
# for available fontconfig fonts
fc-list

## watch - repeating a command with a constant interval
$ watch # executes a program repeatedly with a constant interval
$ watch w # this displays who is logged on to the system update every 2 seconds

## scheduling tasks
scheduling tasks once
$ echo 'command -args' | at 3:40 monday

scheduling tasks regularly
$ cron(8)

# splitting a large file into small files
When a data is too big to backup as a single file, you cna backup its content
after splitting it into, e.g. 2000MiB chunks and merge those chunks back into
the original file late.

$ split -b 2000m large_file
$ cat x* > large_file
Please make sure you do not have any files starting with"x" to avoid name
crashes.

## invisible disk space consumption
All deleted but open files consume disk space although they are not visible
from normal du. They can be listed with their size by the following.
# lsof -s -X / | grep deleted
$ lsof -i   # to list all open network connections

## pretty print a manual page
You can print a manual page in PostScript nicely by one of the following
commands.
$ man -Tps some_manpage | lpr
$ man -Tps some_manpage | mpage -2 | lpr # the second example prints 2 pages
                                           on 1 sheet

## merge two ps and pdf files
You can merge two postscript and portable document format files using gs of
ghostscript.

gs -q -dNOPAUSE -dBATCH -sDEVICE=pswrite -sOutputFile=bla.ps -f f1.ps f2.ps
gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=bla.pdf -f f1.pdf f2.pdf

https://www.ibm.com/developerworks/library/l-lpic1-map/index.html

## parted - a partition manipulation program
parted --list # lists partition layout on all block devices
parted --help # displays a help message
parted --machine # displays machine parseable output

## lpinfo - show available devices or drivers
lpinfo -v # list all devices
lpinfo -m # list all drivers
lpinfo --make-and-model "HP LaserJet" -m # list drivers matching "HP LaserJet"

## Shell Scripting Tricks
# To see how much disk space is left
df
df -h # for a human readable format in megs & gigs

# To see where all my big files are
cd /somewhere
du -sk * | sort -nr
Repeat as necessary

# To see big directories instead of files
du -x /somewhere | sort -nr | head -10
This shows the 10 largest directories under /somewhere
Another way:
find /somewhere -size +2000k -ls
This shows all files over 2000 kilobytes under /somewhere
If you want to keep your .debs, you can filter them out of the list:
find / -size +2000k ! -name "*.deb" -ls
This shows all files on your entire system over 2000 kilobytes that
are not debian pkg.

# To rename them all with underscores instead of spaces
rename 's/ /_/g' *.mp3
This rename command is not a general Unix command but it's included with Perl
which is of course installed by default in Debian.

# To do that recursively
That's a bit tricky. You can get a list of files to pass to rename using find,
but if some of the directories are being renamed as well, that's not something
rename can keep track of. You have to use -depth with find to make sure any
files are renamed before the directories they're in.
cd /somewhere
find . -depth -name '* *' -type f -print0 | xargs -r0 rename 's/ /_/g'

# Systemd
systemd-analyze
systemd-analyze blame (results pipe to less by default, can use --no-pager)
systemd-analyze plot > /tmp/boot_plot.svg
# systemctl (not to be confused with sysctl) is used to manage units
systemctl start sshd.service     # starting a unit
systemctl stop sshd.service      # stopping a unit
systemctl restart sshd.service   # restarting a unit
systemctl enable sshd.service    # enable a unit to start at boot
systemctl disable sshd.service   # disabling service so it does not run at boot
systemctl cat sshd.service       # displaying the contents of a unit file
systemctl status sshd.service    # to get information about a unit
systemctl suspend     # sleep (suspend to RAM)
systemctl hibernate   # hibernate (suspend to Disk)
#  /sbin/shutdown task
systemctl reboot      # reboot
systemctl halt        # halt system (without powering off)
systemctl poweroff    # power off system
systemctl --type service --state active                   # list services
systemctl list-unit-files --type=service                  # list all services
systemctl list-dependencies multi-user.target --no-pager  # list all services by target
systemctl list-units --type target # view list of currently loaded targets
systemctl    # lots of options...
systemctl-   # lots of new commands...
# journalctl is used to interact with the journal
journalctl /path/to/executable  # show all messages by a specific executable
journalctl _PID=456             # show all messages by a specific PID (ex. 456)
journalctl _SYSTEMD_INIT=sshd.service # show all messages by a specific unit
journalctl                  # show all messages in journal (view basic logs)
journalctl -n 10            # view most recent logs (-f to follow)
journalctl -p err           # filter by priority
journalctl -p err --since "2015-2-17 18:00:00" --until "2015-2-17 18:10:00"
                                      # filter by time and priority
# similar to the tail command, the -f flag can be used to follow the journal,
while the -n flag can be used to limit results to a number of most recent
messages, run man journalctl for the full list of options

# mpv
mpv [options] filename
mpv --shuffle /folder-path/*
mpv --playlist=filename.pls
# to activate the gapless audio optiona
mpv --gapless-audio --playlist=filename.pls
mpv --help
mpv --list-options
mpv link_to_youtube_vid --no-video  # audio only format

# route
route -n

# nslookup - query Internet name servers interactively
nslookup <url>
nslookup -query=mx <url>
nslookup -type=ns <url>   # query name server
nslookup -type=any <url>  # query DNS record
nslookup -type-soa <url>  # query start of authority
nslookup -port 56 <url?   # query port number

# dig - a tool for querying DNS nameservers for information about host address,
        mail exchanges, nameservers, and related information
dig <url>
dig <url> +nocomments    # turn off comment lines
dig <url> +noauthority   # turn off authrity section
dig <url> +noadditional  # turn off additional section
dig <url> +nostats       # turn off stats section
dig <url> +noanswer      # turn off answer section
dig <url> +noall         # disable all section at once

# uptime - verify what happened actually when the server was unattended
# wall "mesg" - sends mesg to everybody logged with their mesg permission yes
# mesg [n|y] - contrl people who can use the "write" command
# write <user_name> - send text directly to the screen of another linux machine
# talk <username> - lets you talk to the logged in users
# w - combination of uptime and who commands

# mysqldump
mysqldump -u root -p -all-databases > /home/server/Desktop/backupfile.sql

# mkpasswd
mkpasswd -l 10

# imagemagick
convert source.jpg -colorspace Gray destination.jpg # true grayscale only
convert source.jpg -monochrome destination.jpg # true black and white
convert source.jpg -separate destination.jpg # separate into gray channels
identify <image> # list size of image
convert -crop +0+100 <input_image> <output_image> # crop the image from top
convert -crop +0-100 <input_image> <output_image> # crop the image from bottom
convert -crop -5+0 <input_image> <output_image> # crop the image from right
convert -crop +5+0 <input_image> <output_image> # crop the image from left


# Installing the same selection of packages several times
It can be useful to systematically install the same list of packages on several
computers. This can be done quite easily.

First, retrieve the list of packages installed on the computer which will
serve as the "model" to copy.

$ dpkg --get-selections > pkg-list

The pkg-list file then contains the list of installed packages. Next, transfer
the pkg-list file onto the computers you want to update and use the following
commands:

## Update dpkg's database of known packages
# avail='mktemp'
# apt-cache dumpavail > "$avail"
# dpkg --merge-avail "$avail"
# rm -f "$avail"
## Update dpkg's selections
# dpkg --set-selections < pkg-list
## Ask apt-get to install the selected packages
# apt-get dselect-upgrade

The first commands records the list of available packages in the dpkg database,
then dpkg --set-selections restores the selection of packages that you wish to
install, and the apt-get invocation executes the required operations! aptitude
does not have this command.
[Sources: pg. 108 The Debian Administrator's Handbook]

# deborphan
deborphan -sz
sudo apt remove --purge $(deborphan)

$ count the number of sub-directories
find ./* -type d | wc -l
Note: Remember to use ./* if you don't want to include the current directory
itself in the total

Note: Always read offical installation guide for Debian/Ubuntu

# find
find . -print # print all files and directories in current directory
find . -type f -ls # print all files with details
find . -type f # print only files in current directory, no directory
find . -type d # print only directories in current directory, no files
find ~ -name bu -type d # print only 'bu' directories in ~
find . -name my_file.txt # print only my_file.txt in .
find ~ -name '*.txt' # search for '*.txt' in ~
find . -iname '*.jpg' # find files by case-insensitive extension (ex: .jpg)
find . -type f -perm 777 # to find files by octal permission
find . -size +10M -print # find all files larger than 10M
find . -type f -mmin -60 # find all files that were changed within last 60 min.
find . -type f -exec mv {} {}.py # add all files with ".py" at the end
find . -type f -name "*cmd*" # find files whose names have the string "cmd"
# find files whose names have the string
find . -type f -name "*cmd*" or -name "*line*"
# find files whose names have the string
find . -type f -name "*cmd*" and -name "*line*"
# find all directories in the current directory and their disk usage
find * -prune -type d -exec du -skh {} \;

# wn - command line interface to WordNet lexical database
wn perusing -over # to find definition of word perusing

# Text
less -M # for file name, the current and total lines

iostat

# history
history -c # to clear shell history
ctrl P / ctrl N # backward & forward of commands in history in the shell prompt

# wn - command line interface to WordNet lexical database
wn <word> -over  # read wn man page

# nmap - Network exploration tool and security/port scanner
nmap -h # Nmap help
nmap scanme.nmap.org        # default scan 1000 ports
nmap -p 22 scanme.nmap.org  # scan port 22 alone
nmap -A scanme.nmap.org     # OS information
nmap -F scanme.nmap.org     # most important targeted ports
nmap -sV scanme.nmap.org    # short information
nmap --open scanme.nmap.org # open ports alone
nmap -A -T4 slackware.com
ifconfig
ipcalc 192.168.42.136
nmap -F 192.168.42.8/24
sudo arp-scan -l
https://nmap.org/
https://www.nirsoft.net/countryip/  # Major IP Addresses Blocks by Country
who is <ip_address> # in web browser
geoiplookup.net/ip-addresses/<ip_address>

# checking package integrity
The md5sum command is used to compute a 128-bit fingerprint that is strongly
dependant upon the contents of the file to which it is applied. It can be used
to compare against a previously-generated sum to determine whether the file
has changed. It is commonly used to ensure the integrity of updated packages
distributed by a vendor:
sudo md5sum <package-name>

# ufw
sudo ufw app list  # available applications
sudo ufw allow OpenSSH  # to allow SSH connections
sudo ufw enable  # enable the firewall
sudo ufw status  # status of the firewall
sudo ufw status verbose

Block an IP Address
To block all network connections that originate from a specific IP address,
15.15.15.51
$ sudo ufw deny from 15.15.15.51
[source IP address - 15.15.15.51, subnet IP address - 15.15.15.0/24]
Note: the source IP address can be specified in any firewall rule

Block connections to a Network interface:
To block connections from a specific IP address, eg. 15.15.15.51, to a specific
network interface, eg. eth0, use this command.
$ sudo ufw deny in on eth0 from 15.15.15.51
This is the same as the previous example, with the addition of in on eth0.
The netwoek interface can be specified in any firewall rule and is a great
way to limit the rule to a particular network.

Reference:
https://www.digitalocean.com/community/tutorials/ufw-essentials-common-
    firewall-rules-and-commands
https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-
    with-ufw-on-ubuntu-16-04

# apt-file
apt-file is a command line tool for searching packages in Debian repositories
sudo apt install apt-file   # to install apt-file
sudo apt-file update   # to update apt-file database
apt-file search <package_name>  # to query the database for <package_name>
apt-file search path/to/<package_name>
apt-file list <package_name>  # to list the content of an uninstalled package
sudo apt-file purge  # to delete the cache files


# user
# add user Mark [-m tells Linux to create home directory]
sudo useradd -m mark
# to create password for Mark
sudo passwd mark
# to remove a user by name mark
sudo userdel mark
# set date on which the account will expire
sudo usermod -e 2017-12-31 mark
# to change Mark's login name to Henry
sudo usermod -l henry mark
# to temporarily lock Mark out of his account
sudo usermod -L mark
# using -U will unlock and -G will add/remove membership group
sudo usermod -G sudo mark
# to force users to change password (no < 5 but no > 30 days)
sudo chage -m 5 -M 30 mark
# to send Mark a warning seven days before his deadline
sudo chage -W 7 mark
# to list Mark's current settings
sudo chage --list mark
Note: Referance Practical_LPIC-1.pdf (page:102)

# user and permissions
## create a user called ashley
useradd --create-home ashley
## create a user in groups "users" and "dev"
useradd --create-home --groups users dev ashley
## create a user with the UID 1337
useradd --create-home --uid 1337 ashley
## create or change a user password
passwd ashley
<enter password>
## add user to a group "coder"
usermod --append --groups coder ashley
## remove a user from a group "coder"
gpasswd --delete ashley coder
## delete a user from the system
userdel ashley
## delete a user and all user data from the system
userdel --remove ashley
## show which users are currently logged in
w
## show login history
last
## set default permission of files to 644
echo "umask 022" >> /etc/profile
## set default permission of files to 664
echo "umask 002" >> /etc/profile
## change ownership of a file "example.txt" to user "ashley"
chown ashley:users example.txt
## give read permission to user, group, and others
chmod ugo+r example.txt
## give write permissions to user and group
chmod ug+w example.txt
## remove write permission to a file for group members
chmod g-w example.txt
## make a file executable
chmod +x example.txt
## log in as a different user
su - ashley
## run a command "ls" as a different user
su - ashley --command ls

# vagrant

# cfdisk

# Basic text analysis with command line tools in Linux
Book from Internet Archive for text analysis.
Jane Andrews, The Stories Mother Nature Told Her Children (1888, 1894)
(Project Gutenberg collection)
wget http://archive.org/download/thestoriesmother05792gut/stmtn10.txt
file stmtn10.txt
stmtn10.txt: C source, ASCII text, with CRLF line terminators

Note: The output of the file command confirms that this is an ASCII text,
guesses that it is some code in the C programming language (which is incorrect)
and tells us that the ends of the lines in the file are coded with both a
carriage return and a line feed.

This is standard for window computers. Linux expect the ends of lines in an
ASCII text file to be coded only with a line feed. If we want to move text
files between operating systems, this is one thing we have to pay attention to.

The head & tail commands show us the first few and last few lines of the file.
head stmtn10.txt
tail stmtn10.txt

Before modifying files, it is usually a good idea to make a copy of the
original. We can do this with the cp command.
cp stmtn10.txt stmtn10-backup.txt
less -N stmtn10.txt # option -N to include line numbers at the beginning
                      of each line

# command to delete all of the material between lines 2206 and 2525
sed '2206,2525d' stmtn10.txt > stmtn10-nofooter.txt # sed - Linux stream editor
sed '1,40d' stmtn10-nofooter.txt > stmtn10-trimmed.txt

wc -l stmtn10-trimmed.txt # number of lines
wc -m stmtn10-trimmed.txt # number of characters

grep # pattern-matching command
# command below shows lines which contain the word giant,
  this pattern is case sensitive
grep -n "giant" stmtn10-trimmed.txt # option -n to include line numbers
grep -E -n "(G|g)iant" stmtn10-trimmed.txt
grep -i, --ignore-case
grep -o
grep -v
grep -A NUM, --after-context=NUM
grep -B NUM, --before-context=NUM
grep -C NUM, -NUM, --context=NUM

# When we are analyzing the words in a text, it is usually convenient to
create a standardized version that eliminates whitespace and punctuation
and converts all characters to lowercase. We use the tr command to translate
and delete characters of our trimmed text, to create a standardized version.
Note that in this case the tr command requires that we use the redirection
operators to specify both the input file (<) and the output file (>).

# delete all punctuation
tr -d [:punct:] < stmtn10-trimmed.txt > stmtn10-nopunct.txt
# upper to lower
tr [:upper:] [:lower:] < stmtn10-nopunct.txt > stmtn10-lowercase.txt

# Counting word frequencies
# translate ' ' to '\n'
tr ' ' '\n' < stmtn10-lowercaself.txt > stmtn10-oneword.txt
# to sort that file so the words are in alphabetical order,
  and so that if a given word appears a number of times, these
  are listed one after another.
sort stmtn10-newword.txt > stmtn10-onewordsort.txt
# uniq command with the -c option to count the number of repetitions
  of each line
uniq -c stmtn10-onewordsort.txt > stmtn10-wordfreq.txt
# all commands in one line
tr ' ' '\n' < stmtn10-lowercaself.txt | sort | uniq -c > stmtn10-wordfreq2.txt
# search for a particular term with the forward slash (/giant, for example)
Ref: https://williamjturkel.net/2013/06/15/basic-text-analysis-with-command-
line-tools-in-linux/

# Virtual Machine
virtinst - Program to create and clone virtual machines
virt-install
virt-manager

# Running Linux as a guest VM
Reference: Linux_Kernel_Programming.pdf (page no. 14)

# pdftotext
pdftotext -layout report.pdf report.txt

# calcurse
calcurse -t   # List all todo items
calcurse -a   # Print the appointments and events for the current day and exit
calcurse -r7  # Print appointments and events for the next seven days

# Reviewing what you did today
# To list all the files in your home directory tree, excluding direcotries,
that were accessed today: $ find ` -daystart -atime -1 \! -type d
# To list all the files in your home directory tree, excluding directories,
that were modified today: $ find ~ -daystart -mtime -1 \! -type d
# To list all the files in your home directory tree, excluding diretories,
that were accessed between now and two days ago:
$ find ~ -daystart -atime -2 \! -type d
# To list all the files in your home directory tree, excluding directories,
that were modified exactly two days ago: $ find ~ -daystart -mtime 2 \! -type d
# To find out what you did on September 11, 2001, type:
  $ touch -d "Sep 11 2001" /tmp/911
  $ touch -d "Sep 12 2001" /tmp/912
  $ find ~ -daystart -newer /tmp/911 \! -daystart -newer /tmp/912

# expand and unexpand
expand -t 10 filename        # convert every tab to ten spaces
unexpand -t 3 filename       # convert every three spaces to tabs

# format text
fmt -w 60 filename           # start a new line after 60 characters
fmt -t filename              # indent all but the first line of a paragraph

# docker
Method 1: Managing services in linux with systemd
systemctl list-unit-files --type service -all  # list all services
sudo systemctl | grep running  # list running services
systemctl start docker.service # start a service
systemctl start docker.socket
systemctl stop docker.service  # stop a service
systemctl stop docker.socket
systemctl restart docker.service # restart a service
systemctl restart docker.socket
systemctl status docker.service  # check the status of a service
systemctl status docker.socket
Method 2: Managing services in linux with init
service --status-all # list all services ([-] are disabled and [+] are enabled)
service <service-name> start # start a service
service <service-name> stop  # stop a service
service <service-name> restart # restart a service
service <service-name> status  # check the status of a service

# Format USB Drive
Step 1: Insert USB and Identify Volume
Insert a USB drive into your system and identify your USB drive correctly. This
is the step you need to take care, because you may format the wrong disk if not
correctly identify your disk.
$ df -h
Step 2: Format USB Drive in Linux
Whenever we attach a USB drive, it automatically mounted to the system. We can
not format any disk on Linux systems which are already mounted. So first
un-mount /dev/sdc1 USB drive on your system.
$ sudo umount /dev/sdc1
Step 3: Format with vFat file system  $ sudo mkfs.vfat /dev/sdc1
        Format with NTFS file system  $ sudo mkfs.ntfs /dev/sdc1
        Format with EXT4 file system  $ sudo mkfs.ext4 /dev/sdc1

# Translate html to text
cat index.html | w3m -dump -T text/html > index.txt

# Permissions and their likely uses  (chmod 700 /home/user/file)
Permission    Octal_Code   Meaning
rwxrwxrwx     777          Read, write, and execute permissions for all users
rwxr-xr-x     755          Read and execute permission for all users. The file
                           owner also has write permission.
rwxr-x---     750          Read and execute permission for the owner and group.
                           The file's owner also has write permission. Users
                           who are not the file's owner or members of the group
                           have no access to the file.
rwx------     700          Read, write, and execute permissions for the file's
                           owner only; all others have no access.
rw-rw-rw-     666          Read and write permissions for all users. No execute
                           permissions to anybody.
rw-rw-r--     664          Read and write permissions to the owner and group.
                           Read-only permission to all others.
rw-rw----     660          Read and write permissions to the woner and group.
                           No world permissions.
rw-r--r--     644          Read and write permissions to the owner. Read-only
                           permission to all others.
rw-r-----     640          Read and write permissions to the owner, and read-
                           only permission to the group. No permission to other
rw-------     600          Read and write permissions to the owner. No permiss-
                           ion to anybody else.
r--------     400          Read permission to the owner. No permission to any-
                           body else.

# netstat
netstat -p > net_connections.txt   # redirect stdoutput to file
netstat -p 2> error_messages.txt   # redirect stderror to file
netstat -p &> net_connections.txt  # redirect stdoutput and stderror to file

# Common Environment variables and their meanings
Variable_Name     Explanation
$USER             This is your current username. It's a variable that's
                  maintained by the system.
$SHELL            This variable holds the path to the current command shell.
$PWD              This is the present working directory. This environment
                  variable is maintained by the system. Programs may use it to
                  search for files when you don't provide a complete pathname.
$HOSTNAME         This is the current TCP/IP hostname of the computer.
$PATH             This is an unusually important environment variable. It sets
                  the path for a session; the path is a coon-delimited list of
                  directories in which Linux searches for executable programs
                  when you type a program name. For instance, if PATH is
                  /bin:/usr/bin and you type ls, Linux looks for an executable
                  program called ls in /bin and then in /usr/bin If the command
                  you type isn't on the path, Linux responds with a command not
                  found error. The $PATH variable is typically built up in
                  several configuration files, such as /etc/profile and the
                  .bashrc file in the user's home directory.
$HOME             This variable points to your home directory. Some programs
                  use it to help them look for configuration files or as a
                  default location in which to store files.
$LD_LIBRARY_PATH  A few programs use this environment variable to indicate
                  directories in which library files may be found. It wroks
                  much like PATH.
$PS1              This is the default prompt in bash. It generally includes
                  variables of its own, such as \u (for the username),
                  \h (for the hostname), and \# (for the current working
                  directory). This value is frequently set in /etc/profile,
                  but it is often overridden by users.
$TERM             This variable is the name of the current terminal type.
$DISPLAY          This variable identifies the display used by X. It's usually
                  :0.0, which means the first (numbered from 0) display on the
                  current computer. When you use X in a networked environment,
                  though, this value mey be preceded by the name of the
                  computer at which you're sitting, as in machine4.example.com:
                  0.0. This value is set automatically when you log in, but you
                  may change it if necessary. You can run multiple X sessions
                  on one computer, in which case each one gets a different
                  DISPLAY number - for instance, :0.0 for the first session
                  and :1.0 for the second.
$EDITOR           Some programs launch the program pointed to by this
                  environment variable when they need to call a text editor
                  for you to use.

# Verify that the hash matches the downloaded image:
You need to verify that the hash matches the downloaded image, and then verify
that the hash was signed by an official Debian key - as explained in
(https://linuxconfig.org/how-to-verify-an-authenticity-of-downloaded-debian-
iso-images).
1. Download your CD image, a SHA 512 hash, and the hash signature. (debian.org)
2. Verify that the hash matches the image
$ sha512sum debian-8.3.0-i386-netinst.iso > my_hash.txt
$ diff -q my_hash.txt SHA512SUMS
3. Verify the hash is properly signed. You'll probably have to do it twice:
once to get the key ID, and again after you have downloaded the public key.
$ gpg --verify SHA512SUMS.sign SHA512SUMS
gpg: Signature made Mon 25 Jan 2016 05:00:46 AEDT using RSA key ID 6294BE9B
$ gpg --keyserver keyring.debian.org --recv 6294B#9B
$ gpg --verify SHA512SUMS.sign SHA512SUMS
4. Verify that the key fingerprint (the last printed line) is legitimate.
Ideally, you should do this via a web of trust
(https://www.gnupg.org/gph/en/manual/x547.html).
However you can check the key fingerprint against the keys listed on Debian's
website (https://www.debian.org/CD/verify).
Ref: https://unix.stackexchange.com/questions/138603/how-to-verify-debian-iso-
     integrity


# Check the Hard Disk Serial Number in Linux
# to know the device names of your disks
df -h
# displays the hard drive model and vendor but not serial number
hwinfo
# displays the list of hardware installed on your machine
lshw
# displays the serial number
lshw -class disk
# is a tool part of the smartmontools package (sudo apt install smartmontools)
smartctl
# displays the serial number
smartctl -i /dev/sda
# displays the serial number
hdparm -i /dev/sda

# Hex codes
Hex: #101718  RGBA(16,23,24,1)
HeX: #1f2a2e  RGBA(31,42,46,1)

# awk
awk 'NR==100' filename.txt # print text in line number 100 from filename.txt
awk 'NR>=100 && NR<=110' filename.txt # print text between lines 100 and 110

# set default web browser
sudo update-alternatives --list x-www-browser
sudo update-alternatives --config x-www-browser

# xdg-open
xdg-open file.csv # opens file.csv in default application

# tee
echo "institute of mathematical sciences" | tee -a institute.txt

# display cursor size in debian
gsettings get org.gnome.desktop.interface cursor-size
gsettings set org.gnome.desktop.interface cursor-size 20

# show history of recently installed packages, their version number, and the
date/time they were installed:
grep "install" /var/log/dpkg.log

# show a list of recently upgraded packages, the date/time they were upgraded,
as well as the old and new package version:
grep "upgrade" /var/log/dpkg.log

# show a history of recently removed packages and the date/time they were
removed:
grep "remove" /var/log/dpkg.log

# /var/log/dpkg.log contains the package install, update and remove history
for the current month. For the previous month, read the /var/log/dpkg.log.1
log file.
grep "install" /var/log/dpkg.log.1

# want to go back even more in the dpkg history? Use zgrep instead of grep,
and read /var/log/dpkg.log.2.gz, /var/log/dpkg.log.3.gz, /var/log/dpkg.log.4.gz
and so on, which go back two, three and respectively four months:
zgrep "upgrade" /var/log/dpkg.log.2.gz

This is because by default, the dpkg log is set to rotate once a month, keeping
12 old logs (so for 12 months), and compress rotated files by using gzip (.gz).
You can check the Debian Logrotate configuration for dpkg by using cat
/etc/logrotate.d/dpkg

# to check sudo access for a user
sudo -l

# to check init or systemd
ps --no-headers -o comm 1

# init service commands
sudo service ssh status/start/stop/restart/reload/condrestart

# public IP address
curl ifconfig.me
# private IP address
hostname -I
ip addr
ifconfig

# rename user in linux
usermod -l nname oname

# ssh with port number
ssh -p 22 user@hostname

# cat two files
cat -n {file1.txt,file2.txt}
cat -n file1.txt file2.txt

# make directory with mode
mkdir -m777 <directoryname>

# MySQL
sudo dpkg-reconfigure mysql-apt-config
apt-policy mysql-server


dpkg -l | grep -i 'mysql'

# List only sub-directories of the current directory
ls -d */

# qutebrowser
.venv/bin/python -m qutebrowser --temp-basedir

# change bash shell prompt
PS1="g62231tx:~$ "

# extract .7z file
sudo apt-get install p7zip p7zip-full p7zip-rar
man p7zip; man 7za
7za e <filename>.7z

# reference the last argument of the previous command
ls Downloads/
cd $_

# work on files and directories
mkdir -v dir_{rpm,txt,zip,pdf}

# find command by example
find . -type f -name '*statistical*' | less -N
find . -iname '*statistical*' | less -N

# xdg-open
xdg-open <filename>

# access last argument of the previous command
mkdir Example
cd !$  (equivalent to 'cd Example')

# shellcheck
shellcheck file.sh

# change to previous directory and change to directory
cd ../Directory

# locate command
sudo updatedb  # update database
locate -i 'searchtext'

# aspell
echo word | aspell -a
aspell check text.txt

# path of the command
type -P <command>
which <command>

# display one-line manual page descriptions
whatis <command>

# linux version
cat /etc/os-release
hostnamectl
lsb_release -a
uname -r

# merge images in command line
convert image1.png image2.png image3.png -append Sprite/verticalsprite.png
convert image1.png image2.png image3.png +append Sprite/horizontalsprite.png

convert image{1..12}.png -append Sprite/verticalsprite.png
convert image{1..12}.png +append Sprite/horizontalsprite.png

gm convert image1.png image2.png image3.png -append Sprite/verticalsprite.png
gm convert image1.png image2.png image3.png +append Sprite/horizontalsprite.png

# resize an image [Ref: https://imagemagick.org/]
magick mogrify -resize 50% image.jpg     # reduce the size of image.jpg by 50%
magick mogrify -resize 256x256 image.jpg # resize image.jpg to a dimension of 256x256
magick mogrify -format jpg image.png     # convert PNG format image.png to the JPEG format

convert input.png -resize 200x100 output.png   # preserve the aspect ratio
convert input.png -resize 200x100! output.png  # don't preserve the aspect ratio
convert input.png -resize 200 output.png       # resize an image to a width of 200
convert input.png -resize x100 output.png      # resize an image to a height of 100

# rotate an image
convert input.jpg -rotate 180 output.jpg
for image in *.jpg; do convert $image -rotate 90 $image; done
mogrify -rotate 90 image.jpg

# applying effects
convert input.png -charcoal 2 output.png # apply charcoal effect to an image (2=strength)
convert input.png -implode 1 output.png  # apply implode effect to an image

# combining operations
convert input.png -resize 400x400 -rotate 180 -charcoal 4 -quality 95 output.png

# xset - user preference utility for X
xset q - information on the current settings

# package management
aptitude update # updating package database
aptitude upgrade --simulate # showing available updates
aptitude install pkg1 pkg2 # updating a particular package
apt-get install pkg1 pkg2 # installing packages
apt-get install --reinstall pkg1 pkg1 # reinstalling a particular package
apt-cache search <searchword> # searching the package database
apt-cache search --full --names-only <searchword> # long package information
apt-get remove package1 package2 # removing packages
apt-get remove --purge package1 package2 # remove along with all configuration files
apt-get install --download-only pkg1 pkg2 # only downloading packages
apt-get clean # cleaning up downloaded packages
apt-get autoclean # remove outdated packages

# 50 linux commands
Linux commands for navigating the file system:
pwd - print work directory
ls - list files
cd - change directory
mkdir - make directory
rmdir - remove empty directory
lsblk - list block devices
mount - mount SD card or USB in existing filesystem
df - information about the  disk space on filesystem
Linux commands for manipulating the system:
uname - system information (name, version, ...)
ps - processes currently running
kill - stop processes that are struck due to resource constraints
service - invoke system-wide services (System V init)
batch - run system services in apre-defined schedule (automation shell scripts)
shutdown - shutdown the system
Linux commands for taking care of your files:
touch - creating a valid empty file (command for changing timestamps)
cat - concatenating multiple files
head - view the beginning of a file
tail - vies the ending of a file
cp - copy a file or directory
mv - move/rename a file or directory
comm - compare two files for common and distinct lines
less - viewing the contents of a file
ln - creating symbolic links to a file
cmp - compare two files and print the result tot the standard output stream
dd - copy and convert files from one type to another (creating bootable live USB sticks)
alias - replace a word by another string in files directly from the terminal
Funny Linux commands to explore:
cal - calendar in an ASCII text format
fortune - print poignant, inspirational, or silly phrases in terminal
history - history of terminal sessions
yes - ouput a string repeatedly until killed
banner - ASCII banners
rev - print text reversing each character
Linux commands for network administrators:
wget - download files from the web from the terminal
iptables - control the incoming and outgoing internet traffic on a host machine
traceroute - determine the route a network packet takes on its way form machine to another
curl - transferring files over a network (network related shell scripts)
Linux commands for search and regular expression:
find - search for files based on file permissions, ownership, modification date, size, ...
which - locate the path of a command
locate - location of a specific file by regular expression
grep - search for patterns
sed - manipulate a file or stream by replacing specific parts
Linux commands that deal with I/O and ownership:
clear - clear out existing terminal screen
echo - output a specific text to the terminal console
sort - sort out a file in an alphabetical or reverse manner
sudo - lets non-privileged users access
chmod - change/modify the access permissions of system files or directory
chown - change/modify the ownership of a file or directory
Linux command miscellaneous for use:
man - manual
tar - archiving files and extracting them
whatis - display one-line manual page descriptions

# column
dpkg -S $(which column)
dpkg -l | grep -i 'util-linux'
man column
column -t /etc/passwd
column -t -s ":" -n /etc/passwd
column -e -t -s "," file.txt
column -x file.txt

# rotate screen in linux with xrandr
xrandr -o left      # orientation left
xrandr -o right     # orientation right
xrandr -o normal    # orientation normal

# sponge command to replace changes/manipulates in data file
column -t data.csv | sponge data.csv

# list installed deb packages by size
dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n
wajig large
dpigs
dpkg-query -W --showformat='$PInstalled-Size}\t${Package}\n' | sort -nr | less

glimpse
nethogs

# display the GPU info
sudo lshw -C display -short
lspci -v | less

# GPU performance testing tool
apt-get install mesa-utils
glxgears

# command-line based download accelerator
sudo apt-get install axel
axel https://releases.ubuntu.com/20.04.2.0/ubuntu-20.04.2.0-desktop-amd64.iso

# uuencode
sudo apt-get install sharutils

# identify (required imagemagick)
identify -verbose <filename.jpg>

# ffprobe
ffprobe <filename.mp3>  # information regarding audio file

# mediainfo
mediainfo <filename.mp3>

# sox
soxi <filename.mp3>

# unoconv - convert any document from and to any LibreOffice supported format

# change default cursor theme
list of cursors theme available:
crystalcursors  dmz-cursor-theme    xcursor-themes  comixcursors    Chamelon-cursor-theme
installing crystalcursors:
sudo apt-get install crystalcursors
run the update-alternative to set/change the current-cursor-theme:
sudo update-alternatives --config x-cursor-theme

# hide the cursor pointer
sudo apt-get install unclutter
put the following in a script 'unclutter.sh': unclutter -idle 1 -root -grab -visible
chmod u+x unclutter.sh
unclutter &
[add this line to ~/.xinitrc: unclutter &]
to stop unclutter: sudo killall unclutter
put "SDL_VIDEO_X11_DGAMOUSE=0" in ~/.profile [Ref: https://wiki.debian.org/unclutter]

# copy and paste with xclip
$ xclip -sel clip ~/.ssh/id_ed25519.pub
$ tail -n 30 logfile.log | xclip -sel clip
$ pandoc -t html file.md | xclip -sel clip
$ cat /etc/apt/sources.list | xclip -sel clip
$ xclip -o -sel clip > file.txt

# list installed packages in DEB based systems
apt-cache policy
/etc/apt/sources.list.d/

# tar files:
1. compress an entire directory /home/saran/Tardirectory/
$ tar -czvf file.tar.gz /home/saran/Tardirectory/
2. compress multiple directories and files:
$ tar -czvf file.tar.gz /home/saran/Documents/ /home/saran/Music/
/home/saran/Pictures/image.png
3. bzip2 compression instead of gzip by passing -j option:
$ tar -cjvf file.tar.bz2 /home/saran/Tardirectory/
4. exclude files when using tar:
$ tar -czvf archive.tar.gz --exclude='file.txt' /home/saran/Directory/
5. exclude ~/Downloads directory:
$ tar -czvf /nfs/backup.tar.gz --exclude="Downloads" /home/saran/
6. view files stored in an archive:
$ tar -ztvf file.tar.gz
$ tar -jtvf file.tar.bz2
7. extracting an archive:
$ tar -xzvf file.tar.gz
$ tar -xjvf file.tar.bz2
8. extract the contents of the archive into a specific directory:
$ tar -xzvf file.tar.gz -C /home/saran/Backup/
$ tar -xjvf file.tar.bz2 -C /home/saran/Backup/

# check ip address in termex: (check for 'UP')
ip -br addr | grep -v DOWN
sshd    # start ssh demeon in termex

# ssh to termex:
ssh -p 8022 u0_a499@192.168.43.128    [5Tw@4rEWn3]
scp -P 8022 file.tar.gz u0_a499@192.168.43.128:~/Tardirectory/
scp -v -P 8022 file.tar.gz u0_a499@192.168.43.128:~/Tardirectory/

ssh -p 8022 u0_a275@192.168.29.121 # realme

# bootable usb
sudo mkfs -t vfat /dev/sdb1
sudo umount /dev/sdb1 /media/user/
sudo dd bs=4M if=/home/saran/file.iso of=/dev/sdb1 status=progress oflag=sync

# command
command ls
command -V ls

# check which shells are installed
cat /etc/shells

# analsys of text speech
cat speech.txt | tr ' ' '\n' | tr -d '[:punct:]' | tr '[A-Z]' '[a-z]' | sort | \
 uniq -c | sort -nr | less

# directory usage
du -sh * | less

mv `ls | grep -i 'cheat'` /path/to/directory/

# number of lines in a file
nl <filename>

# graphics card [Ref: https://wiki.debian.org/GraphicsCard]
If your graphic card was manufactured in 2007 and newer, try uninstalling the
xserver-xorg-video-intel package and use the builtin modesetting driver xserver-xorg-core
instead. For older devices: xserver-xorg-video-intel

# change console font type and size
In Debian, you can use "console-setup" configuration file for "setupcon" which is used to
configure font and keyboard layout for the console. The standard location of the
console-setup configuration file is /etc/default/console-setup

sudo dpkg-reconfigure console-setup

# factor
factor 12

# hostnamectl
1. static hostname
2. icon name
3. chassis
4. machine id
5. boot id
6. operating system
7. kernel
8. architecture

# debian version
cat /etc/debian_version

# install kernel headers
apt-get install linux-headers-$(uname -r)

# getting started with dpkg
dpkg --get-selections   # listing installed software packages
dpkg --print-avail      # display available version details about installed packages
dpkg-query --list       # display software package information in a tabular form
sudo dpkg --install ~/Downloads/qutebrowser_amd64.deb # installing software packages
sudo apt install -f     # restoring dependencies
dpkg --audit            # to check for broken packages (missing dependencies)
sudo dpkg-reconfigure virtualbox-6.1    # re-configuring a software package
sudo dpkg --remove qutebrowser          # removing software package
sudo dpkg --purge qutebrowser           # removing software package and its configurations
sudo dpkg --info ~/Downloads/qutebrowser_amd64.deb  # information about a software package
dpkg --contents ~/Downloads/qutebrowser_amd64.deb   # inspecting software package content
sudo dpkg --status zathura              # finding out if a package is installed
dpkg --help                             # to learn more about the dpkg command
man dpkg                                # for more detailed info about the dpkg command

# fslint (whereis fslint) [Note: read man fslint]
/usr/share/fslint/fslint/findup     - find duplicate files
/usr/share/fslint/fslint/findnl     - find name lint (problems with filenames)
/usr/share/fslint/fslint/findu8     - find filenames with invalid utf8 encoding
/usr/share/fslint/fslint/findbl     - find bad links (various problems with symlinks)
/usr/share/fslint/fslint/findsn     - find same name (problems with clashing names)
/usr/share/fslint/fslint/finded     - find empty directories
/usr/share/fslint/fslint/findid     - find files with dead user IDs
/usr/share/fslint/fslint/findns     - find non stripped executables
/usr/share/fslint/fslint/findrs     - find redundant whitespace in files
/usr/share/fslint/fslint/findtf     - find temporary files
/usr/share/fslint/fslint/findul     - find possibly unused libraries
/usr/share/fslint/fslint/zipdir     - reclaim wasted space in ext2 directory entries

# check disk partitioning scheme in command line (check system uses MBR or GPT partition)
sudo parted -l  # in the output, look for the line starting with Partition Table

# check if you are using UEFI or BIOS
ls /sys/firmware/efi        # the directory will be missing if system is using BIOS
sudo apt-get install efibootmgr
sudo efibootmgr             # to check UEFI or BIOS

# check whether computer is 32-bit or 64-bit
lscpu   # if 64-bit system, it's better (and recommended) to install 64-bit OS
uname -m
arch

# uname
uname -s    # kernel name
uname -r    # kernel release
uname -v    # kernel version
uname -n    # network node name (hostname)
uname -m    # machine architecture
uname -p    # processor architecture
uname -i    # hardware platform (os architecture)
uname -o    # operating system

# check weather in terminal
curl wttr.in/chennai

# display worldmap in terminal
telnet mapscii.me

# view package update history
grep installed /var/log/dpkg.log
# view the previous install log
grep installed /var/log/dpkg.log.l
# view the archived logs
zgrep installed /var/log/dpkg.log.2.gz
# view apt history
/var/log/apt/history.log

# cd to raman's home directory
cd ~raman

# correct and tidy up the markup of HTML, XHTML, and XML files
tidy file.html

# print sensor chips information (e.g. temperature)
sensors

# perf - performance analysis tools for linux
sudo apt-get install linux-perf
sudo sh -c 'echo 1 > /proc/sys/kernel/perf_event_paranoid' # run perf as normal user

# performance counters of a program using perf
perf stat -e cycles,instructions,cache-references,cache-misses,branches,branch-misses,\
task-clock,faults,minor-faults,cs,migrations python program.py

# using cat to edit a file
cat > filename      # edit a file
cat >> filename     # append to a file
ctrl + D            # save and exist the file

# display file size
ls -sh filename
du -sh filename

# find the MAC address
ip addr sh              [lint/ether MACID]
ip link show            [link/ether MACID]
sudo lshw -C network    [serial: MACID]

# network interface [/etc/network/interfaces]   [/etc/network/interfaces.d/setup]
## The loopback network interface
auto lo
iface lo inet loopback

## The eno1 network interface
auto eno1
allow-hotplug eno1
iface eno1 inet dhcp

## The enp0s20u1 network interface
auto enp0s20u1
allow-hotplug enp0s20u1
iface enp0s20u1 inet dhcp

## The enp0s20u2 network interface
auto enp0s20u2
allow-hotplug enp0s20u2
iface enp0s20u2 inet dhcp

# "no public key available" on apt-get update [Ref: https://keyring.debian.org/]
Debian Public Key Server
gpg --keyserver keyring.debian.org --recv-keys <PUBLIC KEY>

# To run history event number 1, type:
$ !1

# find the number of CPU cores
cat /proc/cpuinfo
# find the number of physical CPUs
cat /proc/cpuinfo | grep "^physical id" | sort | uniq | wc -l
# find the number of cores per CPU
cat /proc/cpuinfo | grep "^cpu cores" | uniq
# find the total number of processors
cat /proc/cpuinfo | grep "^processor" | wc -l

# to check which program is using a port
sudo lsof -i:<port_number> -n -P
# to list all open TCP ports along with their associated process
sudo lsof -i -n -P | grep TCP

# enable X11 forwarding using SSH
X11 forwarding is a mechanism that allows graphical interfaces of X11 programs running on
a remote server to be displayed on a local client machine.

## Server-side preparation:
sudo apt-get install xauth

Enable X11 forwarding system-wide:
sudo vi /etc/ssh/sshd_config
X11Forwarding yes

sudo /etc/init.d/sshd restart

Enable X11 forwarding on a per-user basis:
vi $HOME/.ssh/config
ForwardX11 yes

## Client-side test
Unlike a remote server host, a local client machine needs to have X11 system installed.
ssh -X user@remote_server

# speed up X11 forwarding in SSH over wide area networks
use less computation heavy ciphers in SSH, the default 'AES' cipher used by OpenSSH is
known to be slow. 'arcfour' and 'blowfish' ciphers are faster than 'AES'. 'blowfish' is a
fast block cipher which is also very secure, 'arcfour' stream cipher is known to be nore
vulnerable than common block ciphers.
$ ssh -XC -c blowfish-cbc,arcfour user@remote_server

## specify these options in an SSH configuration file
to edit system-wide SSH configuration file
$ sudo vi /etc/ssh/ssh_config
to edit per-user SSH configuration file
$ vi ~/.ssh/config
in either SSH configuration file, add the following:
Host remote_server
    Compression yes
    ForwardX11 yes
    Ciphers blowfish-cbc,arcfour
then SSH to the remote host without using any command-line options
$ ssh user@remote_server
Note that there are some caveats in switching to a different cipher in SSH. The
performance of a particular cipher may vary across different processor architecture. In
recent generations of Intel processors come with hardware support for 'AES'.

# build RPM or DEB package from the source with checkinstall
installating a linux program from its source by running 'make install', it becomes really
tricky to remove it completely, unless the author of the program provides an uninstall
target in the Makefile.

installing checkinstall in debian
sudo apt-get install checkinstall

build an DEB package with checkinstall
sudo ./configure
sudo checkinstall

# check battery status using command line
cat /sys/class/power_supply/BAT0/capacity   # for the battery capacity
find /sys/class/power_supply/BAT0/ -type f | xargs -tn1 cat     # detailed battery info
Ref: https://www.linuxjournal.com/content/how-check-battery-status-using-linux-command-line

# scan IP address connected to the network
sudo nmap -sn <ip address being used locally>
Example: sudo nmap -sn 192.168.30.0/22

# nmap commands
nmap 192.168.1.1
sudo nmap -O -Pn 192.168.1.1
nmap -Pn -p 22,24 192.168.1.1
nmap -F 192.168.1.1
nmap -Pn --open 192.168.1.1

# display file system type
df -Th
sudo fsck -N /dev/sda1
lsblk -f
sudo file -sL /dev/sda1

# df commands to check disk space
df                          # check file system disk space usage
df -a                       # display information of all file system disk space usage
df -h                       # show disk space usage in human readable format
df -hT /home                # display information of /home file system
df -k                       # display information of file system in bytes
df -m                       # display information of file system in MB
df -h                       # display information of file system in GB
df -i                       # display file system inodes
df -T                       # display file system type
df -t ext3                  # include certain file system type
df -x ext3                  # exclude certain file system type
df --help                   # display information of df command

# system and hardware info
uname                       # system name
uname -s                    # kernel name
uname -n                    # network hostname
uname -v                    # kernel version
uname -r                    # kernel release
uname -m                    # machine hardware name
uname -a                    # all the above info
sudo lshw                   # print information about Linux system hardware
sudo lshw -short            # print a summary of hardware information
sudo lshw -html > lshw.html # generate output as html file
lscpu                       # information about CPU
lsblk                       # information about block devices
lsblk -a                    # view all block devices on system
lsusb                       # information about USB controllers and all devices connected
lsusb -v                    # generate detailed information about each USB device
lspci                       # print information about PCI devices
lspci -t                    # produce output in tree format
lspci -v                    # produce detailed information about each connected device
lsscsi                      # print SCSI devices information
lsscsi -s                   # show device sizes
sudo hdparm /dev/sda1       # information about sata devices on system
sudo hdparm -g /dev/sda1    # information about device geometry
sudo fdisk -l               # information file system partitions
sudo dmidecode -t memory    # print information about memory
sudo dmidecode -t system    # print information about system
sudo dmidecode -t bios      # print information about bios
sudo dmidecode -t processor # print information about processor

# print how many seconds the current shell has been running
echo $SECONDS
units '1624 seconds' 'minutes'

# outputing a random number
echo $RANDOM
yes `echo $RANDOM`

# controlling processes
ps          # list of the processes running on system
ps -aux     # list all running processes
ps -a       # list all processors from all users
ps -u       # list more information including user names, %cpu usage, and %mem usage etc
ps -x       # list processes without controlling terminals
ps -l       # display information including UID and nice value
ps --forest # to see the process hierarchy
pstree      # displays the processes in the form of a tree structure
pgrep <process_name>    # finding the process id of a particular process
pgrep -u <user_name> -l <process_name> # find the process id of particular process by user

# command to find function meanings
declare

# network manager
sudo apt-get install network-manager
sudo systemctl restart network-manager

# make unmanaged network interface to connected with nmcli
[check first if the ip address is displayed then follow the steps below]
nmcli dev status    # if [device] eno1 [type] ethernet [state] unmanaged
sudo vi /etc/NetworkManager/NetworkManager.conf

[ifupdown]
managed=true        # change managed=false to managed=true

sudo systemctl restart network-manager      # restart the network manager
nmcli dev status    # check the interface status
ping -c 3 startpage.com     # check if able to ping

# exit an unresponsive ssh session without closing the terminal
press [Enter] ----> press [~] ----> press [.]

# recursively lists package dependencies
apt-rdepends <package_name>

# check installed architectures
# verify 64-bit architecture
dpkg --print-architecture   # should respond with "amd64"
# verify if 32-bit architecture is already installed
dpkg --print-foreign-architectures  # should respond with "i386"
# if it does not display "i386"
sudo dpkg --add-architecture i386
# re-check
dpkg --print-foreign-architectures

# convert-im6.q16: attempt to perform an operation not allowed by the security policy
ImageMagick disables many ghostscript format types.
This can be seen in the file: /etc/ImageMagick-6/policy.xml
<!-- disable ghostscript format types -->
<policy domain="coder" rights="none" pattern="ps" />
<policy domain="coder" rights="none" pattern="EPS" />
<policy domain="coder" rights="none" pattern="PDF" />
<policy domain="coder" rights="none" pattern="XPS" />
Modify this to allow: rights="read"

# transform a text file into an image
convert -size 360x360 xc:white -font "FreeMono" -pointsize 12 -fill black
-draw @file.txt image.png

convert -size 360x360 xc:white -font "FreeMono" -pointsize 12 -fill black
-annotate +15+15 @file.txt image.png

convert -size 1000x2000 xc:white -font "FreeMono" -pointsize 12 -fill black
-annotate +15+15 @file.txt -trim -bordercolor "#FFF" -border 10 +repage img.png

convert -list font # display all of the possible arguments to -font

Reference:
https://unix.stackexchange.com/questions/138804/how-to-transform-a-text-file-
into-a-picture

# manually installing Xorg (https://adercon.com/posts/manually-installing-xorg)
Xorg install with a simple 'apt-get install xorg' pulls in all drivers - most
of which not need and install Xorg in a minimalistic approach.

First need to find the video card (and driver) for the video card:

* find video card: lspci | grep -i 'vga'
The result of the above command should be something similar to the following:
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor
Graphics Controller (rev 09)
In the above output, "00:02.0" is the device id.

* get card details: lspci -v -s DEVICE-ID
The above command should return something similar to the following:
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor
Graphics Controller (rev 09) (prog-if 00 [VGA controller])
 Subsystem: Hewlett-Packard Company 3rd Gen Core processor Graphics Controller
    Flags: bus master, fast devsel, latency 0, IRQ 29
    Memory at c3000000 (64-bit, non-prefetchable) [size=4M]
    Memory at b0000000 (64-bit, prefetchable) [size=256M]
    I/O ports at 5000 [size=64]
    [virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
    Capabilities: <access denied>
    Kernel driver in use: i915
    Kernel modules: i915

* you want to note the vendor ("Intel Corporation")
  and the kernel driver ("1915")

* now install Xorg with the following:
apt-get install xserver-xorg-core xserver-xorg-input-all
xserver-xorg-video-fbdev xserver-xorg-video-(yourcard:
                                             can be intel, nouveau, or ati)

# remove all Xorg
The following code will remove xorg, display managers, etc. and leave you
with a more clean/trim server:
apt-get remove --purge xorg*
apt-get remove --purge nautilus*
apt-get remove --purge desktop-base
apt-get remove --purge python-gnome*
apt-get remove --purge gdm3
apt-get remove --purge x11-*
apt-get remove --purge gnome-*
apt-get autoremove --purge

# cleanup unused package configs
# (https://adercon.com/posts/cleanup-unused-package-configs/)
apt-get purge `dpkg --get-selections | grep deinstall | cut -f1`

# Possible missing firmware /lib/firmware/rtl_nic/rtl8105e-1.fw of module r8169
sudo apt-get install firmware-realtek
Ref:
https://unix.stackexchange.com/questions/682990/linux-upgrade-debian-possible-
missing-firmware-problem

# for info on Graphics
inxi -G

# convert epub to pdf
https://ebook.online-convert.com/convert-to-pdf
Base font size: 10
Embed font: Serif
Change input encoding: Automatic detection
Enable ASCII output
Enable heuristics
Target ebook reader: Default
Add border: 0-20 cm

# patching st terminal
$ git clone https://git.suckless.org/st
$ mv st/ .st/
$ cd .st/
$ wget https://st.suckless.org/patches/anysize/st-expected-anysize-0.9.diff
$ patch -Np1 -i st-expected-anysize-0.9.diff (in directory ~/.st/)
$ doas make clean install   [Note: edit config.h before make clean install]

# tmux not sourcing my .tmux.conf
Binding keys in tmux: https://www.seanh.cc/2020/12/28/binding-keys-in-tmux/
tmux ls
tmux kill-server

Ref:
https://unix.stackexchange.com/questions/66606/tmux-not-sourcing-my-tmux-conf/

# mod + ctrl + (workspace)

# top command options
top                     # to launch top
press the z key         # to change the color of the output
press the 1 key         # to see a graphical representation of each CPU core
press the m key         # display memory usage graphically
top -u 'username'       # display processes started by a specific user
top -i                  # list of idle processes on your system
top -d 5                # update interval to an arbitrary value in seconds
top -n 2                # number of iterations to two and then exits
Press 'shift + L' to locate a process by name. This creates a prompt just above
the bold table header line. Type in the name of the process you're looking for
and then press 'Enter' to see the instances of that process highlighted in the
newly sorted process list.
Stop or 'kill' a running process. Find the process you want to stop using
'shift + L'. Next, press K and enter the process ID you want to stop.
Ref: https://opensource.com/article/22/3/linux-top-command

# change font type and size in st terminal
st -f "Inconsolata:style=Regular:size=14"

# compare two files
cmp file1.txt file2.txt     # if file1.txt == file2.txt then no output printed

# display the differences between two files
diff file1.txt file2.txt    # if file1.txt == file2.txt then no output printed

# croping pdf file
pdfcrop --margins '-100 -15 -100 -490' input.pdf output.pdf

# google sign-in issues
for versions between v1.10.0 (inclusive) and v2.3.1 (exclusive), run
:set -u https://accounts.google.com/* content.headers.user_agent
"Mozilla/5.0 ({0s_info}; rv:90.0) Gecko/20100101 Firefox/90.0" for versions
between v1.2.0 (inclusive) and v1.10.0 (exclusive), use X11; Linux x86_64 in
place of {os_info}.

# config in qutebrowser
:set zoom.default 70
:set url.start_pages "https://startpage.com/{}"

~/.config/qutebrowser/config.py
config.load_autoconfig()
c.url.start_pages =  ["https://startpage.com/{}"]
c.url.searchengines = {
    "DEFAULT": "https://startpage.com/{}"
}


# install liberation mono   (https://wiki.debian.org/Fonts)
sudo apt-get install fonts-liberation/fonts-liberation2

# ssh
sudo apt-get update
sudo apt-get install openssh-server
sudo systemctl status ssh
sudo systemctl start ssh    # in case not started, start ssh service
sudo systemctl enable ssh   # to enable the ssh service on system boot
ssh saravanan@10.21.106.28  # login the remote machine

# scp
scp file.txt saravanan@10.21.106.28:~/Documents/    # copy local file to remote
scp saravanan@10.21.106.28:~/Documents/file.txt /home/saran/ # remote to local

# find number of CPU cores
nproc command shows the number of processing units available
$ nproc
lscpu gathers CPU architecture information from /proc/cpuinfo in human-readable
$ lscpu
/proc/cpuinfo and sysfs stores info about CPU architecture like number of CPUs,
threads, cores, sockets, NUMA nodes, information about CPU caches, CPU family
less /proc/cpuinfo

# youtube-dl
xclip -o -selection clipboard > url.txt
youtube-dl -F $(< url.txt)
youtube-dl -f bestaudio+bestvideo $(< url.txt)

# list of sound cards
$ cat /proc/asound/cards
$ lspci | grep -i audio
$ aplay --list-devices

# setting the default device sound card
Find your desired card with:
$ cat /proc/asound/cards
and then create /etc/asound.conf with following:
defaults.pcm.card 1
defaults.ctl.card 1
Note: Replace "1" with number of your card determined above.
Ref: https://www.alsa-project.org/wiki/Setting_the_default_device

                -----------------------------
    SOFTWARE    |       Applications        |
                -----------------------------
                              |
                -----------------------------
                |   PipeWire  |     JACK    |
SYSTEM SOFTWARE -----------------------------
SOUND SERVERS   |       PulseAudio          |
                -----------------------------
                              |
                -----------------------------
                |      Advanced Linux       |
KERNEL, DRIVERS |    Sound Architecture     |
                |           (ALSA)          |
                -----------------------------
                              |
                -----------------------------
                |  Equalizer  | Sound board |
                -----------------------------
    HARDWARE    |   Mixer     | Sound card  |
                -----------------------------
                |            ...            |
                -----------------------------

# enable sound in debian linux
$ doas apt-get install alsa-utils
$ alsamixer
press F6 to select sound card [HDA Intel PCH]
Ref: https://vitux.com/how-to-control-audio-on-the-debian-command-line/

# using amixer to control audio properties through commands
$ amixer --help
$ amixer -c [card-number] set [control] [value] # amixer command example
$ amixer scontrols # view the list of controls
$ amixer -c 0 set Master 100% # set the volume on the Master of 1st sound card to 100%
$ amixer -c 1 set Speaker 50% # set the volume on the Speaker of 2nd sound card to 50%
$ amixer -c 1 set Speaker 3db # set the volume on the Speaker of 2nd sound card to 3db
$ amixer -c 1 set Speaker 2db+ # increase the volume on the Speaker of 2nd card by 2db
$ amixer -c 0 set Mic mute # mute the Mic of 1st sound card
$ amixer -c 0 set Mic unmute # unmute the Mic of 1st sound card

# pipewire
$ doas apt-get install pipewire-audio

# change cursor theme
/usr/share/icons/default/index.theme
/usr/share/icons/Adwaita/cursor.theme

# git commands
git config
git config --global init.defaultBranch master

# remove welcome screen text in termux
Open termux app and type:
cd ../usr/etc/
rm motd

# Fonts
# Console /usr/share/consolefonts/Lat15-Fixed16.psf.gz
Encoding to use on the console: UTF-8
Please choose the character set that should be supported by the console font:
    Latin1 and Latin5 - western Europe and Turkic languages
Font for the console: Fixed
Font size: 8x16
# St terminal
apt-get install fonts-unifont
st -f "Unifont:pixelsize=16:antialias=true:autohint=true"

# set console fonts
setfont /usr/share/consolefonts/Lat15-Fixed16.psf.gz

# commandline system information
inxi --recommends   # after installation check dependencies & directories
inxi            # show linux system information
inxi -S         # show linux kernel and distribution info
inxi -M         # show linux machine information
inxi -C         # show CPU and CPU speed information
inxi -G         # show graphic card information
inxi -A         # show audio/sound card information
inxi -N         # show network card information
inxi -D         # show hard disk information
inxi -b         # summarize full system information
inxi -p         # show hard disk partition details
inxi -F         # show full system information
inxi -I         # monitor linux processes memory usage
inxi -t c       # monitoring processes by CPU usage
inxi -t m       # monitoring processes by memory usage
inxi -t cm      # monitoring processes by CPU and memory usage
inxi -t cm10    # monitoring top 10 most active processes eating CPU and memory
inxi -Nni       # monitor linux network interfaces
inxi -s         # monitor linux CPU temperature and fan speed
inxi -w         # show weather report (inxi -W Tokyo,Japan)
inxi -r         # show all linux repository information
inxi -B         # show battery information
inxi -V         # show inxi version
inxi -h         # show inxi help

# get the GPU information
sudo lshw -C display    # shows product, vendor, physical id, bus info, clock
glxinfo                 # OpenGL information for the graphics processor
nvidia-smi              # show information for nvidia gpu
lspci | grep -i 'nvidia'# device id of VGA compatible controller & audio device
cat /proc/driver/nvidia/gpus/<device id>/information # gpu model and uuid
nvidia-smi -q           # display more detailed gpu information
nvidia-smi --list-gpus  # list names of the video cards on the machine
gpustat -cp             # gpu memory information

# installing custom fonts
* create a folder .fonts in $HOME directory
* download fonts of choice (ttf or odf format to ~/.fonts)
* refresh the fonts cache with fc-cache -fv
* verify if fonts got added to the cache using fc-list
  example: fc-list -f "%{family}\n" | sort -u | grep -i 'inconsolata'

# system reboot with users logedin
systemctl reboot -i

# finding nvidia cuda version
nvcc --version  # nvidia cuda compiler
nvidia-smi      # to check CUDA version
cat /usr/lib/cuda/version.txt (find /usr -type d -name cuda; whereis cuda)
cat "$(dpkg -L nvidia-cuda-toolkit | grep 'version.txt')"
apt info nvidia-cuda-toolkit

# check nvidia driver supports kernel version
uname -r
dkms status
apt-cache policy nvidia-driver-470

# nvidia driver is not loaded
sudo prime-select nvidia
sudo reboot

# soft link creation for nvcc
sudo ln -s /usr/local/cuda-11.4/bin/nvcc /usr/bin/nvcc

# issue: # updates could not be installed automatically.
For more details, see /var/log/unattended-upgrades/unattended-upgrades.log

Find the pin in /etc/apt/preferences.d: ls /etc/apt/preferences.d

We can obtain the packages that have been kept back:
/var/lib/unattended-upgrades/kept-back

solution 1: sudo apt-get --with-new-pkgs upgrade
solution 2: sudo apt-get install nvidia-driver-470
solution 3: sudo apt-get dist-upgrade
solution 4: sudo apt purge cuda*
            sudo rm /etc/apt/preferences.d/cuda-repository-pin-600
            sudo apt autoremove
            sudo apt autoclean

using apt-get install <pkg> instead will mark <pkg> as 'manually installed !'
to mark it again as 'automatically installed' use apt-mark auto <pkg>
Ref: https://askubuntu.com/questions/601/the-following-packages-have-been-kept-
back-why-and-how-do-i-solve-it
https://askubuntu.com/questions/1344136/unattended-upgrades-could-not-be-
installed-automatically

profiling python program

# Failed to connect to https://changelogs.ubuntu.com/meta-release-lts.
# Check your Internet connection or proxy settings.
sudo truncate -s 0 /var/lib/ubuntu-release-upgrader/release-upgrade-available

# aptitude safe-upgrade instead of apt-get dist-upgrade
sudo aptitude safe-upgrade

# fonts selection
fc-match sans-serif
fc-match serif
fc-match monospace

# convert pdf to jpg
pdftoppm -jpeg -r 300 file.pdf output

# darkweb definition (Ref: page: 6, Weaving the Dark Web)
Although Freenet, Tor and I2P implement anonymous web technologies in very
different ways, they all provide readers and publishers with anonymity by
allowing them to browse and publish anonymous websites. Here, the connotation
of "darkness" in Dark Web has more to do with encryption, annonymization, and
leaving standard communications channels (as in the phrase used by James Comey,
"going dark", meaning avoiding overly public communications channels). "Web"
refers, of course, to websites, web browsers, HTML, and CSS.

If anything, the Dark Web functions much like the regular web - with the key
exception that one needs special routing software to access it, software that
can protect the identity of site readers and publishers. It is not deeper than
the regular web in any logical sense.

# xterm
xterm -bg black -fg white -fa 'Iosevka' -fs 10

# bash shell tricks
Ctrl + x and Ctrl + V   # find the version of the bash shell
type $ and press tab key twice # system environment variables
type ~ and press tab key twice # show all user from /etc/passwd file
s=$((5*5)) then type echo $s   # arithmetic operation in bash shell
PS1="$ "    # change the bash shell prompt by assigning PS1 variable
env | less  # show the environment
set | less  # display the names and values of shell variables

# paste in st terminal
Ctrl + Shift + V

# removing password from pdf file
gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputfile=unencrypted.pdf -c
.setpdfwrite -f encrypted.pdf

# regex
$ cd nmsc
$ find . -regex ".*leg\.py"
./scripts/python/gaussquad/gaussleg.py
./Endsem/Scripts/gaussleg.py
This search tells find to start from the current directory (the ".") and find
all files that have zero or more characters (the ".*") before the string "leg",
which comes before the "." (which must be escaped because "." normally means
"any single character"), followed by the file extension of python.

# list namespaces
$ sudo lsns

# build html file from rst file
$ rst2html5.py --link-stylesheet --cloak-email-addresses --toc-top-backlinks
--stylesheet=book.css --stylesheet-dirs=. book.rst

# grammer-check a text file
$ diction textfile.txt | less

# grammer checking using gramma (install with nodejs npm)
# check file
gramma check [file]     # interactive fix
gramma check -p [file]  # just print potential mistakes and return status code
# check string
gramma listen [text]    # interactive fix
gramma listen -p [text] # just print potential mistakes and return status code

# apt-cache search lisp scheme
scm - Scheme language interpreter
sloccount - programs for counting physical source lines of code (SLOC)
src2tex - converter from source program files to TeX format files
texify - Beautify source code for use with LaTeX

# set the date for 02:07:30 PM on August 19 without changing the year
$ sudo date 08191407.30

# find difference between two directories using diff
To know what files/subdirectories are common in the two directories, those that
are present in one directory but not in the other. The conventional syntax for
running diff is as follows:
$ diff directory1/ directory2/
By default, its output is ordered alphabetically by files/subdirectory name. In
this command, the -q switch tells diff to report only when files differ.
$ diff -q directory1/ directory2/
Again diff doesn't go into the subdirectories, but we can use the -r switch to
read the subdirectories as well.
$ diff -qr directory1/ directory2/

# numactl
$ numactl --hardware
$ numactl --show

# linux malware incident response tools
netcat - arbitrary TCP and UDP connections and listens
cryptcat - twofish encryption enabled version of nc
memdump - memory dumper
dc3dd - convert and copy a file
linuxinfo - displays system information about system
logname - print user's login name
printenv - print all or part of environment
acct - switch process accounting on or off
fuser - identify processes using files or sockets
route - route traffic control filter
procinfo - tools to display information from /proc and /sys
pslist - utility that controls a process and its descendants
ss - another utility to investigate sockets
users - print the user names of users currently logged in to the current host
w - show who is logged on and what they are doing
w --ip-addr - display IP address instead of hostname for from field
w <username> - show information about the specified user only
moreutils - additional unix utilities (ifdata, ifne, isutf8, lckdo, parallel)

# remove logo with the ffmpeg delogo
ffplay input.mp4 -vf "delogo=x=1400:y=20:w=500:h=480:show=1"

# combine/concat multiple audio tracks into single audio
ffmpeg -f concat -safe 0 -1 list.txt audio.mp3

list.txt contains paths to all segments:
file audiofile1.mp3
file audiofile2.mp3
file audiofile3.mp3

# combine one image + one audio file to make one video using ffmpeg
ffmpeg -r 1 -loop 1 -i image.jpg -i audio.mp3 -acodec copy -r 1 -shortest -vf
scale=1280:720 video.mp4

# combine/concat multiple video files into single video
ffmpeg -f concat -i list.txt video.mp4

list.txt contains paths to all segments:
file videofile1.mp4
file videofile2.mp4
file videofile3.mp4

# 32 bits or 64 bits
$ getconf LONG_BIT

# list all bash shortcuts
$ bind -P

# quick access to ASCII code of a key
$ showkey --ascii           # display the decimal/octal/hex values of the keys

# simulate typing
echo "You can simulate on-screen typing just like in the movies" | pv -qL 10

# set audible alarm when an IP address comes online
ping -i 60 -a IP_address

# base conversions with bc
$ echo "obase=2; 27" | bc -l

# identify long lines in a file
$ awk 'length>80' file.txt

# find if the command has an alias
$ type -all <command> | grep aliased

# dictionaries with dict command (http://www.dict.org/)

Install dict on Linux:
$ sudo apt-get install dictd dict-gcide

No additional configuration is required. The minimalistic
/usr/share/doc/dictd/dict1.conf file specifies the remote server for the
dictionary databases. This tool uses the Dictionary Server Protocol (DCT) on
port 2628.

In a terminal session as a non-root user, type dict <word> to get a list of
definitions from one or more dictionaries and the thesaurus.

$ dict <word> | less

You can also look up multi-word phrases by enclosing them in quotes, either
double or single.

$ dict 'air gapped'

The dict command uses several online dictionaries, including legal & technical
ones. Dictionaries are also available for many languages. You can list the
available dictionary databases as shown here:

$ dict -D | less

You can specify individual dictionaries with the -d option:

$ dict -d gcide


# sharing linux terminals with tmux
[https://www.admin-magazine.com/HPC/Articles/Sharing-Linux-Terminals]

$ tmux new -s remote

tmux new, starts a new session, with the name of the session, remote, placed
after the -s flag.

# yt-dlp
Download the latest version of yt-dlp from GitHub repository:

with curl:
$ doas curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp
    -o /usr/local/bin/yt-dlp

with wget:
$ doas wget -qO /usr/local/bin/yt-dlp
    https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp

$ doas chmod a+rx /usr/local/bin/yt-dlp
$ yt-dlp --help
$ yt-dlp --version
$ doas yt-dlp --update
$ yt-dlp --list-formats "$(< url.txt)"
$ yt-dlp -f 'bv*+ba/b' "$(< url.txt)"
$ doas rm -rf /usr/local/bin/yt-dlp     # uninstall yt-dlp
Reference: https://linuxconfig.org/yt-dlp-vs-youtube-dl

# yt-dlp commands
1. To keep the program up to date, use the -U option
$ yt-dlp -U
2. View the available formats and resolutions of a video with the --list-formats
option
$ yt-dlp --list-formats <url>
3. Download the highest quality video stream and the highest quality audio
stream for a video, and combine them into a single file
$ yt-dlp -f 'bv*+ba' <url>
4. Download 1080p video quality and combine it with the best possible audio
stream
$ yt-dlp -f 'bv*[height=1080]+ba' <url>
5. Save only the audio stream (best quality possible) and put into an mp3 file
$ yt-dlp -f 'ba' -x --audio-format mp3 <url>
6. Playlist video to start at (default is 1)
$ yt-dlp --playlist-start NUMBER <url>
7. Playlist video to end at (default is last)
$ yt-dlp --playlist-end NUMBER <url>
8. Playlist video items to download
$ yt-dlp --playlist-items 1-3,7,10-13 <url> [index: 1, 2, 3, 7, 10, 11, 12, 13]

# report the number of hyperthreads
grep processor /proc/cpuinfo
grep -c processor /proc/cpuinfo

# find exact linux installation date and time
1. using stat command (stat command is part of GNU coreutils package)
$ stat / | grep 'Birth' | sed 's/Birth: //g' | cut -b 2-11
$ stat / | awk '/Birth: /{print $2}'
$ stat -c %w /
2. analyze pacman logs on arch linux
$ head -n1 /var/log/pacman.log
3. check syslog entries on debian-based systems
$ sudo head -n1 /var/log/installer/syslog

# physical cpu cores vs logical cpu cores
(https://unix.stackexchange.com/questions/88283/
so-what-are-logical-cpu-cores-as-opposed-to-physical-cpu-cores)

$ lscpu
Architecture:                    i686
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   36 bits physical, 48 bits virtual
CPU(s):                          2
On-line CPU(s) list:             0,1
Thread(s) per core:              1
Core(s) per socket:              2
Socket(s):                       1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           42
Model name:                      Intel(R) Pentium(R) CPU B950 @ 2.10GHz
Stepping:                        7
CPU MHz:                         798.120
CPU max MHz:                     2100.0000
CPU min MHz:                     800.0000
BogoMIPS:                        4190.20
L1d cache:                       64 KiB
L1i cache:                       64 KiB
L2 cache:                        512 KiB
L3 cache:                        2 MiB

* In the above Intel(R) Pentium(R) laptop has 2 "CPUs" in total.
  CPU(s): 2

* Of which there are 2 physical cores (1 socket x 2 cores/socket = 2 cores)
  Core(s) per socket: 2
  Socket(s): 1

* Of which each can run upto 1 threads
  Thread(s) per core: 1
  at the same time. These threads are the core's logical capabilities.


$ sudo dmidecode | egrep "Socket Designation: Proc|((Thread|Core) Count)"
Socket Designation: Proc 1
        Core Count: 14
        Thread Count: 28
Socket Designation: Proc 2
        Core Count: 14
        Thread Count: 28

* Two sockets.
* Each socket has 14 physical cores.
* Each core has two threads (28/14).
* Total number of logical "cpus" or logical processing units is 56.
  (that's what top and some other commands would show you as number of "cpus")

# show applications using the internet
$ lsof -i | awk '{print $1}' | uniq | tail -n +2
* lsof -i shows us a lot of information pertaining to
  processes accessing the Internet
* awk '{print $1}' filters the last output to only shown
  us the COMMAND column
* uniq filters out multiple occurrences of a single application
* tail -n +2 removes the first COMMAND line

# concatenating pdf files in linux
To convert an image to a pdf we can use imagemagick:
$ convert -quality 100 image.pdf scanned.pdf
To combine or concatenate multiple pdf files, we can use ghostscript:
$ gs -sDEVICE=pdfwrite \
     -sOUTPUTFILE=output.pdf \
     -dNOPAUSE \
     -dBATCH \
     input0.pdf input1.pdf input2.pdf

Flag                Description
-sDEVICE            Device used for processing the output file type. Use
                    pdfwrite to write to a pdf file.
-sOUTPUTFILE        Path to save the resulting file output.
-dNOPAUSE           Disables the prompting and pausing at the end of each page.
-dBATCH             Finishes interpreting after processing the inputted files.

Alternatively you can use pdftk:
$ pdftk input0.pdf input1.pdf input2.pdf \
    cat output output.pdf

Lastly, you can also use imagemagick. Do note, however, that this program
often leads to larger file sizes.
$ convert input0.pdf input1.pdf input2.pdf output.pdf

One issue is that if the pages were of different sizes. This is often because
the pages can be of different pixel densities.

To check run pdfimages and look at the 3rd to last and 2nd to last columns:
$ pdfimages -list filename.pdf

We can then use imagemagick to enforce a certain pixel density. The tradeoff
being that the file size might increase.
$ convert -density 300 input.pdf output.pdf

# brief cheat sheat with curl
$ curl cheat.sh/<command>

# change and print terminal line settings
$ stty --all            # print all current settings in human-readable form
$ stty --save           # print all current settings in a stty-readable form
$ stty --file=DEVICE    # open and use the specified DEVICE instead of stdin
$ stty --help           # display this help and exit
$ stty --version        # output version information and exit

# use plaintext gmail
1. Compose a new message
2. On the bottom right, click the three-dotted icon
3. Select the plaintext option

Gmail will remember your preference next time.
Gmail wraps plain text messages at 78 characters.

# how to add swap space on debian 11
Swap is a portion of hard drive storage that has been set aside for the
operating system to temporarily store data that it can no longer hold in RAM.

checking the system for swap information:
$ sudo swapon --show    # if the system has any configured swap

If you don't get back any output, this means your system does not have swap
space available currently.

$ free -h   # verify that there is no active swap using the free utility

checking available space on the hard drive partition:
$ df -h     # check current disk usage to make sure we have enough space

The device with / in the 'Mounted on' column is our disk in this case.

creating a swap file:
We will allocate a file of the size that we want called swapfile in our root
directory.

$ sudo fallocate -l 4G /swapfile    # creating a swap file with the fallocate

$ ls -lh /swapfile      # verify that the correct amount of space was reserved

enabling the swap file:

$ sudo chmod 600 /swapfile  # make the file only accessible to root

$ ls -lh /swapfile          # verify the permissions change

$ sudo mkswap /swapfile     # mark the file as swap space

$ sudo swapon /swapfile     # enable the swap file, allowing our system to use

$ sudo swapon --show        # verify that the swap is available

$ free -h           # check the output of the free utility again to corroborate

making the swap file permanent:

$ sudo cp /etc/fstab /etc/fstab.bk      # back up the /etc/fstab file

Add the swap file information to the end of your /etc/fstab file
$ echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

adjusting the swappiness property:
The swappiness parameter configures how often your system swaps data out of RAM
to the swap space. This is a value between 0 and 100 that represents a
percentage.

$ cat /proc/sys/vm/swappiness   # the current swappiness value

$ sudo sysctl vm.swappiness=10  # using the sysctl command to set swappiness

This setting will persist until the next reboot. We can set this value
automatically at restart by adding the line to /etc/sysctl.conf file
$ sudo vi /etc/sysctl.conf

At the bottom, you can add
vm.swappiness=10

adjusting the cache pressure setting:
This setting configures how much the system will choose to cache inode and
dentry information over other data.

$ cat /proc/sys/vm/vfs_cache_pressure   # current value by querying the proc

$ sudo sysctl vm.vfs_cache_pressure=50  # set the cache pressure setting value

This setting will persist until the next reboot. We can set this value
automatically at restart by adding the line to /etc/sysctl.conf file
$ sudo vi /etc/sysctl.conf

At the bottom, add the line that specifies the new value
vm.vfs_cache_pressure=50

Ref: digitalocean.com/community/tutorials/how-to-add-swap-space-on-debian-11

# find ip address of remote machine for ssh connection
1. find ip address of local machine
$ ip addr   [192.168.29.175/24]
2. use nmap to find all ip address in the network
$ nmap -sn 192.168.29.0/24

$ nmap -sn 192.168.29.1-254/24 | egrep "scan report" | awk '{print $5}'
$ doas arp-scan --interface=eno1 --localnet # discover all hosts on the local network

# drivers used by sound card
$ lspci -knn | grep -iA2 audio

# swiss army knife
strace
ngrep
netstat
wireshark
eBPF
/proc
perf
ps

# strace - diagnostic, instructional, and debugging tool
$ strace ./revlist.py
$ strace -e file ./revlist.py               # file calls
$ strace -e read ./revlist.py               # read calls
$ strace -e write ./revlist.py              # write calls
$ strace -e network ./revlist.py            # network calls
$ strace -e network curl mark.mcnally.je    # network socket information
$ strace -e connect curl mark.mcnally.je    # connect information
$ strace -p <process id> -e connect         # network calls <process id> making
$ strace -o file.txt ./revlist.py           # save output in file.txt
$ strace -t ./revlist.py                    # include time stamp
$ strace -c ./revlist.py                    # summarize
$ strace -p <vim process id> -e read        # strace vim read calls

# regex
$ grep text file.txt
$ grep text file.txt -o | wc -l
$ grep Text file.txt -o | wc -l
$ grep [tT]ext file.txt -o | wc -l
$ grep "^t" file.txt                        # ^ - matches t at start of the line
$ grep "t$" file.txt                        # $ - matches t at end of the line
$ grep "t." file.txt                        # . - matches t and one char after t
$ grep "t.*" file.txt                       # * - matches t and all other char after t
$ grep -E [t]{2} file.txt                   # repeat t twice (example: tt in twitter)
$ grep -E "^[a-z]*@.*$" file.txt            # grep email address (E - extended)

# fzf - a general-purpose command-line fuzzy finder
# install
$ git clone --depth 1 https://github.com/junegunn/fzf.git ~/.fzf
$ ~/.fzf/install
# upgrading
$ cd ~/.fzf && git pull && ./install
# command-line
$ history | fzf

# find and replace string with sed
Ref: https://linuxize.com/post/how-to-use-sed-to-find-and-replace-string-in-files/

$ sed -i 's/search_regex/replacement/g' *.txt
where,
 -i             By default, sed writes its output to the standard output. This
                option tells sed to edit files in place. If an extension is supplied
                (ex: i.bak), a backup of the original file is created.

 s              The substitute command, probably the most used command in sed.

 / / /          Delimiter character. It can be any character but usually the slash (/)
                character is used.

 search_regex   Normal string or a regular expression to search for.

 replacement    The replacement string.

 g              Global replacement flag. By default, sed reads the file line by line
                and changes only the first occurrence of the search_regex on a line.
                When the replacement flag is provided, all occurrences are replaced.

Note: It is a good practice to put quotes around the argument so the shell
meta-characters won't expand.

# command line tool for manipulating multi-page djvu document
Reference: https://manpages.debian.org/testing/djvulibre-bin/index.html
$ doas apt-get install djvulibre-bin
$ man djvused
$ djvused -e n file.djvu   # print the number of pages of the file file.djvu
$ djvm -delete file.djvu 2 # remove page 2 from file.djvu and save as file.djvu
$ djvused -v file.djvu     # interactive mode
djvused: type "help" to see available commands.
djvused: ok.
<Ctrl> C

# djvu -> ps -> pdf
$ djvups -mode=foreground -gray input.djvu output.ps
$ djvups -mode=black input.djvu output.ps
$ djvups -mode=color input.djvu output.ps   # default -mode=color
$ ps2pdf output.ps

# djvu rendering library
$ doas apt-get install djvulibre-bin
* djvm - manipulate bundled multi-page DjVu documents
* djvudigital - creates DjVu files from PS or PDF files
* djvuextract - extract chunks from DjVu image files
* djvups - convert DjVu documents to PostScript
* djvutoxml - DjVuLibre XML tools
* djvuxmlparser - DjvuLibre XML tools
* djvmcvt - convert multi-page DjVu documents
* djvudump - display internal structure of DjVu files
* djvumake - assemble DjVu image files
* djvused - multi-purpose DjVu document editor
* djvutxt - extract the hidden text from DjVu documents

# compare directories
$ diff -qr /path/to/directory1 /path/to/directory2
$ diff -qrs /path/to/directory1 /path/to/directory2
options:
 -q (--brief) report only when files differ
 -r (--recursive) recursively compare any subdirectories found
 -s (--report-identical-files) report when two files are the same

# pdf rendering library
$ doas apt-get install poppler-utils
* pdfdetach - lists or extracts embedded files (attachments)
* pdffonts - font analyzer
* pdfimages - image extractor
* pdfinfo - document information
* pdfseparate - page extraction tool
* pdfsig - verifies digital signatures
* pdftocairo - PDF to PNG/JPEG/PDF/PS/EPS/SVG converter using Cairo
* pdftohtml - PDF to HTML converter
* pdftoppm - PDF to PPM/PNG/JPEG image converter
* pdftops - PDF to PostScript (PS) converter
* pdftotext - text extraction
* pdfunite - document merging tool

# pdfjam
pdfjam is part of the TeX Live distribution.
$ doas apt-get install texlive-extra-utils
Among other features, it can be used to adjust PDF page sizes with an intuitive
syntax:
$ pdfjam --outfile output.pdf --paper a2paper input.pdf
$ pdfsize output.pdf
The --paper option must be followed by a valid LaTeX paper size. However, we can
use custom sizes with this syntax:
$ pdfjam --outfile output.pdf --papersize '{200mm,350mm}' input.pdf
$ pdfsize output.pdf
Unfortunately, hyperlinks and bookmarks are lost.
$ pdfjam infile1.pdf '{},2-' infile2.pdf '10,3-6' --outfile outfile.pdf
$ pdfjam infile.pdf --fitpaper true '4-' --outfile outfile.pdf

# pdfposter
pdfposter is to create posters by printing multiple pages of a PDF. However, we
can also use it to resize a PDF.
$ doas apt-get install pdfposter
Let's do a US letter to A2 conversion:
$ pdfposter -mA2 input.pdf output.pdf
$ pdfsize output.pdf
We can also specify custom sizes, 200x350mm:
$ pdfposter -m200x350mm input.pdf output.pdf
$ pdfsize output.pdf

# find minimum and maximum values
$ cut -f1 -d ' ' filename.txt | sort -n | head -1   # minimum
$ cut -f1 -d ' ' filename.txt | sort -n | tail -1   # maximum

# scan IP address connected to the network
nmap -sn 192.168.29.0/24 # disable port scanning, host discover only
ssh user@192.168.29.132