This is a quick view (not to say draft) of useful shell commands you might use on your daily tasks while dealing with production environments and developing applications.

Copy key to host :

$ ssh-copy-id user@10.2.2.3

SCP:

scp robot.jar user@example.com:/home/user

Ex : fetch backup from machine :

List files remotely using ssh :

$ ssh user@10.2.2.2 ls -l /home/user/2021-04-22*

fetch the backup :

$ scp user@10.2.2.2:/home/user/2021-04-14/backup.sql.gz backup.sql.gz

Access host via other host :

$ ssh -L 5434:10.2.2.2:5432 comp-portal or $ ssh -L 5434:10.2.2.2:5432 user@10.2.3.3

Now you can get access to 10.2.2.2 while you keep the connection open

scp between 2 hosts

ssh -A dest_host

list and remove a package

$ dpkg -l | grep postgres

SED

https://www.cyberciti.biz/faq/unix-linux-sed-print-only-matching-lines-command/

• Remove all lines starting with #

$ sed ‘/^#/ d’ inputFile.txt > outputFile.txt

• Removing blank lines :

$ sed ‘/^$/d’ main.txt > out.txt

Replace IP in a file :

sed ‘s/[SOME UP]/[OTHER IP]/g’ default.conf

-i option is used to edit in place on the file hello.txt.
-e option indicates the expression/command to run, in this case s/.

Note: It’s important that you use -i -e to search/replace. If you do -ie you create a backup of every file with the letter ‘e’ appended

Trim chars :

$ sed -e ‘s/^[ \t]*//’ $ sed ‘s/ *$//g’

Replace white space with dots :

$ sed -e "s/ /,/g"

Cut some chars :

$ snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2

And with trim and adding . :

$ snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2 | sed -e ‘s/^[ \t]*//’ | sed ‘s/ *$//g’ | sed -e "s/ /./g"

And with no spaces and Hex String to be formatted :

$ snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2 | sed -e ‘s/^[ \t]*//’ | sed ‘s/ *$//g’ | sed -e "s/ //g" | sed ‘s/../0x& /g’

-> Convert now :

snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2 | sed -e ‘s/^[ \t]*//’ | sed ‘s/ *$//g’ | sed -e "s/ //g" | sed ‘s/../0x& /g’ | xargs printf ‘%d.%d.%d.%d\n’

Add comma at end of every line

sed ‘s/$/,/’ file > out.txt

Add surrounding ‘

sed "s/^/’/;s/$/’/" file > out.txt

Other commands :

Find most recent files (having a day – current dir – not recursive) $ find . -maxdepth 1 -mtime -1

CAT

zcat for compressed files.

$ cat application.log | grep whatever -C 10

CURL

Follow redirect :

$ curl -L google.com

Follow insecure/invalid certif :

$ curl -k https://google.com

Show headers :

$ curl -v google.com

Request with headers (–header or -H) :

$ curl -kLv –header "X-user-Username:z_read_only" -H "X-user-Admin-Secret:thesecret" https://10.2.1.2/web.do

Request with basic auth [pretty using jq]:

$ curl -kLv –user user:pass http://demo.user.com/api/ | jq

Bash aliases

https://doc.ubuntu-fr.org/alias

Example for using VPN -> Add to .bash_aliases :

alias vpn="docker run -it –rm –net host –cap-add NET_ADMIN –device /dev/ppp tianon/dell-netextender netExtend er –username=user –domain=user.com auth-server.com:4433" -v /etc/resolv.conf:/etc/resolv.conf

Tar

tar czf opensuse.tgz opensuse

Screen

Create a new instance :

$ screen

Detach from screen without quit :

Ctrl+A then D

Retrieve back the screen :

$ screen -x

If multiple screens, then a list will be provided :

$ screen -x <id_of_screen>

You might generate a name for your screen :

$ screen -S ‘Gitlab’

Then when getting back to it :

$ screen -x ‘Gitlab’

Whenever a connection issue occurs, the executing processes within a screen are still up and running. Just connect back using ssh and reattach to the screen to access your activities.

For loop

String format

Display as IP with . :

printf ‘%d.%d.%d.%d\n’ 12 12 11 11

REGEX

https://www.digitalocean.com/community/tutorials/using-grep-regular-expressions-to-search-for-text-patterns-in-linux

Use case : read the logs to identify the failing test (GITlab)

Check the log under log_tmp.log (generated from gitlab raw file)

Now issue that :

cat log_tmp.log | grep "Failures: [1-9]" -2

cat logs.txt | grep "Errors: [1-9]" -4

ln – symbolic link

Creating Symlink To a File

To create a symbolic link to a given file, open your terminal and type:

ln -s source_file symbolic_link

Cat & time