Useful Bash commands

This is a quick view (not to say draft) of useful shell commands you might use on your daily tasks while dealing with production environments and developing applications.

Copy key to host :

$ ssh-copy-id user@10.2.2.3

SCP:

scp robot.jar user@example.com:/home/user

Ex : fetch backup from machine :

List files remotely using ssh :

$ ssh user@10.2.2.2 ls -l /home/user/2021-04-22*

fetch the backup :

$ scp user@10.2.2.2:/home/user/2021-04-14/backup.sql.gz backup.sql.gz

Access host via other host :

$ ssh -L 5434:10.2.2.2:5432 comp-portal or $ ssh -L 5434:10.2.2.2:5432 user@10.2.3.3

Now you can get access to 10.2.2.2 while you keep the connection open

scp between 2 hosts

ssh -A dest_host

list and remove a package

$ dpkg -l | grep postgres

SED

https://www.cyberciti.biz/faq/unix-linux-sed-print-only-matching-lines-command/

  • Remove all lines starting with #

$ sed ‘/^#/ d’ inputFile.txt > outputFile.txt

  • Removing blank lines :

$ sed ‘/^$/d’ main.txt > out.txt

Replace IP in a file :

sed ‘s/[SOME UP]/[OTHER IP]/g’ default.conf

-i option is used to edit in place on the file hello.txt.
-e option indicates the expression/command to run, in this case s/.

Note: It’s important that you use -i -e to search/replace. If you do -ie you create a backup of every file with the letter ‘e’ appended

Trim chars :

$ sed -e ‘s/^[ \t]*//’ $ sed ‘s/ *$//g’

Replace white space with dots :

$ sed -e "s/ /,/g"

Cut some chars :

$ snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2

And with trim and adding . :

$ snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2 | sed -e ‘s/^[ \t]*//’ | sed ‘s/ *$//g’ | sed -e "s/ /./g"

And with no spaces and Hex String to be formatted :

$ snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2 | sed -e ‘s/^[ \t]*//’ | sed ‘s/ *$//g’ | sed -e "s/ //g" | sed ‘s/../0x& /g’

-> Convert now :

snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2 | sed -e ‘s/^[ \t]*//’ | sed ‘s/ *$//g’ | sed -e "s/ //g" | sed ‘s/../0x& /g’ | xargs printf ‘%d.%d.%d.%d\n’

Add comma at end of every line

sed ‘s/$/,/’ file > out.txt

Add surrounding ‘

sed "s/^/’/;s/$/’/" file > out.txt

Other commands :

Find most recent files (having a day – current dir – not recursive) $ find . -maxdepth 1 -mtime -1

CAT

zcat for compressed files.

$ cat application.log | grep whatever -C 10

CURL

Follow redirect :

$ curl -L google.com

Follow insecure/invalid certif :

$ curl -k https://google.com

Show headers :

$ curl -v google.com

Request with headers (–header or -H) :

$ curl -kLv –header "X-user-Username:z_read_only" -H "X-user-Admin-Secret:thesecret" https://10.2.1.2/web.do

Request with basic auth [pretty using jq]:

$ curl -kLv –user user:pass http://demo.user.com/api/ | jq

Bash aliases

https://doc.ubuntu-fr.org/alias

Example for using VPN -> Add to .bash_aliases :

alias vpn="docker run -it –rm –net host –cap-add NET_ADMIN –device /dev/ppp tianon/dell-netextender netExtend er –username=user –domain=user.com auth-server.com:4433" -v /etc/resolv.conf:/etc/resolv.conf

Tar

tar czf opensuse.tgz opensuse

Screen

Create a new instance :

$ screen

Detach from screen without quit :

Ctrl+A then D

Retrieve back the screen :

$ screen -x

If multiple screens, then a list will be provided :

$ screen -x <id_of_screen>

You might generate a name for your screen :

$ screen -S ‘Gitlab’

Then when getting back to it :

$ screen -x ‘Gitlab’

Whenever a connection issue occurs, the executing processes within a screen are still up and running. Just connect back using ssh and reattach to the screen to access your activities.

For loop

String format

Display as IP with . :

printf ‘%d.%d.%d.%d\n’ 12 12 11 11

REGEX

https://www.digitalocean.com/community/tutorials/using-grep-regular-expressions-to-search-for-text-patterns-in-linux

Use case : read the logs to identify the failing test (GITlab)

Check the log under log_tmp.log (generated from gitlab raw file)

Now issue that :

cat log_tmp.log | grep "Failures: [1-9]" -2

cat logs.txt | grep "Errors: [1-9]" -4

ln – symbolic link

Creating Symlink To a File

To create a symbolic link to a given file, open your terminal and type:

ln -s source_file symbolic_link

Cat & time

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Want more information?

Related links will be displayed within articles for you to pick up another good spot to get more details about software development, deployment & monitoring.

Stay tuned by following us on Youtube.

%d bloggers like this: