This is a quick view (not to say draft) of useful shell commands you might use on your daily tasks while dealing with production environments and developing applications.
Copy key to host :
$ ssh-copy-id user@10.2.2.3
SCP:
scp robot.jar user@example.com:/home/user
Ex : fetch backup from machine :
List files remotely using ssh :
$ ssh user@10.2.2.2 ls -l /home/user/2021-04-22*
fetch the backup :
$ scp user@10.2.2.2:/home/user/2021-04-14/backup.sql.gz backup.sql.gz
Access host via other host :
$ ssh -L 5434:10.2.2.2:5432 comp-portal or $ ssh -L 5434:10.2.2.2:5432 user@10.2.3.3
Now you can get access to 10.2.2.2 while you keep the connection open
scp between 2 hosts
ssh -A dest_host
list and remove a package
$ dpkg -l | grep postgres
SED
https://www.cyberciti.biz/faq/unix-linux-sed-print-only-matching-lines-command/
- Remove all lines starting with #
$ sed ‘/^#/ d’ inputFile.txt > outputFile.txt
- Removing blank lines :
$ sed ‘/^$/d’ main.txt > out.txt
Replace IP in a file :
sed ‘s/[SOME UP]/[OTHER IP]/g’ default.conf
-i option is used to edit in place on the file hello.txt.
-e option indicates the expression/command to run, in this case s/.
Note: It’s important that you use -i -e to search/replace. If you do -ie you create a backup of every file with the letter ‘e’ appended
Trim chars :
$ sed -e ‘s/^[ \t]*//’ $ sed ‘s/ *$//g’
Replace white space with dots :
$ sed -e "s/ /,/g"
Cut some chars :
$ snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2
And with trim and adding . :
$ snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2 | sed -e ‘s/^[ \t]*//’ | sed ‘s/ *$//g’ | sed -e "s/ /./g"
And with no spaces and Hex String to be formatted :
$ snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2 | sed -e ‘s/^[ \t]*//’ | sed ‘s/ *$//g’ | sed -e "s/ //g" | sed ‘s/../0x& /g’
-> Convert now :
snmpwalk -v2c -c community 127.0.0.1:1024 1.2.3.4.5.6 | cut -d’:’ -f2 | sed -e ‘s/^[ \t]*//’ | sed ‘s/ *$//g’ | sed -e "s/ //g" | sed ‘s/../0x& /g’ | xargs printf ‘%d.%d.%d.%d\n’
Add comma at end of every line
sed ‘s/$/,/’ file > out.txt
Add surrounding ‘
sed "s/^/’/;s/$/’/" file > out.txt
Other commands :
Find most recent files (having a day – current dir – not recursive) $ find . -maxdepth 1 -mtime -1
CAT
zcat for compressed files.
$ cat application.log | grep whatever -C 10
CURL
Follow redirect :
$ curl -L google.com
Follow insecure/invalid certif :
$ curl -k https://google.com
Show headers :
$ curl -v google.com
Request with headers (–header or -H) :
$ curl -kLv –header "X-user-Username:z_read_only" -H "X-user-Admin-Secret:thesecret" https://10.2.1.2/web.do
Request with basic auth [pretty using jq]:
$ curl -kLv –user user:pass http://demo.user.com/api/ | jq
Bash aliases
https://doc.ubuntu-fr.org/alias
Example for using VPN -> Add to .bash_aliases :
alias vpn="docker run -it –rm –net host –cap-add NET_ADMIN –device /dev/ppp tianon/dell-netextender netExtend er –username=user –domain=user.com auth-server.com:4433" -v /etc/resolv.conf:/etc/resolv.conf
Tar
tar czf opensuse.tgz opensuse
Screen
Create a new instance :
$ screen
Detach from screen without quit :
Ctrl+A then D
Retrieve back the screen :
$ screen -x
If multiple screens, then a list will be provided :
$ screen -x <id_of_screen>
You might generate a name for your screen :
$ screen -S ‘Gitlab’
Then when getting back to it :
$ screen -x ‘Gitlab’
Whenever a connection issue occurs, the executing processes within a screen are still up and running. Just connect back using ssh and reattach to the screen to access your activities.
For loop
String format
Display as IP with . :
printf ‘%d.%d.%d.%d\n’ 12 12 11 11
REGEX
https://www.digitalocean.com/community/tutorials/using-grep-regular-expressions-to-search-for-text-patterns-in-linux
Use case : read the logs to identify the failing test (GITlab)
Check the log under log_tmp.log (generated from gitlab raw file)
Now issue that :
cat log_tmp.log | grep "Failures: [1-9]" -2
cat logs.txt | grep "Errors: [1-9]" -4
ln – symbolic link
Creating Symlink To a File
To create a symbolic link to a given file, open your terminal and type:
ln -s source_file symbolic_link
Leave a Reply