Linux:Tools

Commands

Quick access

# Scroll through a file with less
less -s myfile.txt

# Select line 5 from the output
cat example.txt | sel -e '5'

# Select lines from the output, starting from the top
cat example.txt | head -5

# Select lines from the output, starting from the bottom
cat example.txt | tail -5

Commands

# Display the full path of a file(assuming the syslog file is available in the current folder)
readlink -f syslog

# Unzip a file
gunzip /var/log/messages.2.gz

Uncommon commands

https://ngelinux.com/what-is-proc-sysrq-trigger-in-linux-and-how-to-use-sysrq-kernel-feature/

# CrASHing THIs SERVer, WiTH no SurVIvORS!
echo c > /proc/sysrq-trigger

Network

ping

# Ping with an interval of 5 seconds
ping -i 5 192.168.0.1

# Ping 8.8.8.8 for 20 times
ping -c 20 8.8.8.8

# Ping google.com using IPv4
ping -4 google.com

# Ping google.com using IPv6
ping -6 google.com

# Ping using packets of size 264
ping -s 264 1.1.1.1

traceroute

Package mtr (My traceroute) is also very good

# Show the traversed hops towards google.com using IPv4
traceroute -4 google.com

# Show the traversed hops towards google.com using IPv6
traceroute -6 google.com

# Does the same as "traceroute -6 google.com"
traceroute6 google.com

# Use ICMP for checking hops
traceroute -4 -I brammerloo.nl

route

# List configured routes
route

# List routes but display IPs instead of hostnames
route -n

# Delete default route 
ip route del 0.0.0.0/0 via 192.168.10.1 dev ens3

# Delete default route (explicit)
ip route del default via 192.168.0.1 dev eth0 proto static metric 100

# Add a default route via a specific IP and interface
ip route add default via 192.168.0.1 dev eth0 proto static metric 90

# Add route for a network via gateway on an interface
ip route add 10.0.100.0/24 via 10.0.100.254 dev ens5

# Add default route met een specifieke metric
ip route add default via 10.0.180.1 dev ens7 proto static metric 90

netstat

# Display network connections and current states
netstat

# Check listening ports, connected remote IPs, processes, states and more
netstat -taupen

# Check listening ports and IPs of the local server
netstat -tulpn

# List the routing table
netstat -r

# List verbose common TCP and ICMP information
netstat -s

ss

Replacement for netstat

# Check open ports, connected IPs, processes, states and more
ss -taupen

tcpdump

# Listen on interface eth0 for traffic coming from host  172.16.0.11
tcpdump -i eth0 host 172.16.0.11

# Listen on interface eno2 for traffic coming from host  172.16.1.20, going to port 443
tcpdump -i en02 host 172.16.1.20 port 443

uuidgen

# Generate a unique UUID (for an interface)
uuidgen eth0
7bb91614-6ffe-4bdc-9b37-c6e9d37f6987

ip

# Show network information 
ip address
ip a

# Show all configured routes
ip r show

# Display statistics for all interfaces
ip -s link

# Display detailed statistics for all interfaces
ip -s -s link

# Execute the ifconfig command within a specific router
ip netns exec qrouter-asdwe49-as8d7-asd2-ert0-cvb7klj2 "ifconfig"

Package managers

apt

# Check for updates
apt update

# List packages that can be upgraded
apt list --upgradable

# Installed available updates
apt upgrade

# List installed packages
apt list --installed

# List package details and description
apt show net-tools

# Search inside all package descriptions for your keyword
apt-cache search ssh

rpm

# List all local RPM packages
rpm -qa

# Query for a specific installed rpm package
rpm -qi nginx

yum

#  Search for all available packages that include string "nginx"
yum search nginx

# Install the package named Nginx
yum install nginx

# List installed packages
yum list installed

dnf

#  Upgrade and install updates
dnf upgrade

# Remove the podman package
dnf remove podman

# Show information about the zlib package
dnf info zlib

# Show mandatory/optional/default packages within the Networking Tools group
dnf group info "Networking Tools"

Filesystem

fdisk

cfdisk is also nice

Checks

# Check your disks and partitions
fdisk -l

# Enter fdisk interactive mode
fdisk /dev/nvme0n2p1

# List available partition types
l

Configuration

# Format /dev/vdb as BTRFS
echo -e "n\np\n1\n\n\nt\n8E\np\nw" | fdisk /dev/vdb

Other

man + mandb

# Open the manual for the man tool
man man

# Open the manual for the ls tool
man ls

# 'Update' mandb by purging and or processing manuals
mandb

# Purge everything and regenerate manuals
mandb --create

ls

# List folders sorted by modified date
ls -trol

# List folder contents recursively
ls -alsR myfolder/

# List folder contents sorted by time, newest first and reverse order
ls -latr myfolder

# Print 9th column of folder contents
ll /mnt/btrfs/share1/ | awk '{print $9}'

grep

# Search for any occurences of "inet_interface" in a file
grep inet_interface /etc/postfix/main.cf

# Search for pattern "audit" in file /var/log/syslog
grep -e "audit" /var/log/syslog

# Search for text "started" in everything in /var/log/, and list the filename for each occurence
grep -H "started" /var/log/*

# Search for any mention of "md" within a file, by piping to grep
cat /var/log/messages | grep md

# Search for any of text "test" within the /etc folder recursively, also shows filename by default
grep -r "test" /etc

# Recursively search for any mention of "audit" in each file within the specified directory, display linenumber and ignore low/upper case
grep -rni audit /var/log/

lsof

# List what has files opened on the directory/mount
lsof /data/mount/lustre-01

# List processes listening on port 443
lsof -i :443

awk

# List the first column of the output generated by docker ps
docker ps | awk '{print $1}'

# Print 9th column of folder contents
ll /mnt/btrfs/share1/ | awk '{print $9}'

tar

# Compress the destination directory and keep the source path within the zipped file
tar -czvf name-of-archive.tar.gz /path/to/directory-or-file

# Compress the destination directory, but put the folder contents into the . within the zipped file
tar -czvf name-of-archive.tar.gz -C /path/to/directory-or-file .

# Extract a tar.gz file to the current folder
tar -xzvf name-of-archive.tar.gz

find

# Basic find command
find / -name name-to-search-for

# Search the current folder for all files
find . -name \* 

# Search the current folder for all files and count them
find . -name \* | wc -l

# Find all files with the SUID bit set
find / -name "*" -perm /u+s

# Find the current folder for files that were modified in the last 15 minutes
find . -mmin -15 -type f -name "*"

# Search for all modified files between 2023-01-01 and 2023-12-30
find /var/log/ -type f -name "*" -newermt 2023-01-01 ! -newermt 2023-12-30

# Search for all modified folders between 2022-01-01 and 2022-02-10, limited to a single folders' depth
find /data/research001/ -maxdepth 1 -type d -newermt 2022-01-01 ! -newermt 2022-02-10

# Search the current folder for all .log files and search & output any line containing string "error"
find . -name \*.log -exec grep -H error {} \;

# Screwing around
for i in $(find URL* -name "*.report" | sort); do echo "$i" && cat "$i" | grep TOTAL_SIZE ; done
for i in $(find URL* -name "*.report"); do basename "$i" | sort && cat "$i" | grep TOTAL_SIZE ; done
for i in $(find URL* -name "*.report"); do basename "$i" | sort && cat "$i" | grep TOTAL_SIZE | awk '{print $2}'; done
for i in $(find URL* -name "*.report"); do basename "$i" && cat "$i" | grep TOTAL_SIZE | awk '{print $2}'; done
for i in $(find * -name "*.report"); do basename "$i" && cat "$i" | grep TOTAL_SIZE | awk '{print $2}'; done
ll URL* | awk '{print $9}' | grep "*.report" | sort | for i in $(find * -name "*.report"); do basename "$i" && cat "$i" | grep TOTAL_SIZE | awk '{print $2}'; done
ll URL* | awk '{print $9}' | grep .report | sort | for i in $(find * -name "*.report"); do basename "$i" && cat "$i" | grep TOTAL_SIZE

find URL1 -name \*.report -exec grep -H TOTAL_SIZE {} \; | LC_ALL=C awk -M 'BEGIN{FS=OFS="\t"} {printf("%s\t%.02f\n", $1, $2/(1024*1024*1024))}' | sed -e 's~^.*/~~' -e 's~\..*SIZE~~' | sort

less

25     = Go to line 25
g      = Go to top of file
G      = Go to bottom of file
/      = Activate search mode
/Error = Search for "Error"
n      = Move to next search result
N      = Move to previous search result

# Don't wrap long lines to the current screen (move left or right to see non-truncated line)
less -S /var/log/syslog

# Output a file's contents and read it with less
cat /etc/snmpd/snmp.conf | less -S

# Number the lines when viewing
less -N /var/log/messages

# Open less at the first search result for "error". (Do not use space between the -p parameter and your search query)
less -p"Error" /var/log/messages

ssh

# Connect to a server using a specific user
ssh mirelurk@192.168.0.1

# Connect to a server using a specific RSA private key
ssh 192.168.0.1 -i /home/john/.ssh/id_rsa_key-5

# Connect to a server using a specific SSH port
ssh 192.168.0.1 -p 1111

# Show verbose information when connecting to a server
ssh -v 192.168.0.1

# Execute 'ls' on a remote server and output the result to your shell session
echo 'ls' | ssh -T user@10.0.20.75

# Execute a command on a remote server and output the result to a local file
echo 'ls' | ssh -T user@10.0.20.75 > <filename>.log

# Log in by providing a password in the CLI
sshpass 'MyPassword' ssh -XY root@10.100.25.1

# Copy a local file to another server
scp /home/root/myfiletocopy ubuntu@192.168.0.10:/home/ubuntu

vim

Esc   Switches between input/command mode

o     Create a new line below the current cursor position and switch to input mode
:wq   Save (write) and quit the file
:q!   Quit immediately without applying any changes

j     Move the cursor one line downwards

# Enter the Vim tutorial
vimtutor

rsync

Also see rclone for enterprise storage enviroments.

# Copy contents of source /mnt/science/data/ to target /home/garyon/backup/science/ recursively
rsync -a /mnt/science/data/ /home/garyon/backup/science/

# Copy  everything: symlinks, hardlinks, extended attributes, modified times, files, folders, etc
rsync -avHXS --progress /mnt/science/data/ /home/mayra/backup/science/

# Show progress during a transfer
rsync -avHXS --progress /mnt/science/data/ /home/stefanie/backup/science/

# rsync is additive by default
# After an initial rsync, delete files in the target that were deleted in the source
rsync --delete -avHXS /mnt/science/data/ /home/bob/backup/science/

# Sync using SSH
rsync -avrS --delete /data/cardio/ 192.168.0.15:/backup/cardio/

# Sync using a specific SSH port
rsync -avrS --rsh='ssh -p2020' --delete /data/science/ 192.168.0.20:/backup/science/

cron

https://crontab.guru/

# List cron jobs for the current user
crontab -l

# Modify cron jobs for the current user
crontab -eq

git

Checks

# List your current branch and situation
git status

# List all branches and your current one
git branch --all

# List all available tags
git tag

# List the current selected tag
git describe
git describe --tags

# Compare changes in the current working directory to the committed tree, and list what files have been changed
git diff-files

# Compare changes in the current working directory to the committed tree, and list what has changed
git diff-files -p

# Compare the committed tree to the current working directory, and list what has changed
git diff HEAD

# Create a folder and initialize it for use by git
mkdir gitrepo1; cd gitrepo1; git init

# Switch to another branch
git checkout stable/zed

# Switch to a specific tag
git checkout tags/14.11.0

# Fetch data from the current upstream branch
git pull

# Pull data from a specific branch
git pull origin unmaintained/yoga

rclone

https://rclone.org/

Configuration

Example configuration based on OpenStack swift. Config should be in the homefolder of your user .config/rclone/rclone.conf:

[swift-ssd]
type = swift
user = patrick
key = <PASSWORD>
auth = https://openstack.brammerloo.nl:5000/v3
region = Rotterdam
domain = Default
tenant = patrickproject

Commands

# List all containers, buckets and or folders of container "swift-ssd"
rclone lsd "swift-ssd:"
          20 2025-02-10 09:46:00         2 ssd-container
           0 2025-02-10 09:46:00         1 swift-ssd
           0 2025-02-10 09:46:00         1 swift-ssd2

rclone lsd "swift-ssd:ssd-container"
           0 2025-02-10 09:48:02        -1 mystorage

# List contents, files, folders of bucket "ssd-container", within container "swift-ssd"
rclone ls "swift-ssd:ssd-container"

# List the contents of file "asd"
rclone cat "swift-ssd:ssd-container/asd"

# Mount an object storage to local folder /mnt/object-ssd/
rclone mount swift-ssd:ssd-container /mnt/object-ssd

# Synchronize a local folder to a destination folder inside a bucket, in interactive mode
rclone sync -i /etc/rsyslog.d swift-ssd:ssd-container/mystorage/

mdtest

This chapter was written and contributed by Ivo Palli.

General

mdtest is part of the ior performance test package.

Installation

wget https://github.com/hpc/ior/releases/download/3.3.0/ior-3.3.0.tar.bz2
tar xjf ior-*.tar.bz2
cd ior-*/

yum install openmpi-devel environment-modules
# Relog your shell so 'module' is available
module load mpi
module list
./configure
make -j4
make install

Usage

Note: Number of items should be a multiple of depth x branching factor

module load mpi
mpirun --oversubscribe --allow-run-as-root -n 10 mdtest -n 2000 -z 5 -b 2 -d /mnt/nfs

Links

Linux:Tools

Commands

Quick access

Commands

Uncommon commands

Network

ping

traceroute

route

netstat

ss

tcpdump

uuidgen

ip

Package managers

apt

rpm

yum

dnf

Filesystem

fdisk

Checks

Configuration

Other

man + mandb

ls

grep

lsof

awk

tar

find

less

ssh

vim

rsync

cron

git

Checks

rclone

Configuration

Commands

mdtest

General

Installation

Usage

Links

Navigation menu

Search