Linux:Tools

From Cheatsheet
Jump to navigationJump to search


Commands

Quick access

# Scroll through a file with less
less -s myfile.txt

# Select line 5 from the output
cat example.txt | sel -e '5'

# Select lines from the output, starting from the top
cat example.txt | head -5

# Select lines from the output, starting from the bottom
cat example.txt | tail -5

Commands

# Display the full path of a file(assuming the syslog file is available in the current folder)
readlink -f syslog

# Unzip a file
gunzip /var/log/messages.2.gz

Uncommon commands

# CrASHing THIs SERVer, WiTH no SurVIvORS!
echo c > /proc/sysrq-trigger

Network

ping

Troubleshooting MTU: https://access.redhat.com/solutions/2440411 

# Ping with an interval of 5 seconds
ping -i 5 192.168.0.1

# Ping 192.168.10.5 using a specific interface
ping -I bond0 192.168.10.5

# Ping 8.8.8.8 for 20 times
ping -c 20 8.8.8.8

# Ping google.com using IPv4
ping -4 google.com

# Ping google.com using IPv6
ping -6 google.com

# Ping using packets of size 264
ping -s 264 1.1.1.1

# Test an MTU-size of 9000 by sending non-fragmented packages of size 8972 (28 bytes left for the headers)
ping -M do -s 8972 192.168.77.88

traceroute

Package mtr (My traceroute) is also very good 

# Show the traversed hops towards google.com using IPv4
traceroute -4 google.com

# Show the traversed hops towards google.com using IPv6
traceroute -6 google.com

# Does the same as "traceroute -6 google.com"
traceroute6 google.com

# Use ICMP for checking hops
traceroute -4 -I brammerloo.nl

route

# List configured routes
route

# List routes but display IPs instead of hostnames
route -n

# Delete default route 
ip route del 0.0.0.0/0 via 192.168.10.1 dev ens3

# Delete default route (explicit)
ip route del default via 192.168.0.1 dev eth0 proto static metric 100

# Add a default route via a specific IP and interface
ip route add default via 192.168.0.1 dev eth0 proto static metric 90

# Add route for a network via gateway on an interface
ip route add 10.0.100.0/24 via 10.0.100.254 dev ens5

# Add default route met een specifieke metric
ip route add default via 10.0.180.1 dev ens7 proto static metric 90

netstat

# Display network connections and current states
netstat

# Check listening ports, connected remote IPs, processes, states and more
netstat -taupen

# Check listening ports and IPs of the local server
netstat -tulpn

# List the routing table
netstat -r

# List verbose common TCP and ICMP information
netstat -s

ss

Replacement for netstat 

# Check open ports, connected IPs, processes, states and more
ss -taupen

tcpdump

# Listen on interface eth0 for traffic coming from host  172.16.0.11
tcpdump -i eth0 host 172.16.0.11

# Listen on interface eno2 for traffic coming from host  172.16.1.20, going to port 443
tcpdump -i en02 host 172.16.1.20 port 443

uuidgen

# Generate a unique UUID (for an interface)
uuidgen eth0
7bb91614-6ffe-4bdc-9b37-c6e9d37f6987

ip

# Show network information 
ip address
ip a

# Show all configured routes
ip r show

# Display statistics for all interfaces
ip -s link

# Display detailed statistics for all interfaces
ip -s -s link

# Execute the ifconfig command within a specific router
ip netns exec qrouter-asdwe49-as8d7-asd2-ert0-cvb7klj2 "ifconfig"

DNS | dig & nslookup

# Lookup reverse DNS host information
dig -x 10.0.2.15

# Lookup the nameservers of google.com, by asking nameserver 1.1.1.1
dig google.com  @1.1.1.1 NS

# Lookup reverse DNS host information
host 10.0.2.15

# Lookup DNS host information
nslookup 10.0.2.15

# Lookup host information for google.com while using DNS-server 8.8.8.8
nslookup google.com 8.8.8.8

Package managers

apt

# Check for updates
apt update

# List packages that can be upgraded
apt list --upgradable

# Installed available updates
apt upgrade

# List installed packages
apt list --installed

# List package details and description
apt show net-tools

# Search inside all package descriptions for your keyword
apt-cache search ssh

rpm

# List all local RPM packages
rpm -qa

# Query for a specific installed rpm package
rpm -qi nginx

yum

#  Search for all available packages that include string "nginx"
yum search nginx

# Install the package named Nginx
yum install nginx

# List installed packages
yum list installed

dnf

#  Upgrade and install updates
dnf upgrade

# Remove the podman package
dnf remove podman

# Show information about the zlib package
dnf info zlib

# Show mandatory/optional/default packages within the Networking Tools group
dnf group info "Networking Tools"

Filesystem

fdisk

cfdisk is also nice

Checks

# Check your disks and partitions
fdisk -l

# Enter fdisk interactive mode
fdisk /dev/nvme0n2p1

# List available partition types
l

Configuration

# Format /dev/vdb as BTRFS
echo -e "n\np\n1\n\n\nt\n8E\np\nw" | fdisk /dev/vdb

Other

man + mandb

# Open the manual for the man tool
man man

# Open the manual for the ls tool
man ls

# 'Update' mandb by purging and or processing manuals
mandb

# Purge everything and regenerate manuals
mandb --create

ls

# List folders sorted by modified date
ls -trol

# List folder contents recursively
ls -alsR myfolder/

# List folder contents sorted by time, newest first and reverse order
ls -latr myfolder

grep

# Search for any occurences of "inet_interface" in a file
grep inet_interface /etc/postfix/main.cf

# Search for pattern "audit" in file /var/log/syslog
grep -e "audit" /var/log/syslog

# Search for text "started" in everything in /var/log/, and list the filename for each occurence
grep -H "started" /var/log/*

# Search for any mention of "md" within a file, by piping to grep
cat /var/log/messages | grep md

# Search for any of text "test" within the /etc folder recursively, also shows filename by default
grep -r "test" /etc

# Recursively search for any mention of "audit" in each file within the specified directory, display linenumber and ignore low/upper case
grep -rni audit /var/log/

lsof

# List what has files opened on the directory/mount
lsof /data/mount/lustre-01

# List processes listening on port 443
lsof -i :443

awk

# List the first column of the output generated by docker ps
docker ps | awk '{print $1}'

# Print 9th column of folder contents
ll /mnt/btrfs/share1/ | awk '{print $9}'

tar

# Compress the destination directory and keep the source path within the zipped file
tar -czvf name-of-archive.tar.gz /path/to/directory-or-file

# Compress the destination directory, but put the folder contents into the . within the zipped file
tar -czvf name-of-archive.tar.gz -C /path/to/directory-or-file .

# Extract a tar.gz file to the current folder
tar -xzvf name-of-archive.tar.gz

find

# Basic find command
find / -name name-to-search-for

# Search the current folder for all files
find . -name \* 

# Search the current folder for all files and count them
find . -name \* | wc -l

# Find all files with the SUID bit set
find / -name "*" -perm /u+s

# Find the current folder for files that were modified in the last 15 minutes
find . -mmin -15 -type f -name "*"

# Search for all modified files between 2023-01-01 and 2023-12-30
find /var/log/ -type f -name "*" -newermt 2023-01-01 ! -newermt 2023-12-30

# Search for all modified folders between 2022-01-01 and 2022-02-10, limited to a single folders' depth
find /data/research001/ -maxdepth 1 -type d -newermt 2022-01-01 ! -newermt 2022-02-10

# Search the current folder for all .log files and search & output any line containing string "error"
find . -name \*.log -exec grep -H error {} \;

# Screwing around
for i in $(find URL* -name "*.report" | sort); do echo "$i" && cat "$i" | grep TOTAL_SIZE ; done
for i in $(find URL* -name "*.report"); do basename "$i" | sort && cat "$i" | grep TOTAL_SIZE ; done
for i in $(find URL* -name "*.report"); do basename "$i" | sort && cat "$i" | grep TOTAL_SIZE | awk '{print $2}'; done
for i in $(find URL* -name "*.report"); do basename "$i" && cat "$i" | grep TOTAL_SIZE | awk '{print $2}'; done
for i in $(find * -name "*.report"); do basename "$i" && cat "$i" | grep TOTAL_SIZE | awk '{print $2}'; done
ll URL* | awk '{print $9}' | grep "*.report" | sort | for i in $(find * -name "*.report"); do basename "$i" && cat "$i" | grep TOTAL_SIZE | awk '{print $2}'; done
ll URL* | awk '{print $9}' | grep .report | sort | for i in $(find * -name "*.report"); do basename "$i" && cat "$i" | grep TOTAL_SIZE

find URL1 -name \*.report -exec grep -H TOTAL_SIZE {} \; | LC_ALL=C awk -M 'BEGIN{FS=OFS="\t"} {printf("%s\t%.02f\n", $1, $2/(1024*1024*1024))}' | sed -e 's~^.*/~~' -e 's~\..*SIZE~~' | sort

less

25     = Go to line 25
g      = Go to top of file
G      = Go to bottom of file
/      = Activate search mode
/Error = Search for "Error"
n      = Move to next search result
N      = Move to previous search result

# Don't wrap long lines to the current screen (move left or right to see non-truncated line)
less -S /var/log/syslog

# Output a file's contents and read it with less
cat /etc/snmpd/snmp.conf | less -S

# Number the lines when viewing
less -N /var/log/messages

# Open less at the first search result for "error". (Do not use space between the -p parameter and your search query)
less -p"Error" /var/log/messages

ssh

# Stolen from https://www.openssh.com/legacy.html
ssh -Q cipher       # List supported ciphers
ssh -Q mac          # List supported MACs
ssh -Q key          # List supported public key types
ssh -Q kex          # List supported key exchange algorithms
# Connect to a server using a specific user
ssh mirelurk@192.168.0.1

# Connect to a server using a specific RSA private key
ssh 192.168.0.1 -i /home/john/.ssh/id_rsa_key-5

# Connect to a server using a specific SSH port
ssh 192.168.0.1 -p 1111

# Show verbose information when connecting to a server
ssh -v 192.168.0.1

# Connect using an ancient algorithm and keytype
ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 -c aes128-cbc  admin@10.50.10.50

# Execute 'ls' on a remote server and output the result to your shell session
echo 'ls' | ssh -T user@10.0.20.75

# Execute a command on a remote server and output the result to a local file
echo 'ls' | ssh -T user@10.0.20.75 > <filename>.log

# Log in by providing a password in the CLI
sshpass 'MyPassword' ssh -XY root@10.100.25.1

# Copy a local file to another server
scp /home/root/myfiletocopy ubuntu@192.168.0.10:/home/ubuntu

vim

Esc   Switches between input/command mode

o     Create a new line below the current cursor position and switch to input mode
:wq   Save (write) and quit the file
:q!   Quit immediately without applying any changes

j     Move the cursor one line downwards

# Enter the Vim tutorial
vimtutor

rsync

Also see rclone for enterprise storage enviroments. 

# Copy contents of source /mnt/science/data/ to target /home/garyon/backup/science/ recursively
rsync -a /mnt/science/data/ /home/garyon/backup/science/

# Copy  everything: symlinks, hardlinks, extended attributes, modified times, files, folders, etc
rsync -avHXS --progress /mnt/science/data/ /home/mayra/backup/science/

# Show progress during a transfer
rsync -avHXS --progress /mnt/science/data/ /home/stefanie/backup/science/

# rsync is additive by default
# After an initial rsync, delete files in the target that were deleted in the source
rsync --delete -avHXS /mnt/science/data/ /home/bob/backup/science/

# Sync using SSH
rsync -avrS --delete /data/cardio/ 192.168.0.15:/backup/cardio/

# Sync using a specific SSH port
rsync -avrS --rsh='ssh -p2020' --delete /data/science/ 192.168.0.20:/backup/science/

cron

Run tasks at specific intervals. 

# List cron jobs for the current user
crontab -l

# Modify cron jobs for the current user
crontab -eq

# Run  the "ls" command every 5 minutes
*/5 * * * * ps aux

screen

Create virtual sessions on the server you're connected to. 

# List all current sessions
screen -list

# Create new session "mynewsession"
screen -S mynewsession

# Detach current session
CTRL + A + D

# Attach session "mynewssion"
screen -r mynewsession

ldapsearch

DC = Domain Component
The values that identify the domain in which the object is located, may contain subdomains too i.e. "DC=customer,DC=amazon,DC=com"

OU =  Organization Unit
A container/folder in which objects or users are stored. Actively used in Microsoft Active Directory's i.e. "OU=janitors,DC=customer,DC=amazon,DC=com"

CN = Canonical Name
The name of the group you're searching for or in i.e. "CN=fullprivilege,OU=janitors,DC=customer,DC=amazon,DC=com"

UID = User Identifier
The unique identifier to find a user with, usually the username i.e. "uid=magicmike,CN=fullprivilege,OU=janitors,DC=customer,DC=amazon,DC=com"

DN = Distinguished Name
The entire path to an object, consisting of a combination of above values, at least the DCs and a CN or UID, i.e. "uid=magicmike,CN=fullprivilege,OU=janitors,DC=customer,DC=amazon,DC=com"


The following assumes domain "brammerloo.nl", based on usage for FreeIPA 

# Search and show attributes for user tonberry in group users in group accounts in domain brammerloo.nl, using the admin user to authenticatie
ldapsearch -W -b "uid=tonberry,cn=users,cn=accounts,dc=brammerloo,dc=nl"  -D "uid=admin,cn=users,cn=accounts,dc=brammerloo,dc=nl"

# Do the same as above, but specify LDAP-server ipa01.brammerloo.nl to send the query to
ldapsearch -W -H ldap://ipa01.brammerloo.nl -b "uid=tonberry,cn=users,cn=accounts,dc=brammerloo,dc=nl"  -D "uid=admin,cn=users,cn=accounts,dc=brammerloo,dc=nl"

# Do the same as above, but specify a specific port
ldapsearch -W -H ldap://ipa01.brammerloo.nl:389 -b "uid=tonberry,cn=users,cn=accounts,dc=brammerloo,dc=nl"  -D "uid=admin,cn=users,cn=accounts,dc=brammerloo,dc=nl"

# Use the "elastic" user to query for attributes of the "elastic-users" group which itself is a member of the "groups" group
ldapsearch -W -b "cn=elastic-users,cn=groups,cn=accounts,dc=brammerloo,dc=nl"  -D "uid=elastic,cn=users,cn=accounts,dc=brammerloo,dc=nl"

# Do the same as above, but specify you only want the member attribute
ldapsearch -W -b "cn=elastic-users,cn=groups,cn=accounts,dc=brammerloo,dc=nl" -D "uid=admin,cn=users,cn=accounts,dc=brammerloo,dc=nl" member

# Show all groups of which tonberry is a member of by searching for the memberOf attribute
ldapsearch -W -b "uid=tonberry,cn=users,cn=accounts,dc=brammerloo,dc=nl"  -D "uid=admin,cn=users,cn=accounts,dc=brammerloo,dc=nl" memberOf

# List attributes for all groups in the group "groups"
ldapsearch -W -b "cn=groups,cn=accounts,dc=brammerloo,dc=nl" -D "uid=admin,cn=users,cn=accounts,dc=brammerloo,dc=nl"

git

Checks

# List your current branch and situation
git status

# List all branches and your current one
git branch --all

# List all available tags
git tag

# List the current selected tag
git describe
git describe --tags

# Compare changes in the current working directory to the committed tree, and list what files have been changed
git diff-files

# Compare changes in the current working directory to the committed tree, and list what has changed
git diff-files -p

# Compare the committed tree to the current working directory, and list what has changed
git diff HEAD

Common

# Create a folder and initialize it for use by git
mkdir gitrepo1; cd gitrepo1; git init

# Switch to another branch
git checkout stable/zed

# Switch to a specific tag
git checkout tags/14.11.0

# Fetch data from the current upstream branch
git pull

# Pull data from a specific branch
git pull origin unmaintained/yoga

rclone

Installation

# Install the latest version from the website
curl https://rclone.org/install.sh | sudo bash

Configuration

Example configuration based on OpenStack swift. Config should be in the homefolder of your user .config/rclone/rclone.conf: 

[swift-ssd]
type = swift
user = patrick
key = <PASSWORD>
auth = https://openstack.brammerloo.nl:5000/v3
region = Rotterdam
domain = Default
tenant = patrickproject

Commands

# List all containers, buckets and or folders of container "swift-ssd"
rclone lsd "swift-ssd:"
          20 2025-02-10 09:46:00         2 ssd-container
           0 2025-02-10 09:46:00         1 swift-ssd
           0 2025-02-10 09:46:00         1 swift-ssd2

rclone lsd "swift-ssd:ssd-container"
           0 2025-02-10 09:48:02        -1 mystorage

# List contents, files, folders of bucket "ssd-container", within container "swift-ssd"
rclone ls "swift-ssd:ssd-container"

# List the contents of file "asd"
rclone cat "swift-ssd:ssd-container/asd"

# Mount an object storage to local folder /mnt/object-ssd/
rclone mount swift-ssd:ssd-container /mnt/object-ssd

# Synchronize a local folder to a destination folder inside a bucket, in interactive mode
rclone sync -i /etc/rsyslog.d swift-ssd:ssd-container/mystorage/

mdtest

This chapter was mostly written and contributed by Ivo Palli.

General

mdtest is part of the ior performance test package.

RHEL Installation

wget https://github.com/hpc/ior/releases/download/3.3.0/ior-3.3.0.tar.bz2
tar xjf ior-*.tar.bz2
cd ior-*/

yum install openmpi-devel environment-modules
# Relog your shell so 'module' is available
module load mpi
module list
./configure
make -j4
make install

Ubuntu Installation

VERSION=4.0.0
wget https://github.com/hpc/ior/releases/download/$VERSION/ior-$VERSION.tar.gz
tar -xzvf ior-$VERSION.tar.gz
cd ior-$VERSION/

apt install libopenmpi-dev environment-modules openmpi-bin openmpi-common libgtk2.0-dev -y
./configure

make -j4
make install
Usage

Note: Number of items should be a multiple of depth x branching factor 

module load mpi

# Run command "mdtest -n 2000 -z 5 -b 2 -d /mnt/ssd/" 10 times in a row
mpirun --oversubscribe --allow-run-as-root -n 10 mdtest -n 2000 -z 5 -b 2 -d /mnt/nfs
Links
Retrieved from "https://wiki.brammerloo.nl/index.php?title=Linux:Tools&oldid=838"