Array in Bash
string="0123456789" # create a string of 10 characters
array=(0 1 2 3 4 5 6 7 8 9) # create an indexed array of 10 elements
declare -A hash
hash=([one]=1 [two]=2 [three]=3) # create an associative array of 3 elements
echo "string length is: ${#string}" # length of string
echo "array length is: ${#array[@]}" # length of array using @ as the index
echo "array length is: ${#array[*]}" # length of array using * as the index
echo "hash length is: ${#hash[@]}" # length of array using @ as the index
echo "hash length is: ${#hash[*]}" # length of array using * as the index
Special characters in Bash
Special Variable Variable Details
- $1 to $n : $1 is the first arguments, $2 is second argument till $n n’th arguments.
From 10’th argument, you must need to inclose them in braces like ${10}, ${11} and so on
- $0 : The name of script itself
- $$ : Process id of current bash
- $* : Values of all the arguments. All agruments are double quoted
- $# : Total number of arguments passed to script
- $@ : Values of all the arguments
- $? : Exit status id of last command
- $! : Process id of last command
Linux rename username
The really right way? Say you want to change user ‘peter’ to ‘paul’.
groupadd paul usermod -d /home/paul -m -g paul -l paul peter
list of file used by PID
lsof -a -p <PID>
# ls - l /proc/<PID>/fd
ps -aeo pid,pcpu,args --sort=-%cpu | head
Linux SiG Signal list of file used by PID
SIGHUP -HUP gracefully reloads the workers and the application
SIGTERM -TERM "brutally" reloads
SIGINT -INT and SIGQUIT -QUIT kills all the workers immediately
SIGUSR1 -USR1 prints statistics (stdout)
SIGUSR2 -USR2 prints worker status
SIGURG -URG restores a snapshot
SIGTSTP -TSTP pauses, suspends or resumes an instance
SIGWINCH -WINCH wakes up a worker blocked in a syscall
Remove all package marked as rc
dpkg --list |grep "^rc" | cut -d " " -f 3 | xargs sudo dpkg --purge
how-to-loop-through-file-names-returned-by-find
Execute process once for each file
find . -name '*.txt' -exec process {} \;
If you have time, read through the rest to see several different ways and the problems with most of them.
The full answer:
The best way depends on what you want to do, but here are a few options. As long as no file or folder in the subtree has whitespace in its name, you can just loop over the files:
for i in $x; do # Not recommended, will break on whitespace
process "$i"
done
# Marginally better, cut out the temporary variable x:
for i in $(find -name \*.txt); do # Not recommended, will break on whitespace
process "$i"
done
#It is much better to glob when you can. White-space safe, for files in the current directory:
for i in *.txt; do # Whitespace-safe but not recursive.
process "$i"
done
# By enabling the globstar option, you can glob all matching files in this directory and all subdirectories:
# Make sure globstar is enabled
shopt -s globstar
for i in **/*.txt; do # Whitespace-safe and recursive
process "$i"
done
# IFS= makes sure it doesn't trim leading and trailing whitespace
# -r prevents interpretation of \ escapes.
while IFS= read -r line; do # Whitespace-safe EXCEPT newlines
process "$line"
done < filename
# read can be used safely in combination with find by setting the delimiter appropriately:
find . -name '*.txt' -print0 |
while IFS= read -r -d '' line; do
process "$line"
done
# For more complex searches, you will probably want to use find,
# either with its -exec option or with -print0 | xargs -0:
# execute `process` once for each file
find . -name \*.txt -exec process {} \;
# execute `process` once with all the files as arguments*:
find . -name \*.txt -exec process {} +
# using xargs*
find . -name \*.txt -print0 | xargs -0 process
# using xargs with arguments after each filename
# (implies one run per filename)
find . -name \*.txt -print0 | xargs -0 -I{} process {} argument
Rsync ssh script
#!/usr/bin/env bash
checkBinary() {
command -v "$1" >/dev/null 2>&1 || { echo >&2 "please install binary : $1. Aborting."; exit 1; }
}
checkBinary "rsync"
clear
STIME=$(date +"%X")
SOURCE_USER="appREMOTE"
SOURCE_HOST="app.mydomain.com"
SOURCE_DIR="/mnt/production/web/uploads/media/*"
DEST_DIR="/efs/media/"
rsync -avz \
--chown=1000:1000 \
-e "ssh -o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-i /home/ubuntu/.ssh/id_rsa" \
${SOURCE_USER}@"${SOURCE_HOST}":"${SOURCE_DIR}" \
"${DEST_DIR1}"
printf "### Initiated at : %s\n" ${STIME}
printf "### Terminated at : %s\n" $(date +"%X")
exit 0
Benchmark script with strace
Now let’s remove all the packages marked as rc.
strace -o trace -c -Ttt ./scrip
# where:
# -c is to trace the time spent by cpu on specific call.
# -Ttt will tell you time in microseconds at time of each system call running.
# -o will save output in file "trace".
Compile script and crypt
apt install shc
# to see content of encrypt script :
env bashOPTS=verbose ./test
mount RAID LVM disk recovery
- Boot avec un USB linux live
- Install mdadm
apt install mdadm
Build : RAID1 + LVM
mdadm --detail --scan
cat /proc/mdstat
mdadm -A -R /dev/md/8 /dev/sdc8
mdadm -S /dev/vg/lv
lvdisplay
mount /dev/lg/lv
HTTP load generator
Make ISO command line linux
# Install package
apt install genisoimage
mkisofs -allow-limited-size -l -J -r -iso-level 3 -o
# this command will generate bigger then 4GB ISO
Disabled SWAP on linux
swapon --show
NAME TYPE SIZE USED PRIO
/dev/sda2 partition 4G 0B -1
fallocate -l 1G /swapfile
dd if=/dev/zero of=/swapfile bs=1024 count=1048576
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
nano /etc/fstab
# Edit : /etc/fstab
/swapfile swap swap defaults 0 0
swapon --show
NAME TYPE SIZE USED PRIO
/swapfile file 1024M 507.4M -1
Get detailled disk partitions
findmnt -Do TARGET,SOURCE,USED,SIZE,OPTIONS
Umask Linux
Umask Created Files Created Directories
_______________________________________________
000 666 (rw_rw_rw_) 777 (rwxrwxrwx)
002 664 (rw_rw_r__) 775 (rwxrwxr_x)
022 644 (rw_r__r__) 755 (rwxr_xr_x)
027 640 (rw_r_____) 750 (rwxr_x___)
077 600 (rw_______) 700 (rwx______)
277 400 (r________) 500 (r_x______)
NFS Server show clients
# show clients connected
netstat | grep :nfs
# list all connected clients
showmount -a
# Discover/view NFS shares from the client
showmount -e
# list active share on client
showmount -e <ip_adress>
# Exports all file system paths specified in the /etc/exports file
exportfs -r
Disable Bell on linux
rmmod pcspkr
# Blacklisting the pcspkr module will prevent udev from loading it at boot:
echo "blacklist pcspkr" > /etc/modprobe.d/nobeep.conf
Delete full disk on Linux
shred --verbose --random-source=/dev/urandom -n1 /dev/sda1
Compiling a linux Kernel
# Download the source code and pgp signature from https://www.kernel.org/ to a directory of your choice
mkdir -p /usr/src/
cd /usr/src/
wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.1.16.tar.xz
wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.1.16.tar.sign
# Uncompress the source code and check the signature
xz -d -v linux-5.1.16.tar.xz
gpg --verify linux-5.1.16.tar.sign
# Untar the source code and cd into the directory
tar xf linux-5.1.16.tar
cd linux-5.1.16/
# Copy over actual kernel config file and run 'make menuconfig'
cp -v /boot/config-$(uname -r) .config
# Install necessary packages
apt-get install build-essential libncurses-dev bison flex libssl-dev libelf-dev bc
make menuconfig
# Go to Cryptographic API --> Certificates for signature checking -->
# and leave 'File name or PKCS#11 URI of module signing key' and
# 'Additional X.509 keys for default system keyring' blank if not
# already blank
# Compile using make or make -j n where n is the number of processors to use
make # or
make -j 4
# Install kernel modules
make modules_install
# Optimize and compile new kernel
cd /lib/modules/5.1.16/
find . -name *.ko -exec strip --strip-unneeded {} +
cd /usr/src/linux-5.1.16/
make install
Flush DNS MacOSx
sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder
Openlens block update
sudo chmod -R 000 ~/Library/Application\ Support/Caches/open-lens-updater/pending
Add CA on Centos/Debian/Ubuntu
# Centos/RedHat
1. Copy the .crt file to /etc/pki/ca-trust/source/anchors on your CentOS machine
2. Run update-ca-trust extract
3. Check CA in list : cat /etc/pki/tls/certs/ca-bundle.trust.crt | grep SI2M
# Debian/Ubuntu
$ apt-get install -y ca-certificates
$ cp local-ca.crt /usr/local/share/ca-certificates
$ update-ca-certificates
# Get certificate from URL
openssl s_client -showcerts -verify 5 -connect stackexchange.com:443 < /dev/null
Extract SSL from url
openssl s_client -showcerts -verify 5 -connect wikipedia.org:443 < /dev/null |
awk '/BEGIN CERTIFICATE/,/END CERTIFICATE/{ if(/BEGIN CERTIFICATE/){a++}; out="cert"a".pem"; print >out}'
for cert in *.pem; do
newname=$(openssl x509 -noout -subject -in $cert \
| sed -nE 's/.*CN ?= ?(.*)/\1/; s/[ ,.*]/_/g; s/__/_/g; s/_-_/-/; s/^_//g;p' \
| tr '[:upper:]' '[:lower:]').pem
echo "${newname}"; mv "${cert}" "${newname}"
done
KEY and CRT validation
openssl pkey -in privateKey.key -pubout -outform pem | sha256sum
openssl x509 -in certificate.crt -pubkey -noout -outform pem | sha256sum
openssl req -in CSR.csr -pubkey -noout -outform pem | sha256sum
Tomcat SSL and Keystore
# merge all certificats in one file PEM (CRT)
cat wildcard.domain.com.crt geotrust_CA_intermediate.crt geotrust_CA.crt > all.crt
# convert CRT into P12 (PKCS12)
openssl pkcs12 -export -inkey wildcard.domain.com.key -in all.crt -name sub.domain.com -out sub.domain.com.p12
# import/export from P12
keytool -importkeystore -srckeystore sub.domain.com.p12 -srcstoretype pkcs12 -destkeystore sub.domain.com.jks
keytool -importkeystore -srckeystore sub.domain.com.jks -destkeystore sub.domain.com.jks -deststoretype pkcs12
## keytool -importkeystore -srckeystore sub.domain.com.p12 -srcstoretype pkcs12 -destkeystore sub.domain.com.jks
> Import du fichier de clés sub.domain.com.p12 vers sub.domain.com.jks...
Entrez le mot de passe du fichier de clés de destination :
Ressaisissez le nouveau mot de passe :
Entrez le mot de passe du fichier de clés source :
L'entrée de l'alias sub.domain.com a été importée.
Commande d'import exécutée : 1 entrées importées, échec ou annulation de 0 entrées
Warning:
Le fichier de clés JKS utilise un format propriétaire. Il est recommandé de migrer vers PKCS12,
qui est un format standard de l'industrie en utilisant :
"keytool -importkeystore -srckeystore sub.domain.com.jks -destkeystore sub.domain.com.jks -deststoretype pkcs12".
Environments Variables
Variable Type Description
%ALLUSERSPROFILE%. Local Returns the location of the All Users Profile.
%APPDATA% Local Returns the location where applications store data by default.
%CD% Local Returns the current directory string.
%CMDCMDLINE% Local Returns the exact command line used to start the current Cmd.exe.
%CMDEXTVERSION% System Returns the version number of the current Command Processor Extensions.
%COMPUTERNAME% System Returns the name of the computer.
%COMSPEC% System Returns the exact path to the command shell executable.
%DATE% System Returns the current date. Uses the same format as the date /t command. Generated by Cmd.exe.
%ERRORLEVEL% System Returns the error code of the most recently used command. A non zero value usually indicates an error.
%HOMEDRIVE% System Returns which local workstation drive letter is connected to the user's home directory. Set based on the value of the home directory. The user's home directory is specified in Local Users and Groups.
%HOMEPATH% System Returns the full path of the user's home directory. Set based on the value of the home directory. The user's home directory is specified in Local Users and Groups.
%HOMESHARE% System Returns the network path to the user's shared home directory. Set based on the value of the home directory. The user's home directory is specified in Local Users and Groups.
%LOGONSEVER%. Local Returns the name of the domain controller that validated the current logon session.
%NUMBER_OF_PROCESSORS% System Specifies the number of processors installed on the computer.
%OS% System Returns the operating system name. Windows 2000 displays the operating system as Windows_NT.
%PATH% System Specifies the search path for executable files.
%PATHEXT% System Returns a list of the file extensions that the operating system considers to be executable.
%PROCESSOR_ARCHITECTURE% System Returns the chip architecture of the processor. Values: x86, IA64.
%PROCESSOR_IDENTFIER% System Returns a description of the processor.
%PROCESSOR_LEVEL% System Returns the model number of the processor installed on the computer.
%PROCESSOR_REVISION% System Returns the revision number of the processor.
%PROMPT% Local Returns the command prompt settings for the current interpreter. Generated by Cmd.exe.
%RANDOM% System Returns a random decimal number between 0 and 32767. Generated by Cmd.exe.
%SYSTEMDRIVE% System Returns the drive containing the Windows XP root directory (that is, the system root).
%SYSTEMROOT% System Returns the location of the Windows XP root directory.
%TEMP% and %TMP% System and User Returns the default temporary directories that are used by applications available to users who are currently logged on. Some applications require TEMP and others require TMP.
%TIME% System Returns the current time. Uses the same format as the time /t command. Generated by Cmd.exe.
%USERDOMAIN% Local Returns the name of the domain that contains the user's account.
%USERNAME% Local Returns the name of the user who is currently logged on.
%USERPROFILE%. Local Returns the location of the profile for the current user.
%WINDIR% System Returns the location of the operating system directory.
Reset Windows Update parameters
# Ouvrir un invité de commande en tant qu’administrateur
# Entrer chaque ligne, l’une après l’autre
net stop wuauserv
net stop cryptSvc
net stop bits
net stop msiserver
ren C:\Windows\SoftwareDistribution SoftwareDistribution.old
ren C:\Windows\System32\catroot2 catroot2.old
net start wuauserv
net start cryptSvc
net start bits
net start msiserver