List of assigned IPs in subnet

EC2: List of assigned IPs in subnet

aws ec2 describe-network-interfaces
    --filters "Name=subnet-id,Values=<subnet-id>"
    --query 'NetworkInterfaces[*].PrivateIpAddress'
How to make EC2 user data script run again on startup?

How to make EC2 user data script run again on startup?

rm /var/lib/cloud/instances/*/sem/config_scripts_user
rm /var/lib/cloud/instance/sem/config_scripts_user
Find all images inside containers
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" |\
tr -s '[[:space:]]' '\n' |\
sort |\
uniq -c
Get Limit Requests for all containers
kubectl get pod --all-namespaces --sort-by='.metadata.name' -o json | \
jq -r '[.items[] | {pod_name: .metadata.name, containers: .spec.containers[] | \
[ {container_name: .name, memory_requested: .resources.requests.memory, cpu_requested: .resources.requests.cpu} ] }]' | \
jq  'sort_by(.containers[0].cpu_requested)'
K8S list all container's image from a cluster
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" |\
tr -s '[[:space:]]' '\n' |\
sort |\
uniq -c
kubectl check secrets certificates
kubectl get secret CERTNAME -o "jsonpath={.data['tls\.crt']}" | base64 -D | openssl x509 -enddate -noout
kubernetes volume protected

kubernetes volume protected

This happens when persistent volume is protected. You should be able to cross verify this:

Command:

kubectl describe pvc PVC_NAME | grep Finalizers

Output:
Finalizers:    [kubernetes.io/pvc-protection]

You can fix this by setting finalizers to null using kubectl patch:

kubectl patch pvc PVC_NAME -p '{"metadata":{"finalizers": []}}' --type=merge
EKS CPUthrotting
docker run --cpus CPUS -it python python -m timeit -s 'import hashlib' -n 10000 -v 'haslib.sha512().update(b"foo")'
Export DB dump of a particular database to your machine
        <div class="card">
            <div class="card-body"><p>Export DB dump of a particular database to your machine</p>
$ kubectl exec {{podName}} -n {{namespace}} -- mysqldump -u {{dbUser}} -p{{password}} {{DatabaseName}} > <scriptName>.sql

# Example :
$ kubectl exec mysql-58 -n sql -- mysqldump -u root -proot USERS > dump.sql
</div>
Docker clean up
docker system prune --all --volumes --force
docker volume rm $(docker volume ls -qf dangling=true)
docker rm -vf $(docker ps -aq)
docker rmi -f $(docker images -aq)
docker volume prune -f
Stop all containers
docker stop $(docker ps -qa)
Remove containers
docker rm $(docker ps -qa)
Remove all images
docker images -q  |xargs docker rmi
docker rmi -f $(docker images | grep "<none>" | awk "{print \$3}")
Docker network change default
systemctl stop docker
iptables -t nat -F POSTROUTING
ip link set dev docker0 down
ip addr del 172.17.0.1/16 dev docker0 # delete old route
ip addr add 172.18.32.1/24 dev docker0 # add new one
ip link set dev docker0 up

Permanent options : /etc/default/docker

DOCKER_OPTS : /etc/docker/daemon.json

--bip=172.18.32.1/24 --default-gateway=172.18.32.1

{
 "default-address-pools": [
   {"base":"172.80.0.0/16","size":24},
   {"base":"172.90.0.0/16","size":24}
  ]
}
IP linux command

Understanding ip command OBJECTS syntax OBJECTS can be any one of the following and may be written in full or abbreviated form:

  • Link: Network device (l)
  • address: Protocol (IP or IPv6) address on a device (a)
  • addrlabel: Label configuration for protocol address selection (addrl)
  • neighbour: ARP or NDISC cache entry (neigh)
  • route: Routing table entry (r)
  • rule: Rule in routing policy database (ru)
  • maddress: Multicast address (maddr)
  • mroute: Multicast routing cache entry (mr)
  • tunnel: Tunnel over IP (t)
  • xfrm: Framework for IPsec protocol (x)
ip OBJECT COMMAND
ip [options] OBJECT COMMAND
ip OBJECT help

You can select between IPv4 and IPv6 using the following syntax:

# Only show TCP/IP IPv4
ip -4 a

# Only show TCP/IP IPv6
ip -6 a

It is also possible to specify and list particular interface TCP/IP details:

# Only show eth0 interface
ip a show eth0
ip a list eth0
ip a show dev eth0

# Only show running interfaces
ip link show up

IP route add network command examples

ip route add {NETWORK/MASK} via {GATEWAYIP}
ip route add {NETWORK/MASK} dev {DEVICE}
ip route add default {NETWORK/MASK} dev {DEVICE}
ip route add default {NETWORK/MASK} via {GATEWAYIP}
kubectl basic commands
# connect on Pods
kubectl exec -it --namespace=develop my-pod-id -- /bin/ash
# Show pod network on cluster
kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}'
# Show Pods by <node>
kubectl get pods --all-namespaces -o wide --field-selector spec.nodeName=<node>
# sorting pods
kubectl get pods -o wide --sort-by="{.spec.nodeName}"
kubernetes volume protected

This happens when persistent volume is protected. You should be able to cross verify this:

kubectl describe pvc PVC_NAME | grep Finalizers
Output:
Finalizers:    [kubernetes.io/pvc-protection]

You can fix this by setting finalizers to null using kubectl patch:

kubectl patch pvc PVC_NAME -p '{"metadata":{"finalizers": []}}' --type=merge
EKS CPUthrotting
docker run --cpus CPUS -it python python -m timeit -s 'import hashlib' -n 10000 -v 'haslib.sha512().update(b"foo")'
Export DB dump of a particular database to your machine
$ kubectl exec {{podName}} -n {{namespace}} -- mysqldump -u {{dbUser}} -p{{password}} {{DatabaseName}} > <scriptName>.sql

# Example :
$ kubectl exec mysql-58 -n sql -- mysqldump -u root -proot USERS > dump.sql
MySQLdump with nohup

Connect PostgreSQL database

sh -c 'nohup mysqldump -h mydb.hostname -u admin --triggers --routines --events --compress --all-databases --pXXXXX  > mydatabase.dmp &'
MySQL Size optimisation

Get table size order

SELECT
  TABLE_NAME AS `Table`,
  ROUND((DATA_LENGTH + INDEX_LENGTH) / 1024 / 1024) AS `Size (MB)`
FROM
  information_schema.TABLES
WHERE
  TABLE_SCHEMA = "bookstore"
ORDER BY
  (DATA_LENGTH + INDEX_LENGTH)
DESC;

View optimized size by tables

SELECT table_name, data_free/1024/1024 AS data_free_MB, table_rows FROM information_schema.tables
  WHERE engine LIKE 'InnoDB';

SELECT table_schema AS "Database",
ROUND(SUM(data_length + index_length) / 1024 / 1024, 2) AS "Size (MB)"
FROM information_schema.TABLES
GROUP BY table_schema;
SQL useful commmands

For schema:

SELECT default_character_set_name FROM information_schema.SCHEMATA
WHERE schema_name = "schemaname";

For tables:

SELECT CCSA.character_set_name FROM information_schema.`TABLES` T,
       information_schema.`COLLATION_CHARACTER_SET_APPLICABILITY` CCSA
WHERE CCSA.collation_name = T.table_collation
  AND T.table_schema = "schemaname"
  AND T.table_name = "tablename";

For Columns:

SELECT character_set_name FROM information_schema.`COLUMNS`
WHERE table_schema = "schemaname"
  AND table_name = "tablename"
  AND column_name = "columnname";
MySQLdump with nohup
sh -c 'nohup mysqldump -h mydb.hostname -u admin --triggers --routines --events --compress --all-databases --pXXXXX  > mydatabase.dmp &'
MySQL Grant
> create user 'root'@'%' IDENTIFIED BY 'password'
> grant all privileges on *.* to 'root'@'%' with grant option;
> flush privileges;
> show grants for current_user;
Add CA on Centos/Debian/Ubuntu
# Centos/RedHat
1. Copy the .crt file to /etc/pki/ca-trust/source/anchors on your CentOS machine
2. Run update-ca-trust extract
3. Check CA in list : cat /etc/pki/tls/certs/ca-bundle.trust.crt | grep SI2M

# Debian/Ubuntu
$ apt-get install -y ca-certificates
$ cp local-ca.crt /usr/local/share/ca-certificates
$ update-ca-certificates

# Get certificate from URL
openssl s_client -showcerts -verify 5 -connect stackexchange.com:443 < /dev/null
Extract SSL from url
openssl s_client -showcerts -verify 5 -connect wikipedia.org:443 < /dev/null |
   awk '/BEGIN CERTIFICATE/,/END CERTIFICATE/{ if(/BEGIN CERTIFICATE/){a++}; out="cert"a".pem"; print >out}'
for cert in *.pem; do
    newname=$(openssl x509 -noout -subject -in $cert \
      | sed -nE 's/.*CN ?= ?(.*)/\1/; s/[ ,.*]/_/g; s/__/_/g; s/_-_/-/; s/^_//g;p' \
      | tr '[:upper:]' '[:lower:]').pem
    echo "${newname}"; mv "${cert}" "${newname}"
done
KEY and CRT validation
openssl pkey -in privateKey.key -pubout -outform pem | sha256sum
openssl x509 -in certificate.crt -pubkey -noout -outform pem | sha256sum
openssl req -in CSR.csr -pubkey -noout -outform pem | sha256sum
Tomcat SSL and Keystore
# merge all certificats in one file PEM (CRT)
cat wildcard.domain.com.crt geotrust_CA_intermediate.crt geotrust_CA.crt > all.crt

# convert CRT into P12 (PKCS12)
openssl pkcs12 -export -inkey wildcard.domain.com.key -in all.crt -name sub.domain.com -out sub.domain.com.p12

# import/export from P12
keytool -importkeystore -srckeystore sub.domain.com.p12 -srcstoretype pkcs12 -destkeystore sub.domain.com.jks

keytool -importkeystore -srckeystore sub.domain.com.jks -destkeystore sub.domain.com.jks -deststoretype pkcs12

## keytool -importkeystore -srckeystore sub.domain.com.p12 -srcstoretype pkcs12 -destkeystore sub.domain.com.jks
> Import du fichier de clés sub.domain.com.p12 vers sub.domain.com.jks...
Entrez le mot de passe du fichier de clés de destination :
Ressaisissez le nouveau mot de passe :
Entrez le mot de passe du fichier de clés source :
L'entrée de l'alias sub.domain.com a été importée.
Commande d'import exécutée : 1 entrées importées, échec ou annulation de 0 entrées

Warning:
Le fichier de clés JKS utilise un format propriétaire. Il est recommandé de migrer vers PKCS12,
qui est un format standard de l'industrie en utilisant :
"keytool -importkeystore -srckeystore sub.domain.com.jks -destkeystore sub.domain.com.jks -deststoretype pkcs12".
pgSQL useful commmands

Connect PostgreSQL database

psql -d database -U yser -W
psql -U user -h host "dbname=db sslmode=require"

Switch connection database

\c dbname username
\l : list all databases
\dx : liste des extensions installées
\dt : list all tables
\d table_name : describe a table
\dn : list available schema
\dv : list available functions
\dv : list available views
\du : list all users and their assign roles

\g : execute previous command
\s <filename> : command history (save into filename)
\i filename : execute command from filename

\timing : turn on / off execution time
\e : launch EDITOR defined in your ENV
\ef <function_name> : execute a function named from editor

\a : command switches from aligned to non-aligned column output.
\H : command formats the output to HTML format.

SELECT version();
Create user pgSQL
# See ROLES
psql -U postgres -c "SELECT * FROM pg_roles;"
postgres=# create database mydb;
postgres=# create user myuser with encrypted password 'mypass';
postgres=# grant all privileges on database mydb to myuser;
Check Database Size in PostgreSQL
postgres=# connect mydb;
postgres=# SELECT pg_size_pretty(pg_database_size('mydb'));
Check Tables Size in PostgreSQL
postgres=# connect mydb;
mydb=# SELECT pg_size_pretty( pg_total_relation_size('mytbl'));
PostgeSQL locks errors
SELECT
    nom_base,
    schema_objet_locke,
    nom_objet_locke,
    type_objet_locke,
    duree_bloquage,
    pid_session_bloquante,
    user_session_bloquante,
    client_session_bloquante,
    derniere_requete_session_bloquante,
    heure_debut_session_bloquante,
    heure_debut_requete_bloquante,
    pid_session_bloquee,
    user_session_bloquee,
    client_session_bloquee,
    derniere_requete_session_bloquee,
    heure_debut_requete_bloquee,
    heure_debut_session_bloquee
FROM
(
SELECT distinct
    RANK() OVER (PARTITION BY c.pid ORDER BY g.query_start DESC) as rang,
    c.datname AS nom_base,
    e.nspname AS schema_objet_locke,
    d.relname AS nom_objet_locke,
CASE
    WHEN d.relkind IN ('t','r') THEN 'table'
    WHEN d.relkind = 'i' THEN 'index'
    WHEN d.relkind = 's' THEN 'sequence'
    WHEN d.relkind = 'v' THEN 'vue'
ELSE d.relkind::text
END AS type_objet_locke,
    TO_CHAR(now()-c.query_start,'DD')||'j '||TO_CHAR(now()-c.query_start,'HH24:MI:SS') AS duree_bloquage,
    g.pid AS pid_session_bloquante,
    g.usename AS user_session_bloquante,
    g.client_addr AS client_session_bloquante,
    g.query AS derniere_requete_session_bloquante,
    TO_CHAR(g.backend_start,'YYYYMMDD HH24:MI:SS') AS heure_debut_session_bloquante,
    TO_CHAR(g.query_start,'YYYYMMDD HH24:MI:SS') AS heure_debut_requete_bloquante,
    c.pid AS pid_session_bloquee,
    c.usename AS user_session_bloquee,
    c.client_addr AS client_session_bloquee,
    c.query AS derniere_requete_session_bloquee,
    TO_CHAR(c.query_start,'YYYYMMDD HH24:MI:SS') AS heure_debut_requete_bloquee,
    TO_CHAR(c.backend_start,'YYYYMMDD HH24:MI:SS') AS heure_debut_session_bloquee
FROM
    pg_locks AS a,
    pg_locks AS b,
    pg_stat_activity AS c,
    pg_class AS d,
    pg_namespace AS e,
    pg_locks AS f,
    pg_stat_activity AS g
WHERE a.pid = b.pid
    AND a.pid = c.pid
    AND b.relation = d.oid
    AND d.relnamespace = e.oid
    AND b.relation = f.relation
    AND b.pid <> f.pid
    AND f.pid = g.pid
    AND c.query_start >= g.query_start
    AND a.granted IS FALSE
    AND b.relation::regclass IS NOT NULL
    AND e.nspname NOT IN ('pg_catalog','pg_toast','information_schema')
    AND e.nspname NOT LIKE 'pg_temp_%'
    AND f.granted is true
) AS resultat
WHERE rang = 1
ORDER BY resultat.heure_debut_requete_bloquee,resultat.heure_debut_requete_bloquante ;
Backup/Restore for PostgreSQL
# Backup single database
pg_dump my_database > my_database.sql
# Restore single database
psql my_database < my_database.sql

# Backup All databases
pg_dumpall > alldb.sql
# Restore All databases
psql < alldb.sql

# Backup Compressed database
pg_dump mydb | gzip > mydb.sql.gz
# Restore compressed database
gunzip -c mydb.sql.gz | psql mydb
Hello World

A sample go program is show here.

package main

import "fmt"

func main() {
  message := greetMe("world")
  fmt.Println(message)
}

func greetMe(name string) string {
  return "Hello, " + name + "!"
}

Run the program as below:

$ go run hello.go
Variables

Normal Declaration:

var msg string
msg = "Hello"

Shortcut:

msg := "Hello"
Constants
const Phi = 1.618