List of assigned IPs in subnet
EC2: List of assigned IPs in subnet
aws ec2 describe-network-interfaces
--filters "Name=subnet-id,Values=<subnet-id>"
--query 'NetworkInterfaces[*].PrivateIpAddress'
How to make EC2 user data script run again on startup?
How to make EC2 user data script run again on startup?
rm /var/lib/cloud/instances/*/sem/config_scripts_user
rm /var/lib/cloud/instance/sem/config_scripts_user
Find all images inside containers
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" |\
tr -s '[[:space:]]' '\n' |\
sort |\
uniq -c
Get Limit Requests for all containers
kubectl get pod --all-namespaces --sort-by='.metadata.name' -o json | \
jq -r '[.items[] | {pod_name: .metadata.name, containers: .spec.containers[] | \
[ {container_name: .name, memory_requested: .resources.requests.memory, cpu_requested: .resources.requests.cpu} ] }]' | \
jq 'sort_by(.containers[0].cpu_requested)'
K8S list all container's image from a cluster
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" |\
tr -s '[[:space:]]' '\n' |\
sort |\
uniq -c
kubectl check secrets certificates
kubectl get secret CERTNAME -o "jsonpath={.data['tls\.crt']}" | base64 -D | openssl x509 -enddate -noout
kubernetes volume protected
kubernetes volume protected
This happens when persistent volume is protected. You should be able to cross verify this:
Command:
kubectl describe pvc PVC_NAME | grep Finalizers
Output:
Finalizers: [kubernetes.io/pvc-protection]
You can fix this by setting finalizers to null using kubectl patch:
kubectl patch pvc PVC_NAME -p '{"metadata":{"finalizers": []}}' --type=merge
EKS CPUthrotting
docker run --cpus CPUS -it python python -m timeit -s 'import hashlib' -n 10000 -v 'haslib.sha512().update(b"foo")'
Export DB dump of a particular database to your machine
<div class="card">
<div class="card-body"><p>Export DB dump of a particular database to your machine</p>
$ kubectl exec {{podName}} -n {{namespace}} -- mysqldump -u {{dbUser}} -p{{password}} {{DatabaseName}} > <scriptName>.sql
# Example :
$ kubectl exec mysql-58 -n sql -- mysqldump -u root -proot USERS > dump.sql
</div>
Docker clean up
docker system prune --all --volumes --force
docker volume rm $(docker volume ls -qf dangling=true)
docker rm -vf $(docker ps -aq)
docker rmi -f $(docker images -aq)
docker volume prune -f
Stop all containers
docker stop $(docker ps -qa)
Remove containers
docker rm $(docker ps -qa)
Remove all images
docker images -q |xargs docker rmi
docker rmi -f $(docker images | grep "<none>" | awk "{print \$3}")
Docker network change default
systemctl stop docker
iptables -t nat -F POSTROUTING
ip link set dev docker0 down
ip addr del 172.17.0.1/16 dev docker0 # delete old route
ip addr add 172.18.32.1/24 dev docker0 # add new one
ip link set dev docker0 up
Permanent options : /etc/default/docker
DOCKER_OPTS : /etc/docker/daemon.json
--bip=172.18.32.1/24 --default-gateway=172.18.32.1
{
"default-address-pools": [
{"base":"172.80.0.0/16","size":24},
{"base":"172.90.0.0/16","size":24}
]
}
IP linux command
Understanding ip command OBJECTS syntax OBJECTS can be any one of the following and may be written in full or abbreviated form:
- Link: Network device (l)
- address: Protocol (IP or IPv6) address on a device (a)
- addrlabel: Label configuration for protocol address selection (addrl)
- neighbour: ARP or NDISC cache entry (neigh)
- route: Routing table entry (r)
- rule: Rule in routing policy database (ru)
- maddress: Multicast address (maddr)
- mroute: Multicast routing cache entry (mr)
- tunnel: Tunnel over IP (t)
- xfrm: Framework for IPsec protocol (x)
ip OBJECT COMMAND
ip [options] OBJECT COMMAND
ip OBJECT help
You can select between IPv4 and IPv6 using the following syntax:
# Only show TCP/IP IPv4
ip -4 a
# Only show TCP/IP IPv6
ip -6 a
It is also possible to specify and list particular interface TCP/IP details:
# Only show eth0 interface
ip a show eth0
ip a list eth0
ip a show dev eth0
# Only show running interfaces
ip link show up
IP route add network command examples
ip route add {NETWORK/MASK} via {GATEWAYIP}
ip route add {NETWORK/MASK} dev {DEVICE}
ip route add default {NETWORK/MASK} dev {DEVICE}
ip route add default {NETWORK/MASK} via {GATEWAYIP}
kubectl basic commands
# connect on Pods
kubectl exec -it --namespace=develop my-pod-id -- /bin/ash
# Show pod network on cluster
kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}'
# Show Pods by <node>
kubectl get pods --all-namespaces -o wide --field-selector spec.nodeName=<node>
# sorting pods
kubectl get pods -o wide --sort-by="{.spec.nodeName}"
kubernetes volume protected
This happens when persistent volume is protected. You should be able to cross verify this:
kubectl describe pvc PVC_NAME | grep Finalizers
Output:
Finalizers: [kubernetes.io/pvc-protection]
You can fix this by setting finalizers to null using kubectl patch:
kubectl patch pvc PVC_NAME -p '{"metadata":{"finalizers": []}}' --type=merge
EKS CPUthrotting
docker run --cpus CPUS -it python python -m timeit -s 'import hashlib' -n 10000 -v 'haslib.sha512().update(b"foo")'
Export DB dump of a particular database to your machine
$ kubectl exec {{podName}} -n {{namespace}} -- mysqldump -u {{dbUser}} -p{{password}} {{DatabaseName}} > <scriptName>.sql
# Example :
$ kubectl exec mysql-58 -n sql -- mysqldump -u root -proot USERS > dump.sql
MySQLdump with nohup
Connect PostgreSQL database
sh -c 'nohup mysqldump -h mydb.hostname -u admin --triggers --routines --events --compress --all-databases --pXXXXX > mydatabase.dmp &'
MySQL Size optimisation
Get table size order
SELECT
TABLE_NAME AS `Table`,
ROUND((DATA_LENGTH + INDEX_LENGTH) / 1024 / 1024) AS `Size (MB)`
FROM
information_schema.TABLES
WHERE
TABLE_SCHEMA = "bookstore"
ORDER BY
(DATA_LENGTH + INDEX_LENGTH)
DESC;
View optimized size by tables
SELECT table_name, data_free/1024/1024 AS data_free_MB, table_rows FROM information_schema.tables
WHERE engine LIKE 'InnoDB';
SELECT table_schema AS "Database",
ROUND(SUM(data_length + index_length) / 1024 / 1024, 2) AS "Size (MB)"
FROM information_schema.TABLES
GROUP BY table_schema;
SQL useful commmands
For schema:
SELECT default_character_set_name FROM information_schema.SCHEMATA
WHERE schema_name = "schemaname";
For tables:
SELECT CCSA.character_set_name FROM information_schema.`TABLES` T,
information_schema.`COLLATION_CHARACTER_SET_APPLICABILITY` CCSA
WHERE CCSA.collation_name = T.table_collation
AND T.table_schema = "schemaname"
AND T.table_name = "tablename";
For Columns:
SELECT character_set_name FROM information_schema.`COLUMNS`
WHERE table_schema = "schemaname"
AND table_name = "tablename"
AND column_name = "columnname";
MySQLdump with nohup
sh -c 'nohup mysqldump -h mydb.hostname -u admin --triggers --routines --events --compress --all-databases --pXXXXX > mydatabase.dmp &'
MySQL Grant
> create user 'root'@'%' IDENTIFIED BY 'password'
> grant all privileges on *.* to 'root'@'%' with grant option;
> flush privileges;
> show grants for current_user;
Add CA on Centos/Debian/Ubuntu
# Centos/RedHat
1. Copy the .crt file to /etc/pki/ca-trust/source/anchors on your CentOS machine
2. Run update-ca-trust extract
3. Check CA in list : cat /etc/pki/tls/certs/ca-bundle.trust.crt | grep SI2M
# Debian/Ubuntu
$ apt-get install -y ca-certificates
$ cp local-ca.crt /usr/local/share/ca-certificates
$ update-ca-certificates
# Get certificate from URL
openssl s_client -showcerts -verify 5 -connect stackexchange.com:443 < /dev/null
Extract SSL from url
openssl s_client -showcerts -verify 5 -connect wikipedia.org:443 < /dev/null |
awk '/BEGIN CERTIFICATE/,/END CERTIFICATE/{ if(/BEGIN CERTIFICATE/){a++}; out="cert"a".pem"; print >out}'
for cert in *.pem; do
newname=$(openssl x509 -noout -subject -in $cert \
| sed -nE 's/.*CN ?= ?(.*)/\1/; s/[ ,.*]/_/g; s/__/_/g; s/_-_/-/; s/^_//g;p' \
| tr '[:upper:]' '[:lower:]').pem
echo "${newname}"; mv "${cert}" "${newname}"
done
KEY and CRT validation
openssl pkey -in privateKey.key -pubout -outform pem | sha256sum
openssl x509 -in certificate.crt -pubkey -noout -outform pem | sha256sum
openssl req -in CSR.csr -pubkey -noout -outform pem | sha256sum
Tomcat SSL and Keystore
# merge all certificats in one file PEM (CRT)
cat wildcard.domain.com.crt geotrust_CA_intermediate.crt geotrust_CA.crt > all.crt
# convert CRT into P12 (PKCS12)
openssl pkcs12 -export -inkey wildcard.domain.com.key -in all.crt -name sub.domain.com -out sub.domain.com.p12
# import/export from P12
keytool -importkeystore -srckeystore sub.domain.com.p12 -srcstoretype pkcs12 -destkeystore sub.domain.com.jks
keytool -importkeystore -srckeystore sub.domain.com.jks -destkeystore sub.domain.com.jks -deststoretype pkcs12
## keytool -importkeystore -srckeystore sub.domain.com.p12 -srcstoretype pkcs12 -destkeystore sub.domain.com.jks
> Import du fichier de clés sub.domain.com.p12 vers sub.domain.com.jks...
Entrez le mot de passe du fichier de clés de destination :
Ressaisissez le nouveau mot de passe :
Entrez le mot de passe du fichier de clés source :
L'entrée de l'alias sub.domain.com a été importée.
Commande d'import exécutée : 1 entrées importées, échec ou annulation de 0 entrées
Warning:
Le fichier de clés JKS utilise un format propriétaire. Il est recommandé de migrer vers PKCS12,
qui est un format standard de l'industrie en utilisant :
"keytool -importkeystore -srckeystore sub.domain.com.jks -destkeystore sub.domain.com.jks -deststoretype pkcs12".
pgSQL useful commmands
Connect PostgreSQL database
psql -d database -U yser -W
psql -U user -h host "dbname=db sslmode=require"
Switch connection database
\c dbname username
\l : list all databases
\dx : liste des extensions installées
\dt : list all tables
\d table_name : describe a table
\dn : list available schema
\dv : list available functions
\dv : list available views
\du : list all users and their assign roles
\g : execute previous command
\s <filename> : command history (save into filename)
\i filename : execute command from filename
\timing : turn on / off execution time
\e : launch EDITOR defined in your ENV
\ef <function_name> : execute a function named from editor
\a : command switches from aligned to non-aligned column output.
\H : command formats the output to HTML format.
SELECT version();
Create user pgSQL
# See ROLES
psql -U postgres -c "SELECT * FROM pg_roles;"
postgres=# create database mydb;
postgres=# create user myuser with encrypted password 'mypass';
postgres=# grant all privileges on database mydb to myuser;
Check Database Size in PostgreSQL
postgres=# connect mydb;
postgres=# SELECT pg_size_pretty(pg_database_size('mydb'));
Check Tables Size in PostgreSQL
postgres=# connect mydb;
mydb=# SELECT pg_size_pretty( pg_total_relation_size('mytbl'));
PostgeSQL locks errors
SELECT
nom_base,
schema_objet_locke,
nom_objet_locke,
type_objet_locke,
duree_bloquage,
pid_session_bloquante,
user_session_bloquante,
client_session_bloquante,
derniere_requete_session_bloquante,
heure_debut_session_bloquante,
heure_debut_requete_bloquante,
pid_session_bloquee,
user_session_bloquee,
client_session_bloquee,
derniere_requete_session_bloquee,
heure_debut_requete_bloquee,
heure_debut_session_bloquee
FROM
(
SELECT distinct
RANK() OVER (PARTITION BY c.pid ORDER BY g.query_start DESC) as rang,
c.datname AS nom_base,
e.nspname AS schema_objet_locke,
d.relname AS nom_objet_locke,
CASE
WHEN d.relkind IN ('t','r') THEN 'table'
WHEN d.relkind = 'i' THEN 'index'
WHEN d.relkind = 's' THEN 'sequence'
WHEN d.relkind = 'v' THEN 'vue'
ELSE d.relkind::text
END AS type_objet_locke,
TO_CHAR(now()-c.query_start,'DD')||'j '||TO_CHAR(now()-c.query_start,'HH24:MI:SS') AS duree_bloquage,
g.pid AS pid_session_bloquante,
g.usename AS user_session_bloquante,
g.client_addr AS client_session_bloquante,
g.query AS derniere_requete_session_bloquante,
TO_CHAR(g.backend_start,'YYYYMMDD HH24:MI:SS') AS heure_debut_session_bloquante,
TO_CHAR(g.query_start,'YYYYMMDD HH24:MI:SS') AS heure_debut_requete_bloquante,
c.pid AS pid_session_bloquee,
c.usename AS user_session_bloquee,
c.client_addr AS client_session_bloquee,
c.query AS derniere_requete_session_bloquee,
TO_CHAR(c.query_start,'YYYYMMDD HH24:MI:SS') AS heure_debut_requete_bloquee,
TO_CHAR(c.backend_start,'YYYYMMDD HH24:MI:SS') AS heure_debut_session_bloquee
FROM
pg_locks AS a,
pg_locks AS b,
pg_stat_activity AS c,
pg_class AS d,
pg_namespace AS e,
pg_locks AS f,
pg_stat_activity AS g
WHERE a.pid = b.pid
AND a.pid = c.pid
AND b.relation = d.oid
AND d.relnamespace = e.oid
AND b.relation = f.relation
AND b.pid <> f.pid
AND f.pid = g.pid
AND c.query_start >= g.query_start
AND a.granted IS FALSE
AND b.relation::regclass IS NOT NULL
AND e.nspname NOT IN ('pg_catalog','pg_toast','information_schema')
AND e.nspname NOT LIKE 'pg_temp_%'
AND f.granted is true
) AS resultat
WHERE rang = 1
ORDER BY resultat.heure_debut_requete_bloquee,resultat.heure_debut_requete_bloquante ;
Backup/Restore for PostgreSQL
# Backup single database
pg_dump my_database > my_database.sql
# Restore single database
psql my_database < my_database.sql
# Backup All databases
pg_dumpall > alldb.sql
# Restore All databases
psql < alldb.sql
# Backup Compressed database
pg_dump mydb | gzip > mydb.sql.gz
# Restore compressed database
gunzip -c mydb.sql.gz | psql mydb
Hello World
A sample go program is show here.
package main
import "fmt"
func main() {
message := greetMe("world")
fmt.Println(message)
}
func greetMe(name string) string {
return "Hello, " + name + "!"
}
Run the program as below:
$ go run hello.go
Variables
Normal Declaration:
var msg string
msg = "Hello"
Shortcut:
msg := "Hello"
Constants
const Phi = 1.618