Ubuntu 20.x, 22.x PostgreSQL 15 Yedekli Yapı - Patroni, ETCD, HA - Patroni Yeni Node Ekleme

Doğukan Eren - Nov 6 '23 - - Dev Community

Bu yazımızda yedekli PostgreSQL 15 yapılandırmasını Patroni, Etcd ve HA Proxy kullanarak oluşturacağız.

1 - PostgreSQL Kurulumu 

sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'

sudo apt install gnupg2 ca-certificates -y

wget -O- https://www.postgresql.org/media/keys/ACCC4CF8.asc | gpg --dearmor > pgsql.gpg

sudo mv pgsql.gpg /etc/apt/trusted.gpg.d/pgsql.gpg

sudo apt update

sudo apt install postgresql-15
Enter fullscreen mode Exit fullscreen mode

2 - Patroni Kurulumu 

sudo apt -y install patroni
Enter fullscreen mode Exit fullscreen mode

2.1 Alternatif Olarak pip ile Kurulumu 

apt install python3-pip python3-dev libpq-dev -y
pip3 install --upgrade pip
pip install patroni
pip install python-etcd
pip install psycopg2
Enter fullscreen mode Exit fullscreen mode

PostgreSQL dosyalarına kolay erişebilmek için aşağıdaki gibi linkleyebiliriz.

2.2 Patroni için Link Oluşturalım 

ln -s /usr/lib/postgresql/15/bin/* /usr/sbin/
Enter fullscreen mode Exit fullscreen mode

3 - Patroni Konfigürasyonları

Patroni için verilerin (data ve wall dosyalarının) tutulacağı dizinleri oluşturup yetkilendirme işlemlerini gerçekleştirelim.

sudo mkdir -p /data/patroni
sudo chown -R postgres:postgres /data/
sudo chmod -R 700 /data/
Enter fullscreen mode Exit fullscreen mode

İki node yapsı için, /etc/patroni.yml dosyasını aşağıdaki gibi düzenleyebiliriz veya oluşturabiliriz.

scope: development
namespace: /db/
name: node-1

restapi:
    listen: <node-1-ip-adresi>:8008
    connect_address: <node-1-ip-adresi>:8008

etcd:
    host: <node-1-ip-adresi>:2379

bootstrap:
    dcs:
        ttl: 30
        loop_wait: 10
        retry_timeout: 10
        maximum_lag_on_failover: 1048576
        postgresql:
            use_pg_rewind: true

    initdb:
    - encoding: UTF8
    - data-checksums

    pg_hba:
    - host replication replicator 127.0.0.1/32 md5
    - host replication replicator <node-1-ip-adresi>/24 md5
    - host replication replicator <node-2-ip-adresi>/24 md5
    - host all all 0.0.0.0/0 md5

    users:
        admin:
            password: admin
            options:
                - createrole
                - createdb

postgresql:
    listen: <node-1-ip-adresi>:5432
    connect_address: <node-1-ip-adresi>:5432
    data_dir: /data/patroni
    pgpass: /tmp/pgpass
    authentication:
        replication:
            username: replicator
            password: "1"
        superuser:
            username: postgres
            password: "1"
    parameters:
        unix_socket_directories: '.'

tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false
Enter fullscreen mode Exit fullscreen mode

Patroni servisinin dosyasını (/etc/systemd/system/patroni.service) aşağıdaki gibi oluşturabilir veya düzenleyebiliriz.

[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL
After=syslog.target network.target

[Service]
Type=simple

User=postgres
Group=postgres

ExecStart=/usr/bin/patroni /etc/patroni.yml
KillMode=process
TimeoutSec=30
Restart=no

[Install]
WantedBy=multi-user.targ
Enter fullscreen mode Exit fullscreen mode

Sonrasında servisi sistem açıldığında çalışacak şekide başlatmak için aşağıdaki komutları çalıştırabiliriz.

systemctl daemon-reload
systemctl stop postgresql
systemctl restart patroni
systemctl status patroni
systemctl enable patroni
Enter fullscreen mode Exit fullscreen mode

NOT: Buraya kadar olan adımlar birinci ve ikinci patroni nodlarında yapılamlıdır. Bundan sonrası etcd ve HAProxy kurulacak makina üzerinde yapılması yeterlidir. Patroni 2. nodunda düzenleme yapılırken node1 yazan kısımlar kendisi olacak şekilde, node2 olacak kısımlar ise diğer makina olacak şekilde düşünülabilir.

4 - Etcd Kurulumu 

sudo apt install etcd
Enter fullscreen mode Exit fullscreen mode

Kurulum sonrası /etc/default/etcd dosyasını aiağıdaki gibi düzenlebiliriz.

ETCD_LISTEN_PEER_URLS="http://<etcd-ip-adresi>:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://<etcd-ip-adresi>:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://<etcd-ip-adresi>:2380"
ETCD_INITIAL_CLUSTER="default=http://<etcd-ip-adresi>:2380,"
ETCD_ADVERTISE_CLIENT_URLS="http://<etcd-ip-adresi>:2379"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
Enter fullscreen mode Exit fullscreen mode 
systemctl restart etcd
systemctl status etcd
systemctl enable etcd
Enter fullscreen mode Exit fullscreen mode

5 - HA Proxy Kurulumu 

sudo apt install haproxy
Enter fullscreen mode Exit fullscreen mode

HAProxy konfigürasyonlarının bulunduğu /etc/haproxy/haproxy.cfg dosyasını aşağıdaki gibi düzenliyoruz.

global
    maxconn 1000

defaults
    log global
    mode tcp
    retries 2
    timeout client 30m
    timeout connect 4s
    timeout server 30m
    timeout check 5s

listen stats
    mode http
    bind *:7000
    stats enable
    stats uri /

listen postgres
    bind *:5000
    option httpchk
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
    server psql-node-1 <node-1-ip-adresi>:5432 maxconn 100 check port 8008
    server psql-node-2 <node-2-ip-adresi>:5432 maxconn 100 check port 8008
Enter fullscreen mode Exit fullscreen mode 
systemctl restart haproxy
systemctl status haproxy
systemctl enable haproxy
Enter fullscreen mode Exit fullscreen mode

6 - Yeni Bir Node Eklemek

Ubuntu 20.X kurulu sunucuda 1,2,3,4 adımları gerçekleştiriyoruz.

6.1 Dördüncü adımda Yeni Makine konfigürasyonu

/etc/patroni.yml dosyasının içerisini aşağıdaki gibi dolduruyoruz

scope: development
namespace: /db/
name: node-yeni

restapi:
    listen: <node-yeni-ip-adresi>:8008
    connect_address: <node-yeni-ip-adresi>:8008

etcd:
    host: <etcd-ip-adresi>:2379

bootstrap:
    dcs:
        ttl: 30
        loop_wait: 10
        retry_timeout: 10
        maximum_lag_on_failover: 1048576
        postgresql:
            use_pg_rewind: true

    initdb:
    - encoding: UTF8
    - data-checksums

    pg_hba:
    - host replication replicator 127.0.0.1/32 md5
    - host replication replicator <node-yeni-ip-adresi>/0 md5
    - host replication replicator <node1-ip-adresi>/0 md5
    - host replication replicator <node2-ip-adresi>/0 md5
    - host all all 0.0.0.0/0 md5

    users:
        admin:
            password: admin
            options:
                - createrole
                - createdb

postgresql:
    listen: <node-yeni-ip-adresi>:5432
    connect_address: <node-yeni-ip-adresi>:5432
    data_dir: /data/patroni
    pgpass: /tmp/pgpass
    authentication:
        replication:
            username: replicator
            password: "1"
        superuser:
            username: postgres
            password: "1"
    parameters:
        unix_socket_directories: '.'

tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false
Enter fullscreen mode Exit fullscreen mode

6.2 Var olan Node Konfigürasyonları 

scope: development
namespace: /db/
name: node-3

restapi:
    listen: <node-1-ip-adresi>:8008
    connect_address: <node-1-ip-adresi>:8008

etcd:
    host: <etcd-ip-adresi>:2379

bootstrap:
    dcs:
        ttl: 30
        loop_wait: 10
        retry_timeout: 10
        maximum_lag_on_failover: 1048576
        postgresql:
            use_pg_rewind: true

    initdb:
    - encoding: UTF8
    - data-checksums

    pg_hba:
    - host replication replicator 127.0.0.1/32 md5
    - host replication replicator <node1-ip-adresi>/0 md5
    - host replication replicator <node2-ip-adresi>/0 md5
    - host replication replicator <node3-ip-adresi>/0 md5
    - host all all 0.0.0.0/0 md5

    users:
        admin:
            password: admin
            options:
                - createrole
                - createdb

postgresql:
    listen: <node-1-ip-adresi>:5432
    connect_address: <node-1-ip-adresi>:5432
    data_dir: /data/patroni
    pgpass: /tmp/pgpass
    authentication:
        replication:
            username: replicator
            password: 1
        superuser:
            username: postgres
            password: 1
    parameters:
        unix_socket_directories: '.'

tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false
Enter fullscreen mode Exit fullscreen mode

7.3 HA Proxy Konfigürasyonu 

global
    maxconn 1000

defaults
    log global
    mode tcp
    retries 2
    timeout client 30m
    timeout connect 4s
    timeout server 30m
    timeout check 5s

listen stats
    mode http
    bind *:7000
    stats enable
    stats uri /

listen postgres
    bind *:5000
    option httpchk
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
    server psql-node-1 <node-1-ip-adresi>:5432 maxconn 100 check port 8008
    server psql-node-2 <node-2-ip-adresi>:5432 maxconn 100 check port 8008
    server psql-node-3 <node-2-ip-adresi>:5432 maxconn 100 check port 8008
Enter fullscreen mode Exit fullscreen mode

8 - Patronictl Komutu İle Kontrol Edilmesi 

patronictl -c /etc/patroni.yml list
Enter fullscreen mode Exit fullscreen mode

Bu komutun çıktısında

root@ubuntu-server:/home/ubuntu# patronictl -c /etc/patroni.yml list
+ Cluster: postgreskume (7298293200548063780) -+----+-----------+
| Member    | Host         | Role    | State   | TL | Lag in MB |
+-----------+--------------+---------+---------+----+-----------+
| node-1 | <node-1-ip-adresi> | Leader  | running |  1 |           |
| node-2 | <node-2-ip-adresi> | Replica | running |  1 |         0 |
|
| node-3 | <node-3-ip-adresi> | Replica | running |  1 |         0 |
+-----------+--------------+---------+---------+----+-----------+
Enter fullscreen mode Exit fullscreen mode

Not: Yeni node eklerken Dördüncü adımdaki Patroni servis dosyasını oluşturmayı unutmayalım.

Image
Metode Main Judi Online dengan Optimis
Image
Implementacion de videollamadas multiplataforma iOS - Android

ios, mobile, jitsi, ionic
Image
I didn't know it was this exciting !

devchallenge, hacktoberfestchallenge, hacktoberfest, opensource
Image
I didn't know it was this exciting !

devchallenge, hacktoberfestchallenge, hacktoberfest, opensource
Image
The Reason Everyone Is Talking About Biggest Crypto Casino Right Now
Image
Happy? Halloween? 🤔

halloween, learning, discuss
Image
Isoleren met glaswol: tips en advies van Essent
Image
15 Surprising Facts About Mesothelioma Asbestos Attorney
Image
The 15 Things Your Boss Would Like You To Know You'd Known About Wheelchairs Self Propelled
Image
Untuk Pemula Main Judi Online ! Panduan Permainan Poker Judi Online Untuk Beberapa Pemula Pemain Judi Online!
Image
Meble, które odmieniają wnętrza: 10 inspirujących pomysłów na aranżację!
Image
Making a crypto wallet app with React Native

programming, reactnative, mobile, cryptocurrency
Image
Are You Getting The Most From Your Asbestos Exposure Lawyer?
Image
Halloween.dev 🎃 JavaScript Programming Challenges

halloween, javascript, webdev, beginners
Image
e
Image
Arahan Masuk Taruhan di Agen Judi Online Indonesia
Image
Unlocking the Mystery: Deciphering the Enigmatic Code of URL's Glob Patterns in cy.intercept()

cypress, qa, automation, testing
Image
15 Reasons Not To Be Ignoring Asbestos Cancer Claim
Image
Building an AI-Powered iOS Chat App with Amazon Bedrock and Swift

aws, serverless, ai, ios
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .