Distributing Databases from The Main Server to The Gitlab Private Repository using Integromat.

Imam Ali Mustofa - Jun 15 '20 - - Dev Community

Experiments

In the first phase of the experiment, I want to create a Rest Server using ReactPHP and make the Gitlab Private Repository as data storage. However, I cannot write it down and explain it, because it is not finished yet.

On this occasion I would like to share something that I think can be useful for other developers.

Required:

After everything is prepared, what you need to pay attention to is the endpoints that will interact with Integromat and distribute the database to the destination location.

In this case I am using Codeigniter Framework to create my website and create a Rest API endpoint to interact with Integromat, let's just say this is Webhook (hmm... looks stupid). I will provide a snippet from the Controller that will interact with the Integromat and Library FlySystem as data distributors for Gitlab.

Web App + Rest API

Database Operation endpoint as a data distribution for gitlab that handles Backup Database and uploads Zip Files to the Gitlab Private Repository.

Here's the snippet and I save it in the application/controllers/webmaster/Database_operation.php


<?php

/**
 * Database Opertaion
 */
class Database_operation extends Betta_Controller
{

    public function __construct()
    {
        parent::__construct();
    }

    public function backup()
    {
        $status = false;
        try {
            //load helpers
            $this->load->helper('file');
            $this->load->library('zip');
            $this->load->library('flysystem');

            //load database
            $this->load->dbutil();

            //create format
            $db_format = array('format'=>'zip','filename'=>'backup.sql');
            $backup = $this->dbutil->backup($db_format);

            // file name
            $dbname='backup-on-'.date('Y-m-d H:i').'.zip';
            $save = FCPATH . 'db/' .$dbname;

            // write file
            write_file($save,$backup);
            file_put_contents( FCPATH . 'db/backup.log', $dbname);

            // Upload & Commit to Gitlab
            $this->flysystem->write( $dbname, $backup );
            $status = true;
        } catch (Exception $e) {
            $status = false;
        }

        $this->response([
            'status' => $status,
            'type' => ( $status ) ? 'success' : 'error',
            'msg' => ( $status ) ? 'Database berhasil di backup dengan nama: ' . $dbname . ' Pada tanggal: ' . date( 'Y-m-d H:i:s' ) : 'Gagal Backup Database!',
            'waktu' => date( 'Y-m-d H:i:s' ),
            'file' => $dbname,
            'sumber' => base_url(), 
            'destinasi' => 'gitlab'
        ], Betta_Controller::HTTP_OK, 'json');
    }

    public function remove_backup()
    {
        $status = false;
        $last_log = file( FCPATH . 'db/backup.log' );
        $last = end($last_log);

        // Hapus file backup
        if ( unlink(FCPATH . 'db/' . $last) ) {
            $status = true;
        } else {
            $status = false;
        }

        $this->response([
            'status' => $status,
            'type' => ( $status ) ? 'success' : 'error',
            'msg' => ( $status ) ? 'File backup berhasil dihapus' : 'Gagal Hapus File Backup Database!'
        ], Betta_Controller::HTTP_OK, 'json');
    }
}


Enter fullscreen mode Exit fullscreen mode

On the controller there is a FlySystem library that I integrate with the website application that I made. And following the library's appearance, I save the file in application/libraries/Flysystem.php

<?php

use League\Flysystem\Adapter\Local as LocalAdapter;
use League\Flysystem\Filesystem;
use RoyVoetman\FlysystemGitlab\Client;
use RoyVoetman\FlysystemGitlab\GitlabAdapter;
use TCB\Flysystem\Sync;

defined('BASEPATH') OR exit('No direct script access allowed');

/**
 * Flysystem
 */
class Flysystem {
    /**
     * @var mixed
     */
    private $filesystem;
    /**
     * @var mixed
     */
    private $ci;
    /**
     * @var mixed
     */
    private $client;
    /**
     * @var mixed
     */
    private $adapter;

    public function __construct() {
        $this->ci = &get_instance();
        $this->client     = new Client('GITLAB_PERSONAL_ACCESS_TOKEN', 'PROJECT_ID', 'PROJECT_BRANCH', 'GITLAB_URL');
        $this->adapter    = new GitlabAdapter($this->client);
        $this->filesystem = new Filesystem($this->adapter);
    }

    /**
     * Write File
     * @param  string $path     location of a file
     * @param  string $contents file contents
     * @return boolean true|false
     */
    public function write($path, $contents) {
        return $this->filesystem->write($path, $contents);
    }

    /**
     * Update content of file
     * @param  string $path     location of a file
     * @param  string $contents file contents
     * @return boolean           success boolean
     */
    public function update($path, $contents) {
        return $this->filesystem->update($path, $contents);
    }

    /**
     * Read file
     * @param  string $path location of file
     * @return string       file contents or false on failure
     */
    public function read($path) {
        $contents = $this->filesystem->read($path);

        return $contents;
    }

    /**
     * @param $path
     * @return mixed
     */
    public function has($path) {
        $contents = $this->filesystem->has($path);

        return $contents;
    }

    /**
     * @param $path
     * @return mixed
     */
    public function delete($path) {
        $contents = $this->filesystem->delete($path);

        return $contents;
    }

    /**
     * @param $from
     * @param $to
     * @return mixed
     */
    public function rename($from, $to) {
        $contents = $this->filesystem->rename($from, $to);

        return $contents;
    }

    /**
     * @param $from
     * @param $to
     * @return mixed
     */
    public function copy($from, $to) {
        $contents = $this->filesystem->copy($from, $to);

        return $contents;
    }

    /**
     * @param $path
     * @return mixed
     */
    public function deleteDir($path) {
        $contents = $this->filesystem->deleteDir($path);

        return $contents;
    }

    /**
     * @return mixed
     */
    public function action() {
        return $this->filesystem;
    }

    /**
     * Sync It helps you sync 2 directories at a time. There are two types.
     * @param  [type] $from [description]
     * @param  [type] $to   [description]
     * @return [type]       [description]
     */
    public function sync($from) {
        $remote = $this->filesystem;
        $local  = new Filesystem(new LocalAdapter($from));

        $sync = new Sync($local, $remote, '/');
        $sync->sync();

        return [
            'write_on_remote' => $sync->getUtil()->getWrites(),
            'delete_on_local' => $sync->getUtil()->getDeletes(),
            'update_on_both'  => $sync->getUtil()->getUpdates(),
        ];
    }
}

Enter fullscreen mode Exit fullscreen mode

And now it's time to explain the flow of the scenario on the Integromat that I have made as a cover above.

This time I assume that you have made the above preparations.

Integromat Scenario

HTTP Module #1

This HTTP Module is responsible for making requests to the main server and backing up with the endpoints according to the controller above: https://your_website/webmaster/database-opeation/backup which is only done in a certain period of time according to our settings.

Integromat Router #2

Integromat router will distribute the response to several subsequent modules as recipients of messages or reports.

Telegram Bot #3

Telegram Bot will receive a message from the response received on the first request and then send a message to the Telegram Channel consisting of clients. So the client will get the status of every backup activity carried out by the server.

HTTP Module # 4

Then between the Telegram Bot module and the HTTP Module there is a condition filter, if the Backup is successful the HTTP Module will delete the backup file stored on the main server via the endpoint https://your_website/webmaster/database-operation/remove-backup, because the backup file is successfully distributed to the Gitlab Private Repository.

Google Sheet Module # 5

After everything is done successfully then Integromat will record a report of every backup activity to Google Spreasheet.

Done!

Are you interested in trying it out?

Thanks to:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .