Continuous Integration with Deno and Docker

Tomas Fernandez - Jul 15 '20 - - Dev Community

Deno has been getting a lot of attention. There are already more than 700 third-party modules and the number is quickly rising. People are starting to use it for real business applications. We even have a blockchain-based repository, something that blew my mind. But I haven't seen a lot of tutorials covering CI/CD and Docker with Deno, so I wrote one. I hope you find it handy, so be sure to bookmark it šŸ”–

Node has gained a lot of popularity since it was introduced in 2009. Despite its success, Ryan Dahl, Nodeā€™s original creator, believes thereā€™s room for improvement, and so he has recently released Deno, a new runtime for JavaScript and TypeScript, as its successor.

How is Deno different? Well, Deno, like Node, uses the V8 engine and event-driven architecture. But here is where the similarities end.

  • TypeScript gets first-class support at last. Deno compiles to JavaScript without additional packages.
  • Deno ships as a single executable with built-in, Go-inspired test runners and dependency management.
  • Deno has better security by default. Programs run in a sandbox that doesnā€™t have access to the network, the environment, or the filesystem unless explicitly granted.

The most significant difference, though, is that Deno doesnā€™t have a package manager (say goodbye to npm). That means Node.js modules are largely unsupported. Instead, Deno uses decentralized ES Modules. To compensate, Deno developers have introduced an audited standard library and support for third-party modules.

In this tutorial, weā€™ll learn how to use Semaphore Continuous Integration (CI) to test Deno applications. As a bonus, weā€™ll explore how to release Deno applications in Docker using Continuous Delivery (CD).

Prerequisites

If you wish to do this tutorial along with me, youā€™ll need the following:

To get started quickly, you can use our starter demo project.

GitHub logo TomFern / addressbook-deno

Deno example project in JavaScript.

Example HTTP API Server running on Deno.

This is a port for Deno of my addressbook Node.js demo.

Install and Run

  1. Fork and clone this repository.
  2. Set up environment.
$ cp env-example .env
$ source .env
Enter fullscreen mode Exit fullscreen mode
  1. Install/Update dependencies.
$ deno cache --reload src/deps.ts
Enter fullscreen mode Exit fullscreen mode
  1. Start a postgres database.
$ docker run -it -d -p 5432:5432 postgres
Enter fullscreen mode Exit fullscreen mode
  1. Create the tables.
$ deno run --allow-net --allow-env src/migrate.js
Enter fullscreen mode Exit fullscreen mode
  1. Run the application:
$ deno run --allow-net --allow-env src/app.js
Enter fullscreen mode Exit fullscreen mode

Testing

The project ships with some sample tests that take advantage of Denoā€™s built-in test runner.

Run the unit tests:

$ docker run -it -d -p 5432:5432 postgres
$ deno run --allow-net --allow-env src/migrate.js
$ deno test --allow-net --allow-env src/test/database.test.js
Enter fullscreen mode Exit fullscreen mode

Run the integration tests:

$ docker run -it -d -p 5432:5432 postgres
$ deno run --allow-net --allow-env src/migrate.js
$ deno run --allow-net --allow-env src/app.js &
$ deno test --allow-net --allow-env src/test/app.test.js
Enter fullscreen mode Exit fullscreen mode

Docker

The whole application canā€¦

View on GitHub

Feel free to fork it. It features an oak-based HTTP API service with a PostgreSQL backend and includes integration tests. The project comes with everything you need to build Docker images.

Otherwise, these instructions should work with any Deno application. You may need to make slight adjustments, though, depending on how your code is organized.

Prepare a Testing Image

Semaphoreā€™s composable containers feature lets us work with cutting-edge tools and languages like Deno. We can tailor Docker images to exact specifications and use them to drive CI/CD jobs seamlessly. Semaphore supports any container as long as it includes some basic packages like SSH, Git, and FTP.

Letā€™s take a few minutes to prepare a Dockerfile for Deno.

We can start from a Debian image:

FROM debian:buster
Enter fullscreen mode Exit fullscreen mode

Then, we tweak some settings and install the required packages:

RUN echo 'APT::Get::Assume-Yes "true";' > /etc/apt/apt.conf.d/99semaphore
RUN echo 'DPkg::Options "--force-confnew";' >> /etc/apt/apt.conf.d/99semaphore
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
        && apt-get install -y --no-install-recommends \
               ca-certificates sudo locales netbase netcat \
               procps lftp curl unzip git openssh-client \
        && rm -rf /var/cache/apt/archives
Enter fullscreen mode Exit fullscreen mode

Next, we set up the locale, so logs have the correct dates and format:

RUN ln -sf /usr/share/zoneinfo/Etc/UTC /etc/localtime
RUN locale-gen C.UTF-8 || true
ENV LANG=C.UTF-8
Enter fullscreen mode Exit fullscreen mode

And finally, install Deno. Weā€™ll the most current version at the time of writing this, v1.1.1:

RUN curl -fsSL https://deno.land/x/install/install.sh | sh -s v1.1.1
RUN cp /root/.deno/bin/deno /usr/local/bin && rm -rf /root/.deno
Enter fullscreen mode Exit fullscreen mode

The final result, which weā€™ll call Dockerfile.ci should look like this:

# Dockerfile.ci

FROM debian:buster

RUN echo 'APT::Get::Assume-Yes "true";' > /etc/apt/apt.conf.d/99semaphore
RUN echo 'DPkg::Options "--force-confnew";' >> /etc/apt/apt.conf.d/99semaphore
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
        && apt-get install -y --no-install-recommends \
               ca-certificates sudo locales netbase netcat \
               procps lftp curl unzip git openssh-client \
        && rm -rf /var/cache/apt/archives

RUN ln -sf /usr/share/zoneinfo/Etc/UTC /etc/localtime
RUN locale-gen C.UTF-8 || true
ENV LANG=C.UTF-8

RUN curl -fsSL https://deno.land/x/install/install.sh | sh -s v1.1.1
RUN cp /root/.deno/bin/deno /usr/local/bin && rm -rf /root/.deno

CMD ["/bin/sh"]
Enter fullscreen mode Exit fullscreen mode

Now all we have to do is build the image:

$ docker build -t YOUR_DOCKER_HUB_USER/deno:v1.1.1 -f Dockerfile.ci .
Enter fullscreen mode Exit fullscreen mode

Upload it to Docker Hub:

$ docker login -u YOUR_DOCKER_HUB_USER
$ docker push YOUR_DOCKER_HUB_USER/deno:v1.1.1
Enter fullscreen mode Exit fullscreen mode

And we are set to go.

Add Your Project to Semaphore

To add your project to Semaphore follow these steps:

  • Log in with your account and click on the + (plus sign) next to projects.

Add Project

  • Select your repository from the list.

Choose Repo

  • In the next screen, you can add more people to the project. Once done, click on Go to Workflow Builder to continue.

Invite People

  • Choose the Single Job starter workflow and click on Customize it first.

Choose Starter

You are now at the Workflow Builder, which lets you visually set up the CI/CD pipeline.

  1. The main element in the view is the pipeline. A pipeline consists of a series of blocks which are executed from left to right.
  2. Each block has one or more jobs. Once all jobs in a block complete, the next block starts.
  3. Jobs contain the commands that do the work and run in parallel. If any command fails, the pipeline stops and is marked as failed.

Pipeline Overview
Letā€™s create our first job:

  • Click the pipeline to view its settings. On the right side, under Environment Type select Docker containers.

Using Containers

  • Type the name of the image you uploaded in the previous section: YOUR_DOCKER_HUB_USER/deno:v1.1.1.

Set Container Image

  • Click on the first block in the pipeline to begin editing it.

Block 1

In this block, we only need to download and cache the projectā€™s dependencies. For this, we can combine Semaphoreā€™s cache tool with deno cache:

  1. cache restore takes a list of keys and retrieves the first match. Our project lists all dependencies in src/deps.ts, so we can use it as part of the key: 
cache restore deps-$(checksum src/deps.ts),deps-master
Enter fullscreen mode Exit fullscreen mode
  1. deno cache downloads dependencies without executing any code. To download them in the current directory: 
export DENO_DIR=$PWD/deps
deno cache src/deps.ts
Enter fullscreen mode Exit fullscreen mode
  1. cache store takes a list of keys and a directory and saves it for future runs: 
cache store deps-$(checksum src/deps.ts),deps-master deps
Enter fullscreen mode Exit fullscreen mode

Put together, the commands for the job are:

checkout
export DENO_DIR=$PWD/deps
cache restore deps-$(checksum src/deps.ts),deps-master
deno cache src/deps.ts
cache store deps-$(checksum src/deps.ts),deps-master deps
Enter fullscreen mode Exit fullscreen mode

Download Dependencies

Testing with Continuous Integration

In this section, weā€™ll create a new block with two test jobs. The tests use a PostgreSQL database. The easiest way to get one is to connect a new container since weā€™re already using them in the pipeline.

  • Select the pipeline and click on +Add Container

Add Database Container

  • Call the new container ā€œpostgresā€
  • Type the name of a postgres image on Image: postgres:12
  • Click on +Add environment variable and add the POSTGRES_HOST_AUTH_METHOD variable with value trust to allow connections without a password.

Set Postgres Image

  • Create a new block using +Add Block.

Add Block

  • Open the Prologue section. The commands we put here will be executed before every job in the block. Weā€™ll use these commands to retrieve the dependencies: 
checkout
export DENO_DIR=$PWD/deps
cache restore deps-$(checksum src/deps.ts),deps-maste
Enter fullscreen mode Exit fullscreen mode

Set Prologue

  • Open the Environment Variables section and create the variable DB_HOST = postgres

Set Environment Variables

Create three test jobs:

  • The first job does Database tests. Type the following commands: 
deno run --allow-net --allow-env src/migrate.js
deno test --allow-net --allow-env src/tests/database.test.js
Enter fullscreen mode Exit fullscreen mode
  • Click on +Add another job.
  • The second job does Integration tests. Type the following commands. Note that in this one, we also need to start the application before running the tests. 
deno run --allow-net --allow-env src/app.js &
deno run --allow-net --allow-env src/migrate.js
deno test --allow-net --allow-env src/tests/app.test.js
Enter fullscreen mode Exit fullscreen mode
  • The third job does Static tests. Deno ships with a code linter. To enable it, we need to use the --unstable flag. 
deno lint --unstable src/*.js src/*.ts src/tests/*.js
Enter fullscreen mode Exit fullscreen mode

Test Block

You can try the pipeline with Run the Workflow > Start.

Run Workflow

Tests Done

Run Deno with Docker

Docker lets us create portable images that can run anywhere. In this section, weā€™ll learn how to prepare a production image Deno.

Before Semaphore can push to your Docker registry, you must create a secret with the login details.

To do it:

  • Click on Secrets under Configuration on the left menu.

Go to Secrets

  • Click Create New Secret.
  • Create two variables for your Docker Hub username and password:
    • DOCKER_USERNAME = YOUR DOCKER USERNAME
    • DOCKER_PASSWORD = YOU DOCKER PASSWORD
  • Click on Save changes

Docker Hub Secret

Open the Workflow Builder again and scroll right to the end of the pipeline.

Edit Workflow

Weā€™ll add a second pipeline using a promotion. Promotions lets us tie multiple pipelines together with user-defined conditions.

  • Click on + Add First Promotion.

Add Promotion

  • Check the Enable automatic promotion option. You can set start conditions here.

Promotion Conditions

  • Click on the first block on the new pipeline.
  • Rename the job to ā€œDocker Buildā€.
  • On the right side of the screen, scroll down to the Secrets section and enable the dockerhub secret.

Import Secret on Block 1

The Docker build job consists of four commands:

  • Log in to the Docker registry. 
echo "${DOCKER_PASSWORD}" | docker login -u "${DOCKER_USERNAME}" --password-stdin
Enter fullscreen mode Exit fullscreen mode
  • Pull the latest image available. 
docker pull "${DOCKER_USERNAME}"/addressbook-deno:latest || true
Enter fullscreen mode Exit fullscreen mode
  • Build the new version of the image, reusing layers from previous versions when possible. 
docker build -t "${DOCKER_USERNAME}"/addressbook-deno:latest --cache-from "${DOCKER_USERNAME}"/addressbook-deno:latest .
Enter fullscreen mode Exit fullscreen mode
  • Push the new image version. 
docker push "${DOCKER_USERNAME}"/addressbook-deno:latest
Enter fullscreen mode Exit fullscreen mode

Weā€™ll use this Dockerfile to build the production image:

FROM debian:buster
RUN apt-get update \
        && apt-get install -y --no-install-recommends ca-certificates curl unzip netcat \
        && rm -rf /var/cache/apt/archives
RUN groupadd --gid 1000 deno \
  && useradd --uid 1000 --gid deno --shell /bin/bash --create-home deno
USER deno
RUN curl -fsSL -k https://deno.land/x/install/install.sh | sh -s v1.1.1
ENV HOME "/home/deno"
ENV DENO_INSTALL "${HOME}/.deno"
ENV PATH "${DENO_INSTALL}/bin:${PATH}"
RUN mkdir -p $HOME/app/src
COPY --chown=deno:deno src/ $HOME/app/src
WORKDIR $HOME/app/src
EXPOSE 4000
RUN deno cache deps.ts
CMD deno run --allow-env --allow-net app.js
Enter fullscreen mode Exit fullscreen mode

Compared with the image we used in the CI pipeline, this production image is leaner, has all the code and dependencies baked in, and runs as the deno user instead of root.

The final Docker build job should look like this:

checkout
echo "${DOCKER_PASSWORD}" | docker login -u "${DOCKER_USERNAME}" --password-stdin
docker pull "${DOCKER_USERNAME}"/addressbook-deno:latest || true
docker build -t "${DOCKER_USERNAME}"/addressbook-deno:latest --cache-from "${DOCKER_USERNAME}"/addressbook-deno:latest .
docker push "${DOCKER_USERNAME}"/addressbook-deno:latest
Enter fullscreen mode Exit fullscreen mode

Docker Build Job

Weā€™re done configuring the pipeline. Start it one last time.

Once it completes, click on promote to start the build.

Click Promote

Thatā€™s all! From now on, Semaphore will run the pipelines on every push to GitHub.

All Done

Whatā€™s Next?

You have prepared a production image with your application, now itā€™s time to think about deploying it šŸš€ ā€” donā€™t leave it sitting on Docker Hub, collecting dust.

Do you want to run your Deno application in Kubernetes but donā€™t know how to begin? Download our free CI/CD with Docker and Kubernetes ebook, which explains everything you need to know to get started and includes detailed how-to tutorials.

After that, check out my Docker & Kubernetes guides:

Semaphore

Docker and Java Spring Boot [Part.1: Continuous Integration]

Tomas Fernandez for Semaphore 惻 Jan 22 '20

#java #docker #devops #microservices
Semaphore

Google Cloud Kubernetes in 10 Minutes

Tomas Fernandez for Semaphore 惻 Nov 14 '19

#kubernetes #devops #productivity #tutorial
Semaphore

Continuous Integration and Delivery to AWS Kubernetes

Tomas Fernandez for Semaphore 惻 Oct 11 '19

#kubernetes #aws #devops #tutorial
Semaphore

Speeding up Docker Builds x7 (benchmark)

Tomas Fernandez for Semaphore 惻 Dec 19 '19

#docker #productivity #benchmark #devops

Interested in JavaScript or TypeScript? Check out these posts in the Semaphore blog:

Thanks for reading! Leave a comment if you liked it šŸ‘‹

Image
Five People You Should Know In The Lpg Gas Engineer Industry
Image
On ā€œsuperiorityā€ of (functional) programming and other labels

functional
Image
di mana dapat main slots online memberi keuntungan?
Image
DAO Generator

java, dao, database, persistence
Image
ė”¤ķ† ķ† ģ˜ ė§¤ė „: ģŠ¹ė¦¬ģ˜ ė°©ģ •ģ‹ģ„ ģ°¾ģ•„ģ„œ
Image
spek penting di website slots online bermutu
Image
Five Tools Everybody Involved In In Wall Electric Fire Place Industry Should Be Using
Image
10 Websites To Help You Become An Expert In Crypto Coin Casino
Image
Watch Out: How Key Fob Repair Near Me Is Taking Over And What You Can Do About It
Image
Aluminum Dome Roofs: Why They're Ideal for Storage Solutions
Image
Getting Started with the Minds JavaScript SDK
Image
Managing Google Workspace Part 1: Core Services, Additional Services and Marketplace Apps

googleworkspace, google, certification, notes
Image
All you need to know about Redux
Image
Comprehensive Guide to Biogas Holders: Applications, Benefits, and Features
Image
Jak stworzyć idealne mieszkanie: Przewodnik po stylu i funkcjonalności
Image
Bifold Door Repairs Near Me: The Secret Life Of Bifold Door Repairs Near Me
Image
A Brief History Of Non Asbestos Causes Of Mesothelioma History Of Non Asbestos Causes Of Mesothelioma
Image
10 Tips For Bi Fold Door Repair That Are Unexpected
Image
langkah mengoptimalkan pemasukan judi slots online
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .