Creating a RESTful API with Node, Express.js, HarperDB, and Docker, and testing it with Hoppscotch

Pramit Marattha - Dec 22 '21 - - Dev Community

What is an API?

API the acronym actually stands for Application Programming Interface. It simply provides the interface for the different applications and different parts of the software to communicate with each other safely and in a secure manner. In brief, API is the messenger that takes requests and tells the system what the users want to do and returns back the response. APIs are standardized. Meaning, there is an industry-defined standard for defining the API and there are a few formats that are quite popular like SOAP, GraphQL and REST.

The Origins of  REST.

Prior to the year 2000, there was no consensus on how to create or use an API. Its integration necessitated the use of protocols such as SOAP, which were infamously difficult to create, operate, and debug.

This completely changed in the year 2000 when Roy T. Fielding named and characterized the Web's architectural style in his Ph.D. dissertation. Fielding called his description of the Web's architectural style "Representational State Transfer" (REST).

What are REST APIs?

Web services are purpose-built web servers that cater to the demands of a website or any other application. Client programs communicate with web services using application programming interfaces. API exposes a set of data and functions that allow computer applications to interact with one another and share information. A Web API is the public face of a web service, listening as well as reacting to client requests. REST architectural styles are extensively used in the building of APIs for modern web services. A REST API is a Web API that adheres to the REST architectural style.

Client server request

The presence of a REST API qualifies a web service as "RESTful." A REST API is a collection of interconnected resources. This collection of resources is referred to as the REST API's  resource model. REST APIs that are well-designed can entice client developers to adopt web services. An aesthetically beautiful REST API design is a must-have feature in today's open market.

Creating a CRUD RESTful API.

CRUD is simply an acronym used for Create, Read, Update and Delete. In this blog tutorial, we are going to build the CRUD Restful API using Node(JavaScript runtime), Express(Node.js framework), HarperDB(Hybrid SQL & NoSQL scalable database), Docker(virtualization to deliver software in packages) and Hoppscotch(open source API testing environment).

Node.js

Ryan Dahl created Node.js in 2009, roughly thirteen years after the debut of the first server-side JavaScript environment, Netscape's LiveWire Pro Web. Node is a JavaScript runtime built on chrome’s V8 engine and executes JavaScript code outside of the web-browser. Node.js allows developers to utilize JavaScript to create command line tools and server-side scripting, which involves running scripts on the server to generate dynamic web page content before the page is transmitted to the user's web browser.

Nodejs

Express

Express is an open source back end web application framework for Node.js. It is intended for use in the development of web applications and APIs. It has been dubbed the "de facto standard server framework for Node.js."

express

Docker

Docker is a platform/tool for developers to build, run and share applications using the concept of containers. The process of deploying an app using containers is called containerizations. Docker is not only the technology which uses containers, but it is the most widely used and the current de facto standard.

Docker

What is a docker container?

Docker containers wraps up code and all its dependencies so that the program can be moved from one computing environment to another quickly and reliably.

Features of Containers.

  • Containers are flexible because no matter the complexity, the dependencies and the programming languages, every application can be containerized.

  • Containers are lightweight because they share the same kernel, and they don’t consume a lot of system resources.

  • Containers are portable because they can be built locally and then can be run on any pc where docker is installed.

  • Containers are loosely coupled, they are well encapsulated and highly self-sufficient so you can change, upgrade and delete a container without disrupting the whole system and it is very good for big projects where different developers work on different parts of the same systems at the same time.

  • Containers are scalable, which means the user can adjust and automatically distribute container replicas without any big effort, users can also use orchestration tools like Kubernetes.

  • Containers are secure because they apply aggressive constraints and isolations to processes without any configuration needed.

What is Docker Image?

Each container has its own dedicated file system(fs) which is provided by a docker Image. Docker Image is an executable packaged file that contains everything users need to run the application, code, dependencies, tools, libraries and configuration.

Docker Image is like a class and Docker container is an instance of that class. Hence, users can add as many containers running from the same image at the same time.

Hoppscotch.

Hoppscotch is a fully open sourced API development ecosystem created by Liyas Thomas and other open source contributors. It allows users to test the APIs directly from their browser window without switching back and forth between multiple applications. Hoppscotch offers a plethora of awesome features like custom themes, WebSocket communication, GraphQL testing, user authentications , API requests history , proxy, API documentations, Keyboard shortcuts, API collections and much more.

Hoppscotch

Hoppscotch also allows users to authenticate via github & google account to save and sync their history, collections and environment. Hoppscotch is compatible with a wide number of browsers and devices, and it can be also installed as a PWA (Progressive Web App).

Harper DB

Harper DB is a schema-less single model enterprise-class database implemented in Nodejs, drastically simplifying Big Data Architectures. Combining structured and unstructured data workloads have been difficult and costly. Harper DB has developed a database that uses SQL & NoSQL in a single model with an easy-to-use REST API. Harper DB divides data into independent indices, with each characteristic saved separately, allowing the data to be fully indexed but requiring no additional storage space. Using parallel computing, programmers and developers may run complicated multi-table SQL queries or JSON searches on a unified model. Harper DB reassembles these exploded indices into single object arrays, allowing developers to run structured queries on unstructured data in real-time with no transformational delay. Harper DB is written in node, it leverages a multi-core architecture and is capable of scaling as per the user's hardware, it eliminates row locking and collisions while maintaining ACID compliance. It also eliminates the need to duplicate data, which prevents the use of large amounts of RAM and disk storage. Furthermore, the Nodejs framework enables Harper DB to be highly portable, with a tiny service-oriented architecture and a reduced runtime footprint. Developers and programmers can concentrate on their code rather than DevOps by leveraging technologies and interfaces that they are already familiar with, such as ANSI SQL and REST.

HarperDB

HarperDB is built to handle a wide range of use cases, including edge computing, running an application database, data warehousing, and transactional and document stores, making it ideal for running directly on a micro computing edge device in the cloud or at a data center.

Let's get started and create some CRUD API.

So we'll just make a simple student CRUD REST API that allows users to retrieve all or only one of the student information, create and add student information, delete student information, and ultimately edit student information.

https://studio.harperdb.io/sign-up

Signup HarperDB cloud instance

Now, create a Free instance of Harper DB by clicking on “Create new HarperDB cloud Instance”

Create a HarperDB cloud instance

After that, a popup will appear . Click on “Create HarperDB cloud Instance”.

Create a HarperDB cloud instance

Now, add username, password and Instance name like shown below.

instance information

Leave the default-free Instance ram and storage size and choose your preferred Instance Region.

Instance specs

Click on “I agree” and proceed to add an instance.

Confirm Instance Details

Creating Instance

Creating instance

Click on the instance and Please keep in mind that we do not need to enter all of the column values here; they will be added automatically when needed.

Instance

Schema Name

Schema and Tables

Step-by-step instructions for crafting our API.

  • Create a project folder. Project Folder 
mkdir crud-api
cd crud-api
Enter fullscreen mode Exit fullscreen mode
  • Initialize Node application inside that project folder.

npm init-y 

npm init -y
Enter fullscreen mode Exit fullscreen mode
  • Install four dependencies – express, nodemon, harperive & dotenv

Package Installation

npm install express harperive nodemon dotenv
Enter fullscreen mode Exit fullscreen mode

  • Create ”controllers”, ”routes” and “util” folders.
    Folder structure

  • Create index.js file.
    import express and initialize

const express = require("express");
const app = express();
app.use(express.json());
Enter fullscreen mode Exit fullscreen mode

Import express , initialize it & set headers.
Setting up proper header

const express = require("express");
const app = express();
require("dotenv").config();

app.use(express.json());

const PORT = process.env.PORT || 5000;

app.use((req, res, next) => {
  res.setHeader("Access-Control-Allow-Origin", "*");
  res.setHeader(
    "Access-Control-Allow-Methods",
    "GET, POST, OPTIONS, PUT, PATCH, DELETE"
  );
  res.setHeader(
    "Access-Control-Allow-Headers",
    "X-Requested-With,content-type"
  );
  res.setHeader("Access-Control-Allow-Credentials", true);
  next();
});
Enter fullscreen mode Exit fullscreen mode
  • Setup two routes. One for testing purposes and another one for actual implementation.

Setting up route

app.use("/testing", require("./routes/testing.routes.js"));
app.use("/students", require("./routes/students.routes.js"));
app.listen(process.env.PORT, () => {
  console.log(`App is currently running at http://localhost:${PORT}`);
});
Enter fullscreen mode Exit fullscreen mode
  • Create db.js inside util folder and create connection for HarperDB. 
// create connection for Harper DB
const harperive = require("harperive");
const configuration = {
  username: process.env.HARPER_INSTANCE_USERNAME,
  password: process.env.HARPER_INSTANCE_PASSWORD,
  schema: process.env.HARPER_INSTANCE_SCHEMA,
  harperHost: process.env.HARPER_HOST_INSTANCE_URL,
};
const db = new harperive.Client(configuration);
module.exports = db;
Enter fullscreen mode Exit fullscreen mode
  • Create testing.routes.js file inside routes folder. It is just a test endpoint to test whether the application is working or not. 
const controller = require("../controllers/testing.controllers.js");
const router = require("express").Router();
router.get("/appinfo", controller.getAppInfo);
module.exports = router;
Enter fullscreen mode Exit fullscreen mode
  • Create students.routes.js file inside routes folder and add references to your api endpoint. 
const router = require("express").Router();
const controller = require("../controllers/" + "students" + ".controllers");
router
  .get("/", controller.getAllStudent)
  .get("/:id", controller.getOneStudent)
  .post("/", controller.createOneStudent)
  .put("/:id", controller.updateOneStudent)
  .delete("/:id", controller.deleteOneStudent);
module.exports = router;
Enter fullscreen mode Exit fullscreen mode
  • Create testing.controllers.js file inside controllers folder. This will be used for testing purposes only, to test whether the app / DB instances are running or not. 
exports.getAppInfo = (req, res, next) => {
  return res.status(200).json({ "Aviyel CRUD API Testing": "v1.0.0" });
};
Enter fullscreen mode Exit fullscreen mode
  • Create students.controllers.js file inside controllers folder and add the following code. 
const client = require("../util/db");
const DB_SCHEMA = process.env.HARPER_INSTANCE_SCHEMA;
const TABLE = "students";
Enter fullscreen mode Exit fullscreen mode

getAllStudent method fetches all the student info.

//Get all the student
exports.getAllStudent = async (req, res, next) => {
  try {
    const qry = `SELECT * FROM ${DB_SCHEMA}.${TABLE}`;
    const students = await client.query(qry);
    res.json(students);
  } catch (error) {
    console.error("ERROR while fetching all student " + "Student:", error);
    return res.status(500).json(error)
}
};
Enter fullscreen mode Exit fullscreen mode

getOneStudent method fetches only one student info by their id.

//Get only one student
exports.getOneStudent = async (req, res, next) => {
  try {
    const qry = `SELECT * FROM ${DB_SCHEMA}.${TABLE} WHERE id="${req.params.id}"`;
    const student = await client.query(qry);
    res.json(student);
  } catch (error) {
    console.error("ERROR while fetching student " + "Student:", error);
    return res.status(500).json(error);
  }
};
Enter fullscreen mode Exit fullscreen mode

createOneStudent method add/insert only one student.

//create new student
exports.createOneStudent = async (req, res, next) => {
  try {
    const user = await client.insert({
      table: TABLE,
      records: [
        {
          username: req.body.username,
          password: req.body.password,
          rollNumber: req.body.rollNumber,
        },
      ],
    });
    res.json(user);
  } catch (error) {
    res.json(error);
  }
};
Enter fullscreen mode Exit fullscreen mode

updateOneStudent method updates only one student.

//update one student
exports.updateOneStudent = async (req, res, next) => {
  try {
    const updateStudent = await client.update({
      table: TABLE,
      records: [
        {
          id: req.params.id,
          username: req.body.username,
          password: req.body.password,
          rollNumber: req.body.rollNumber,
        },
      ],
    });
    res.json(updateStudent);
  } catch (error) {
    res.status(500).json(error);
  }
};
Enter fullscreen mode Exit fullscreen mode

deleteOneStudent method deletes only one student.

//Delete one student
exports.deleteOneStudent = async (req, res, next) => {
  try {
    const qry = `DELETE FROM ${DB_SCHEMA}.${TABLE} WHERE id="${req.params.id}"`;
    const deleteStudent = await client.query(qry);
    res.json(deleteStudent);
  } catch (error) {
    res.status(500).json(error);
  }
};
Enter fullscreen mode Exit fullscreen mode

Now, Let’s create a Docker image of the above-crafted application.

https://docs.docker.com/engine/install/

Create three file Dockerfile, docker-compose.yml and .dockerignore . Inside .dockerignore file add all the code below.

# Node
## Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*

## Dependency directories
node_modules
.git
Enter fullscreen mode Exit fullscreen mode

And,inside Dockerfile add all the code below.(Note: A Dockerfile has no file extension)

FROM node:14.1.0

EXPOSE 8080

WORKDIR /src

COPY package*.json ./

RUN npm install

COPY . . 

CMD ["node", "index.js"]
Enter fullscreen mode Exit fullscreen mode

Add all the below code inside docker-compose.yml and give the image name whatever you want.

version: "3.8"
services:
  app:
    container_name: crudapi_aviyel
    image: pramitmarattha/aviyel-crudapi:0.0.1
    build:
      context: .
    ports:
      - "8080:8080"
    env_file: .env
Enter fullscreen mode Exit fullscreen mode

Update the .env file by adding proper configuration.

PORT=8080
HARPER_HOST_INSTANCE_URL=https://----
HARPER_INSTANCE_USERNAME=………
HARPER_INSTANCE_PASSWORD=………
HARPER_INSTANCE_SCHEMA=………….
Enter fullscreen mode Exit fullscreen mode

For “HARPER_HOST_INSTANCE_URL” , head over to the config of that running cloud instance inside the dashboard UI of Harper DB and copy the URL and paste it inside .env .

HarperDB host Instance URL

To retrieve username and password simply toggle to the users section of that running instance.

HarperDB instance username

And finally, the Schema name can be retrieved from the browse section of that instance.

schemas

Finally, It's time to create a docker image.

Note: Make sure your docker is still up and running.

In your CLI type: docker-compose build

Docker building

And After building image run: docker-compose up or docker compose up

Docker Up and Running

Using “Hoppscotch” to test our APIs.

Hoppscotch Logo

When the request is properly completed, the user can see the status code of 200 and JSON response at the Response & Response Body section respectively.

CRUD api Testing

POST

Let’s create and add some students by sending a “POST request”.

Select the “POST” request from the dropdown, enter the API endpoint and choose Content/Type as an application/JSON. Finally, in the raw request body enter the “username”,” password” and “rollNumber” after that hit the Send button. If everything went right, then you should see “status”:”success” in the response body.

Adding Student Data

HarperDB provides a beautiful and aesthetically pleasing dashboard UI which displays the Realtime records of the tables. If everything went right and the “POST” request was made successfully, then a new record should be populated inside that student’s table.

Student added successfully

GET students by id.

To check whether a student exists or not, a GET request should be made using the student’s id as a query argument at the end of URL.

GET student info by id

GET all students.

When the GET request executes effectively, a JSON response containing all the records of the students will be thrown at the Response Body section along with the status response code of 200.

GET all student

Update student by id.

To update the record of a particular student, a PUT request should be made using the student’s id as a query argument at the end of the URL along with the updated payload / req Body of that particular student.

update student by id

Delete Student by Id.

To delete the record of particular student , DELETE request should be made using the student’s id as a query argument at the end of URL of that particular student.

Delete student id

API Collections and History.

After a while, it may be really tough to handle and revisit the API requests. History provides options to review the past made requests, mark them as favorites and re-run the process at any given time.

API request history

In some cases, we may need to make API requests for many different things at the same time. To divide the requests per resource into collections would be ideal. Hoppscotch makes it really very simple to create an unlimited number of collections.

API endpoint collection

Generating Documentation using “Collection”

Hoppscotch's one of the best features is that it uses the collection to automatically generate documentation for API requests.

API documentation auto generate

The entire project source code is available here :
https://github.com/aviyeldevrel/devrel-tutorial-projects/tree/main/CRUD-api-testing-with-Hoppscotch

Main article available here => https://aviyel.com/post/1546

Follow @aviyelHQ or sign-up on Aviyel for early access if you are a project maintainer, contributor, or just an Open Source enthusiast.

Join Aviyel's Discord => Aviyel's world

Twitter =>[https://twitter.com/AviyelHq]

Image
How Proxies Improve Website Performance and Security
Image
Top 10 Web Development Companies in India 2024

webdev, website, api, devops
Image
Langkah Mengelit Kekalahan Taruhan Judi Online
Image
The Role Of Data Science In Creating Innovation In Renewable Energy
Image
Some Wisdom On Scooters Mobility For Sale From A Five-Year-Old
Image
The Three Greatest Moments In Good Accident Lawyers History
Image
VAPT
Image
API7 Enterprise v3.2.16.3 Integrates with AWS Secrets Manager
Image
Mimicking a List of Dictionaries with a Linked List in Python

webdev, python, dsa, beginners
Image
یک مجله آنلاین است که به ارائه اخبار، تحلی
Image
kelebihan provider slots online flow gaming
Image
Education Consultant in Mohali: Your Pathway to Academic Success

learning, career, performance, startup
Image
The Leading Reasons Why People Are Successful At The Bi Fold Door Repair Industry
Image
10 Accident Lawyer Savannah Hacks All Experts Recommend
Image
How to Create a Web Icon in HTML

webdev
Image
The Power of WeChat Official Accounts in China’s Digital Marketing
Image
What Experts From The Field Of Asbestos Lawsuits Want You To Be Able To
Image
Hosting your own PaaS: Open-Source Tools to Help You Exit the Cloud!

linux, opensource, cloud, beginners
Image
JAIIB INSURANCE REGULATORY AND DEVELOPMENT AUTHORITY OF INDIA (IRDAI)

jaiib, bank, exam, jaiibexam
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .