File Database in Node Js from scratch part 1: introduction & setup

Sk - Jun 26 '22 - - Dev Community

Introduction

I have been stuck in my coding journey for a while now, yes I can implement a website from start to finish, a mobile app in ionic to publishing, But I am not content with myself and just knowing CRUD apps, Then started searching, came across this masterpiece what is programming? (noob lessons!) by George Hotz, I instantly knew what to do to grow from just CRUD to a programmer, and this is the start of my journey and I will build and learn in public, hoping to inspire someone out there too.

What I will basically do is take a "lower level concept/problem" that has been solved yes, but I don't know how, example Operating systems, Compilers, Virtual Machines, Front/back-end frameworks and implement them or my version from scratch without or minimal help as possible(basically hacking my way into the solution with the skills I currently have),

however watch tutorials or read books for concepts I am completely unaware of at all, for example OS and implement them in C then try them in JS without help

for concepts I know such as Front-end and db's I will hack my way into a solution without or minimal help, and I will point out when I did seek for help and share links

I hope you start building with me.

setup


// mkdir - create a new folder : database 
// create an index.js file inside the database folder 


database/
    index.js


// cmd command 

npm init // init a new package.json file choose defaults



// add a field to the package.json file

"type": "module" // will allow us to use imports in node
Enter fullscreen mode Exit fullscreen mode

The entire package.json file:


{
  "name": "database",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}




Enter fullscreen mode Exit fullscreen mode

What I Know

what I know about databases is basically the API, also known as CRUD operations, I don't know the inner workings of db's, algorithms and structures used, absolutely clueless, so I will start from CRUD to a file db.

index.js file

import db from "./database.js" // we need .js in node
export default db



Enter fullscreen mode Exit fullscreen mode

This is a common pattern actually in modules or libraries, the entry file just exposes the core API, this is what I am going for here, exposing a db function when called will return an object with functions to operate on the database.

I don't know about you but when I build something I like starting from the user's perspective backwards, which may be bad depending on how you look at it.

But my reasoning behind it is simple, expose a simple as possible interface and build all the complexity around it without changing the endpoint, it a constraint of some sort, and it's my job to figure out how to constrain all complexity towards that simple endpoint. Which currently works well for me, even when refactoring I strive for the endpoint not to change.

database.js

Now I am abstracting away from the tip/endpoint, that is my thought pattern right now, I think of this as a vortex, index.js is the tip of the vortex the more files I add, the more shape the body of the vortex takes, however the user(programmer in this case) will see the tip, unless they want to explore.

I call this second layer file, the composer or consolidator, which I am unsure if it captures the essence of what I am trying to explain, at least it makes sense in my head at the moment, I don't know about you.

I am assuming of course that the db will have a lot of features(files you can think of them) and the job of the composer is to take all these features and pipe/compose them to this tiny endpoint, this is like another protection or a guarantee that every feature will consolidate to db, database.js is another safeguard of sort and feeds the tip of the vortex

import {insert, update, delete_, select} from './operators.js' // features to be composed


function db(options) {
    // I assume metadata is somewhat useful in db's will explain as I use them
    this.meta = {
           length: 0, // of the store
           types: {}, // we can bind types to each column in the store  
           options // passed from the vortex


    }
    this.store = {} // our store for now is a simple object


}

// every instance of db will point to this single prototype

// composing all the features
db.prototype.insert = insert
db.prototype.update = update
db.prototype.select = select
db.prototype.delete_ = delete_



// exporting the endpoint
export default db 



Enter fullscreen mode Exit fullscreen mode

A minor observation is: I actually don't like that the store(actual table) is accessible from the endpoint and can be changed, definitely needs a refactor(will deal with it later) for now its fine

why prototype

well the idea is simple really, every instance points to that single proto object, for example if we have 100 db instances we won't have 100 proto objects with insert, update etc but one

if you are confused by how this work or unfamiliar I made a series of articles creating a prototype emulator and learning OOJS(object oriented JS) you can check them out.

Operators.js

these are the features, the body of the vortex is expanding in both directions, initially I thought of separating them by files but since the codebase is still small I don't see the need yet



import {isInDb} from "./utils.js" // its no library without utils :)




// insert is very simple for now
export function insert(row){


     try{
         // options only handle timeStamp(boolean)
         if(this.meta.options.timeStamp){
            row["timeStamp"] = Date.now() // insert date
         }

         this.store[this.meta.length] = row // insert the row in the next id 
         this.meta.length++ // increase ID/length
         return true   // operation succesfull
     }catch(err){
          console.log(err) // for now
          return false // operation failed


    }


}






export function select(){


}




export function delete_(){



}


export function update(){



}




Enter fullscreen mode Exit fullscreen mode

utils.js

a utility file is very useful for small and reusable functionality, even if you don't use them now, it's good to have them ready.
In this case I sensed i will need a function to tell me whether a document exists in a db


// function does not copy store, rather receives a pointer
// so no memory wastage
export function isInDb(store, id){



   return store[id] !== undefined ? true : false



}




Enter fullscreen mode Exit fullscreen mode

test.js

not an actual test, like code test but checking if the db works as expected


import db from './index.js'



let store = new db({timeStamp: true})



console.log(store)

store.insert({name: "John", surname: "Doe"})
store.insert({name: "Jane", surname: "Doe"})
console.log(store)




Enter fullscreen mode Exit fullscreen mode

I decided to stop here

for one reason really I had so many ideas but didn't really know how to really go about them for example I thought of a string like sqlite ''SELECT * in' however it goes, but also thought of filters etc, then I am deciding to watch a simple python tutorial just to glean on the API and see if I can spin it somehow and make it work here in my own way.

Part two

coming soon!

conclusion

If you want a programming buddyI will be happy to connect on twitter , or you or you know someone who is hiring for a front-end(react or ionic) developer or just a JS developer(modules, scripting etc) I am looking for a job or gig please contact me: mhlungusk@gmail.com, twitter is also fine

Thank you for your time, enjoy your day or night. until next time

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .