Codementor Events

a file Database from scratch in Node JS part 1

Published Jun 27, 2022Last updated Jul 05, 2022
a file Database from scratch in Node JS part 1

part 2

I have been stuck in my coding journey for a while now, yes I can implement a website from start to finish, a mobile app in ionic to publishing, But I am not content with myself and just knowing CRUD apps, Then started searching, came across this masterpiece what is programming? (noob lessons!) by George Hotz, I instantly knew what to do to grow from just CRUD to a programmer, and this is the start of my journey and I will build and learn in public, hoping to inspire someone out there too.

What I will basically do is take a "lower level concept/problem" that has been solved yes, but I don't know how, example Operating systems, Compilers, Virtual Machines, Front/back-end frameworks and implement them or my version from scratch without or minimal help as possible(basically hacking my way into the solution with the skills I currently have),

however watch tutorials or read books for concepts I am completely unaware of at all, for example OS and implement them in C then try them in JS without help

for concepts I know such as Front-end and db's I will hack my way into a solution without or minimal help, and I will point out when I did seek for help and share links

I hope you start building with me.

setup

// mkdir - create a new folder : database 
// create an index.js file inside the database folder 
  database/ 
      index.js 
// cmd command npm init
// init a new package.json file choose defaults 
// add a field to the package.json file "type": "module" 
// will allow us to use imports in node

The entire package.json file:

{ "name": "database", 
"version": "1.0.0", 
"description": "", 
"main": "index.js",
"type": "module",
"scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, 
"author": "",
"license": "ISC"
}

What I Know

what I know about databases is basically the API, also known as CRUD operations, I don't know the inner workings of db's, algorithms and structures used, absolutely clueless, so I will start from CRUD to a file db.

index.js file

import db from "./database.js" // we need .js in node
export default db

This is a common pattern actually in modules or libraries, the entry file just exposes the core API, this is what I am going for here, exposing a db function when called will return an object with functions to operate on the database.

I don't know about you but when I build something I like starting from the user's perspective backwards, which may be bad depending on how you look at it.

But my reasoning behind it is simple, expose a simple as possible interface and build all the complexity around it without changing the endpoint, it a constraint of some sort, and it's my job to figure out how to constrain all complexity towards that simple endpoint. Which currently works well for me, even when refactoring I strive for the endpoint not to change.

database.js

Now I am abstracting away from the tip/endpoint, that is my thought pattern right now, I think of this as a vortex, index.js is the tip of the vortex the more files I add, the more shape the body of the vortex takes, however the user(programmer in this case) will see the tip, unless they want to explore.

I call this second layer file, the composer or consolidator, which I am unsure if it captures the essence of what I am trying to explain, at least it makes sense in my head at the moment, I don't know about you.

I am assuming of course that the db will have a lot of features(files you can think of them) and the job of the composer is to take all these features and pipe/compose them to this tiny endpoint, this is like another protection or a guarantee that every feature will consolidate to db, database.js is another safeguard of sort and feeds the tip of the vortex

import {insert, update, delete_, select} from './operators.js' // features to be composed

function db(options) { 
// I assume metadata is somewhat useful in db's will explain as I use them 

this.meta = { 
     length: 0, // of the store 
     types: {}, // we can bind types to each column in the store 
     options // passed from the vortex
     } 
     
     this.store = {} // our store for now is a simple object 
     
     
   }
     
 
 // every instance of db will point to this single prototype
 // composing all the features
db.prototype.insert = insert
db.prototype.update = update
db.prototype.select = select
db.prototype.delete_ = delete_ 

// exporting the endpoint
export default db

A minor observation is: I actually don't like that the store(actual table) is accessible from the endpoint and can be changed, definitely needs a refactor(will deal with it later) for now its fine

why prototype

well the idea is simple really, every instance points to that single proto object, for example if we have 100 db instances we won't have 100 proto objects with insert, update etc but one

if you are confused by how this work or unfamiliar I made a series of articles creating a prototype emulator and learning OOJS(object oriented JS) you can check them out.

Operators.js

these are the features, the body of the vortex is expanding in both directions, initially I thought of separating them by files but since the codebase is still small I don't see the need yet

import {isInDb} from "./utils.js" // its no library without utils :) 

// insert is very simple for now
export function insert(row){ 
try{ 
// options only handle timeStamp(boolean) 
    if(this.meta.options.timeStamp){ 
    row["timeStamp"] = Date.now() // insert date
    } 
    this.store[this.meta.length] = row // insert the row in the next id               
    this.meta.length++ // increase ID  length 
       return true // operation succesful
    }catch(err){
    console.log(err) // for now 
    return false // operation failed 
     } 
    
    } 
    
    export function select(){ } 
    
    export function delete_(){ } 
    
    export function update(){ }

utils.js

a utility file is very useful for small and reusable functionality, even if you don't use them now, it's good to have them ready. In this case I sensed i will need a function to tell me whether a document exists in a db

// function does not copy store, rather receives a pointer
// so no memory wastage
export function isInDb(store, id){



   return store[id] !== undefined ? true : false



}

test.js

not an actual test, like code test but checking if the db works as expected


import db from './index.js'



let store = new db({timeStamp: true})



console.log(store)

store.insert({name: "John", surname: "Doe"})
store.insert({name: "Jane", surname: "Doe"})
console.log(store)

I decided to stop here

for one reason really I had so many ideas but didn't really know how to really go about them for example I thought of a string like sqlite ''SELECT * in' however it goes, but also thought of filters etc, then I am deciding to watch a simple python tutorial just to glean on the API and see if I can spin it somehow and make it work here in my own way.

Part two

coming soon!

conclusion

If you want a programming buddy I will be happy to connect on twitter , or you or you know someone who is hiring for a front-end(react or ionic) developer or just a JS developer(modules, scripting etc) I am looking for a job or gig please contact me: mhlungusk@gmail.com, twitter will also do

Thank you for your time, enjoy your day or night. until next time

Discover and read more posts from Sfundo Mhlungu
get started
post comments3Replies
Thomas Theiner
2 years ago

Thank you for taking us on this journey. Just one remark: Please check the indentation and line breaks in the operators.js

Looks like the line “this.meta.length++” is commented out…

Thomas Theiner
2 years ago

Did you see my remark?

Sfundo Mhlungu
2 years ago

Thank you Thomas for your kind words and observation, I just updated the article, I am sorry for taking too long to respond.

thanks again