SkCan we push React to the limit today?, Unlike other data types, real-time data presents an entirely new set of thrilling problems to solve, from processing, transforming and displaying the data. solving these problems can be an enjoyable experience.
We will leverage the power of Hooks, Sockets and Data Structure(s) to handle data , with updates arriving as frequently as every 100 milliseconds.
we will go over web-sockets and the pre-built api server quickly, laying the foundation for the Realtime Application in react. Along the way, we'll utilize Hooks, React Table, and reusable components. It's worth noting that prior knowledge of React is expected, as this article is not aimed at beginners. But don't fret, you can power through and make the most of it!.
By the end of this article, you should be comfortable with web-sockets in React, and handling real-time data, which open doors to a multitude of potential web applications, So without further ado, let's dive in!.
Web-sockets
Web-sockets(ws) is a transfer protocol, alongside http 1 and 2, unlike http, ws is a full duplex, bi-directional non blocking transfer protocol. allowing simultaneous data transfer and communication between two parties.
an alternate of ws is http polling where data is explicitly requested at intervals repeatedly.
example: Http polling chat app
With web-sockets, the connection is established only once and maintained throughout the duration of the session, allowing bi directional, real-time communication between the client and server.
I put together a simple ws server, serving real-time data, We will go over it quickly, git clone or download to get started: Backend code.
Subs web-socket server
Compared to frameworks like django, which I personally love, setting up a web-socket server in node is incredibly simple, thanks to the web-socket package
navigate to the backend code and do npm i, I personally use pnpm.
For the initial web-socket handshake/connection a tcp/http server is needed to act a reverse proxy when the web-socket magic(not really) is done, the connection is upgraded from a tcp/http to a web-socket connection.
The native http package will suffice, we need a simple web server.
const http = require("http")
const httpServer = http.createServer((req, res)=> {
console.log("got a request")
})
httpServer.listen(8080, ()=> console.log("listening -> 8080"))
Hooking the ws to a proxy is not difficult either, the web-socket package handles all the intricacies.
...
const WebSocketServer = require("websocket").server
let connection = null
// create a ws server, and pass the proxy to handle all the initials
const websocket = new WebSocketServer({
"httpServer": httpServer
})
Initially the request event will be triggered(via the proxy) asking the ws for a connection, we are handling that below, after a successful one, we save the connection as a global var connection
websocket.on("request", req=> {
connection = req.accept(null, req.origin)
connection.on("open", () => console.log("connection opened"))
connection.on("close",() => console.log("connection closed"))
connection.on("message",(msg) => console.log("client msg", msg.utf8Data))
})
The connection object exposes methods for events handling, triggering, sending data etc.
You can test the socket server from the web console:
// asking for the initial connection,
let ws = new WebSocket('ws://localhost:8080');
// when the connection is opened:
ws.send(JSON.stringify({event: "realtime"}))
We have successfully created a full duplex server, anything else in the backend code is normal JavaScript.
Navigate and open the code, the entry file is server.js, and will run with a simple node server.js.
The processMsg(just a huge switch statement) function coordinates everything: handles incoming events and perform the relevant action.
for example to get real-time data:
let ws = new WebSocket('ws://localhost:8080');
ws.onmessage = m => {handleMsgs(JSON.parse(m.data), ws)}
ws.send(JSON.stringify({event: "realtime"}))
The "subscribers" data should start trickling in, before we implement the React app, let's go thru the format of the data first.
In data.js, faker.js is responsible for generating the dummy data.
"interface":
function createRandomUser(){
return {
_id: faker.datatype.uuid(),
avatar: faker.image.avatar(),
birthday: faker.date.birthdate(),
email: faker.internet.email(),
firstName: faker.name.firstName(),
lastName: faker.name.lastName(),
sex: faker.name.sexType(),
subscriptionTier: faker.helpers.arrayElement(['basic', 'pro', 'enterprise']),
};
}
Simulate.js does one of three things, generate new subscribers, choose random people from subscribers and make to unfortunately hate the content and unsubscribe, or down or upgrade their subscription,
On every tick(simulation) the updated data is sent to the interested client via the web-socket.
The data we get is an object with three properties, subs, unsubs, down_or_upgrade, with arrays respectively.
{subs: [], unsubs: [], down_or_upgrade: []}
Note: a person can be in subs and up_or_downgrade at the same time.
We can have up to 20 000 people in our data: simulate.js line 25.
That's practically all we need to know about the server, if you are interested you can read on simulate.js and maybe add more functionality, like time of subscription and other events, which will make for a nice real-time chart.
At this rate we are ready to move to react, spin a new react project, I personally prefer vite and react-ts.
npm create vite@latest
or just clone/download the completed Frontend on github.
and perform the necessities: npm i or pnpm install
Web-sockets in react.
Create a components and Hooks folder under the src directory, I subscribe to the "index" folder convention, don't know the real name, where each component is a folder with an index.tsx file as the actual component:
Components/
Realtime/
index.tsx
and "global/generic" components(that can be used by any component), are directly in the components folder.
Realtime component stub:
const RealTime: React.FC = () => {
return (
<div>
RTDATA
</div>
)
}
export default RealTime
The same for Hooks
Hooks/
Realtime/
useSubscribers.ts
Import Realtime in App.tsx
import RealTime from "./Components/Realtime"
function App(){
return (
<div>
<RealTime/>
</div>
)
}
export default App
We have a linear goal at this point, connect to the ws server and ask for data, we will think about data structures and processing later.
navigate to the useSubscribers hook and add:
import {useState, useEffect} from "react";
const useSubscribers = ({websocketUrl}: {websocketUrl:string}) = > {
const [ws, setWebsocket] = useState<undefined | Websocket>()
useEffect(() => {
if(websocketUrl.length !== 0){
try {
const w = new WebSocket(websocketUrl);
setWebsocket(w);
}catch(error){
console.log(error)
}
}
}, [websocketUrl])
return ['nothing yet']
}
export default useSubscribers
We only have one parameter, so why the object?:
({websocketUrl}: {websocketUrl:string})
when the equivalent is easier and cleaner to write:
(websocketUrl: string)
That's a valid observation: we can, but what if we want more parameters?, objects make it easier to extract the types to an interface, and de-structure from there, instead of adding them manually and worrying about their position etc
Add the hook to the Realtime component, to connect to the ws, make sure the backend is running: node server.js:
const RealTime: React.FC = () => {
const [some] = useSubscribers({websocketUrl: "ws://localhost:8080"})
...
}
Now we worry about events coming from the ws socket, in useSubscribers add the following function:
import {useState, useEffect, useCallback} from "react";
const handleWsMessages = useCallback(() => {
if(ws){
ws.onmessage = m => {
try {
let parsed: {type: string, data: Record<any, any>} = JSON.parse(m.data);
switch(parsed.type){
case "realtime":
break;
case "closing":
break;
case "opened":
break;
default:
console.log("i don't know", m)
break;
}
}catch(err){
console.log(err, "prolly failed to parse")
}
}
}
}, [ws])
when this useState is triggered:
const [ws, setWebsocket] = useState<undefined | Websocket>()
The useCallback will re-run, recreating handleWsMessages(we will run it later on), if we have the websocket connection(ws), we register the on message event:
ws.onmessage = m => {}
Which handles all the events we care about from the web socket,
from the websocket we expect an object {type: string, data: object}, type being the event, and data well in this case subscribers.
// parsing the data from string,.
let parsed: {type: string, data: Record<any, any>} = JSON.parse(m.data);
// event sent by the server(consult the server code)
switch(parsed.type){
case "realtime":
break;
case "closing":
break;
case "opened":
break;
default:
console.log("i don't know this event", m)
break;
}
events realtime, closing and opened are the only messages we care about from server socket, and they do exactly as their names implies.
EVENT -> "opened"
when the connection is established, the server socket will send this event, This is where we kick start everything, before we do that, let's take a step back and discuss on a high level how we are going to handle this real-time data and why.
Our server produces data in a consistent interval, which is not true in real world real-time data, people don't do actions in intervals, it's very random, 500 people can subscribe in a second, or 10 in 40 days,
for our purpose the data is trickling in quickly, usually what we are about to do is done server side, for our purpose let's assume we don't have access to the backend, we are just consumers, of a badly designed server that pushes raw data every 100 milliseconds and we need to process it in the front-end,
honestly it's way fun this way, pushing the browser to the limit, in a separate article we will move the hook to a separate web thread.
On socket connection we need to tell the server we are ready for data, it can push: with the realtime event:
case "opened":
ws.send(JSON.stringify({event: "realtime"}))
startTick()
break
startTick is the main processing function in our hook, at an interval it will send data to the realtime component, if there's any processing we do it here, we will implement it shortly, just stub it out inside the hook, so TS will stop screaming.
const startTick = useCallback(()=> {
}, [ws])
EVENT -> realtime
Our tick function runs every 1000 ms, while the socket sends data every 100ms.
See the problem here? data is coming faster than we can process, we need way to store access data before the tick function processes it, and we need to maintain the order of the data as it comes in from the server(FIFO - first in first out), yes we need a queue(FIFO data stucture).
declare a queue outside the hook(globally),
const GLOBALQUEUE: Array<Record<any, any> | undefined> = []
in event -> realtime push the data to the queue.
case "realtime":
// received from the ws server
GLOBALQUEUE.push(parsed.data)
break;
The global queue is responsible for storing incoming data, but at every tick we need to cycle that data, data currently in the GLOBALQUEUE needs to be passed down to the component, and GLOBALQUEUE emptied, waiting for more incoming data, a viscous cycle.
let's declare a local queue for passing data to the component.
const [queue, setQueue] = useState<Array<any>>([])
....// hook stuff here
return [queue] // from return ['nothing yet']
in the component:
// Realtime/index.tsx
const RealTime: React.FC = () => {
const [queue] = useSubscribers({websocketUrl: "ws://localhost:8080"})
...
}
All we are missing now is the tick function, there's not much processing, maybe in the future, it's surprisingly simple, take stuff from the global queue pass it to the local and clear Global.
const startTick = useCallback(()=> {
setInterval(()=> {
if(GLOBALQUEUE.length !== 0){
let t = [...GLOBALQUEUE]
setQueu(prev => {
return [...prev, ...t]
});
GLOBALQUEUE = []
}
}, 1000)
}, [ws])
The reason we copy GLOBALQUEUE into t, instead of directly setting it in setQueue, the latter is very unpredictable, generally copying is bad, you should avoid it if possible,
Finally we can kick start everything.
useEffect(()=> {
if(ws){
handleWsMessages()
}
}, [ws])
This useEffect is the first domino, handleWsMessages will cause case event -> opened to query for "realtime" data: ws.send(JSON.stringify({event: "realtime"})) while simultaneously calling the tick function for processing that data.
The hook is complete for now, will be revisited in future articles.
I broke the article into two parts, the second part will focus on visuals, the stats card, line chart and table to display the live data.
Thank you for reading! If you loved this article and share my excitement for Back-end or Desktop development and Machine Learning in JavaScript, you're in for a treat! Get ready for an exhilarating ride with awesome articles and exclusive content on Ko-fi. And that's not all – I've got Python and Golang up my sleeve too! 🚀 Let's embark on this thrilling journey together!
Oh, but wait, there's more! For those of you craving more than just CRUD web apps, we've got an exclusive spot waiting for you in the All Things Data: Data Collection, Science, and Machine Learning in JavaScript, and a Tinsy Bit of Python program. Limited spots available – don't miss out on the party! Let's geek out together, shall we? 🎉
