Create a Simple Image Classifier with ml5js

0xkoji - Mar 24 '19 - - Dev Community

In this case, I will show you how to create a simple image classifier app with p5js and ml5js.

First, what are p5js and ml5js?

I would say it's Processing for js(Actually there is processing.js).

https://p5js.org/

Here at ITP, most students who don't have any experience of programming start using p5js to learn coding.

ml5js is a wrapper of tensorflowjs, so that allows us to use tensorflowjs easily, but it means that we cannot do everything with ml5js as well as tensorflowjs.
https://github.com/ml5js/ml5-library

index.html
Very simple html. just load libraries.
In terms of ml5js, it has been updated recently, but I haven't updated the code, so this app needs to use v0.2.1.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/p5.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.dom.min.js"></script>
    <script src="./p5.speech.js"></script>
    <script src="https://unpkg.com/ml5@0.2.1/dist/ml5.min.js" type="text/javascript"></script>
    <title>img_rec</title>
</head>
<body>
    <script src="./sketch.js"></script>
</body>
</html>
Enter fullscreen mode Exit fullscreen mode

sketch.js
This app is using MobileNet to classify objects.

let classifier;
let video;
let status = '';
let results = '';
const resultsNum = 5;
const voice = new p5.Speech();

function setup() {
    createCanvas(windowWidth, windowHeight);
    video = createCapture(VIDEO);
    classifier = ml5.imageClassifier('MobileNet',  video, modelReady); status = 'loading...';
}
function draw () {
    image(video, 0, 0, width, height);
    fill(255, 0, 0);
    textSize(24);
    text(status, 20, 30);
}

const modelReady = () => {
    status = 'loaded model!';
    classifier.predict(video, gotResult);
}

const gotResult = (err, results) => {
    if (err) {
        console.error(err);
        status = err;
    }
    // console.log(`results: ${results}`);
    status = `class: ${results[0].className}, accuracy: ${results[0].probability.toFixed(4)} \n`;    
    voice.speak(`${results[0].className}`);
    classifier.predict(video, gotResult);
}

Enter fullscreen mode Exit fullscreen mode

demo

Actually, this application tells you what he detects via camera.
https://thepracticaldev.s3.amazonaws.com/i/chmbw4svkmdcmdyxsyb6.gif

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .