0xkojiWhat is Posenet?
https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5
https://github.com/tensorflow/tfjs-models/tree/master/posenet
As you can see, basically we don't need to use nodejs for PoseNet since we need to use canvas to feed data to tensorflowjs lol
Also, we can use tensorflow with python instead of js, but I did try because there are nodejs and tfjs-node(https://github.com/tensorflow/tfjs-node).
steps
- install packages/libs
- install npm packages
- write code & run it
Step1
As I mentioned, need to use canvas, so need to install libs.
By the way, this is for mac, but you can find out information for Linux and Windows on the internet. In addition, now we can use Homebrew on Linux(I tested it on Ubuntu)
$ brew install pkg-config cairo pango libpng jpeg giflib
Step2
I'm using yarn since npm didn't allow me to install tfjs. But, you can use/try npm instead of yarn.
$ yarn add @tensorflow-models/posenet @tensorflow/tfjs @tensorflow/tfjs-node botkit canvas rollup
Step3
The code is messy since just test tfjs-node with posenet.
const tf = require('@tensorflow/tfjs-node');
const posenet = require('@tensorflow-models/posenet');
const {
createCanvas, Image
} = require('canvas')
const imageScaleFactor = 0.5;
const outputStride = 16;
const flipHorizontal = false;
const tryModel = async() => {
console.log('start');
const net = await posenet.load(0.75);
const img = new Image();
img.src = './test.jpg';
const canvas = createCanvas(img.width, img.height);
const ctx = canvas.getContext('2d');
ctx.drawImage(img, 0, 0);
const input = tf.browser.fromPixels(canvas);
const pose = await net.estimateSinglePose(input, imageScaleFactor, flipHorizontal, outputStride);
// console.log(pose);
for(const keypoint of pose.keypoints) {
console.log(`${keypoint.part}: (${keypoint.position.x},${keypoint.position.y})`);
}
console.log('end');
}
tryModel();
Result
Got 17 keypoints, yay!
