Custom Processor for Deep Learning

Happy Mm.. FLaNK Day!

I have been experimenting with the awesome new Apache 2.0 licensed Java Deep Learning Library, DJL. In NiFi I was trying to figure out a quick use case and demo. So I use my Web Camera processor to grab a still shot from my Powerbook webcam and send it to the processor. The results are sent to slack.

Since it's the holidays I think of my favorite holiday movies: The Matrix and Blade Runner. So I thought a Voight-Kampf test would be fun. Since I don't have a Deep Learning QA piece built yet, let's start by seeing if you look human. We'll call them 'person'. I am testing to see if I am a replicant. Sometimes hard to tell. Let's see if DJL thinks I am human.

See : http://nautil.us/blog/-the-science-behind-blade-runners-voight_kampff-test

Okay, so it least it thinks I am a person. The classification of a Christmas tree is vaguely accurate.

It did not identify my giant french bread.

Building A New Custom Processor for Deep Learning

The hardest part of was a good NiFi Integration test. The DJL team provide some great examples and it's really easy to plug into their models.

ZooModel model =

MxModelZoo.SSD.loadModel(criteria, new ProgressBar())

Predictor predictor = model.newPredictor()

DetectedObjects detection = predictor.predict(img);

All the source is in github and references the below DJL sites and repos.

Using a New Custom Processor as part of a Real-time Holiday Flow

We first add the DeepLearningProcessor to our canvas.

An example flow:

• GetWebCameraProcessor : grab an image from an attached webcamera
• UpdateAttribute : Add media type for image
• DeepLearningProcessor : Run our DJL deep learning model from a zoo
• PutSlack : Put DJL results in a text window in slack
• PostSlack : Send our DJL altered image to slack
• Funnel : Send all failures to Valhalla

If we example the provenance we can see how long it took to run and some other interesting attributes.

We place the results of our image analysis in attributes while we return a new image that has a bounding box on the found object(s).

We now a full classification workflow for real-time deep learning analysis on images, could be used for Santa watching, Security, Memes and other important business purposes.

The initial release is available here: https://github.com/tspannhw/nifi-djl-processor/releases/tag/v1.0

Using library and example code from the Deep Java Library (https://djl.ai/).

• https://github.com/awslabs/djl#getting-started

• https://djl.ai/mxnet/mxnet-model-zoo/

• https://github.com/awslabs/djl/blob/master/examples/pom.xml

• https://github.com/tspannhw/djl/blob/master/examples/src/main/java/ai/djl/examples/inference/ObjectDetection.java

• https://github.com/aws-samples/djl-demo

• https://github.com/awslabs/djl/blob/master/jupyter/BERTQA.ipynb

• https://github.com/awslabs/djl/blob/master/examples/src/main/java/ai/djl/examples/inference/BertQaInference.java

*Source Code: * https://github.com/tspannhw/nifi-djl-processor/

And now for something completely different, Christmas Cats: