🎨 Image processing as a service 🐍

Laurent Picard - Apr 7 '22 - - Dev Community

👋 Hello

Have you ever written a script to transform an image? Did you share the script with others or did you run it on multiple computers? How many times did you need to update the script or the setup instructions? Did you end up making it a service or an online app? If your script is useful, you’ll likely want to make it available to others. Deploying processing services is a recurring need – one that comes with its own set of challenges. Serverless technologies let you solve these challenges easily and efficiently.

In this post, you’ll see how to…

  • Create an image processing service that generates coloring pages
  • Make it available online using minimal resources

…and do it all in less than 200 lines of Python and JavaScript!

🛠️ Tools

To build and deploy a coloring page generator, you’ll need a few tools:

  • A library to process images
  • A web application framework
  • A web server
  • A serverless solution to make the demo available 24/7

🧱 Architecture

Here is one possible architecture for a coloring page generator using Cloud Run:

Architecture serving a web app with Cloud Run

And here is the workflow:

  • 1 - The user opens the web app: the browser requests the main page.
  • 2 - Cloud Run serves the app HTML code.
  • 3 - The browser requests the additional needed resources.
  • 4 - Cloud Run serves the CSS, JavaScript, and other resources.
  • A - The user selects an image and the frontend sends the image to the /api/coloring-page endpoint.
  • B - The backend processes the input image and returns an output image, which the user can then visualize, download, or print via the browser.

🐍 Software stack

Of course, there are many different software stacks that you could use to implement such an architecture.

Here is a good one based on Python:

schema

It includes:

  • Gunicorn: A production-grade WSGI HTTP server
  • Flask: A popular web app framework
  • scikit-image: An extensive image processing library

Define these app dependencies in a file named requirements.txt:

# https://pypi.org/project/gunicorn
gunicorn==20.1.0

# https://pypi.org/project/flask
Flask==2.1.1

# https://pypi.org/project/scikit-image
# scikit-image dependencies include NumPy and Pillow
scikit-image==0.19.2
Enter fullscreen mode Exit fullscreen mode

🎨 Image processing

How do you remove colors from an image? One way is by detecting the object edges and removing everything but the edges in the result image. This can be done with a Sobel filter, a convolution filter that detects the regions in which the image intensity changes the most.

Create a Python file named main.py, define an image processing function, and within it use the Sobel filter and other functions from scikit-image:

import numpy as np
import skimage
from PIL import Image
from PIL.Image import Image as PilImage


def generate_coloring_page(input: PilImage) -> PilImage:
    # Convert to grayscale if needed
    if input.mode != "L":
        input = input.convert("L")
    np_image = np.asarray(input)

    # Detect the edges
    np_image = skimage.filters.sobel(np_image)
    # Convert to 8 bpp
    np_image = skimage.util.img_as_ubyte(np_image)
    # Invert to get dark edges on a light background
    np_image = 255 - np_image
    # Improve the contrast
    np_image = skimage.exposure.rescale_intensity(np_image)

    return Image.fromarray(np_image)
Enter fullscreen mode Exit fullscreen mode

Note: The NumPy and Pillow libraries are automatically installed as dependencies of scikit-image.

As an example, here is how the Cloud Run logo is processed at each step:

Colored input transformed into edge-detected grayscale output

✨ Web app

Backend

To expose both endpoints (GET / and POST /api/coloring-page), add Flask routes in main.py:

import io

import flask
from PIL import Image

app = flask.Flask(__name__, static_url_path="")


@app.get("/")
def index():
    return app.send_static_file("index.html")


@app.post("/api/coloring-page")
def coloring_page():
    file = flask.request.files.get("input-image")
    if file is None:
        return "Missing input-image parameter", 400

    input_image = Image.open(file.stream)
    output_image = generate_coloring_page(input_image)

    image_io = io.BytesIO()
    output_format = "png"
    output_image.save(image_io, format=output_format)
    image_io.seek(0)

    return flask.send_file(image_io, mimetype=f"image/{output_format}")
Enter fullscreen mode Exit fullscreen mode

Frontend

On the browser side, write a JavaScript function that calls the /api/coloring-page endpoint and receives the processed image:

async function fetchColoringPage(inputFile) {
    const formData = new FormData()
    formData.append('input-image', inputFile)

    const url = '/api/coloring-page'
    const init = { method: 'POST', body: formData }
    try {
        const response = await fetch(url, init)
        return response.ok ? response.blob() : null
    } catch (error) {
        console.error(error)
        return null
    }
}
Enter fullscreen mode Exit fullscreen mode

The base of your app is there. Now you just need to add a mix of HTML + CSS + JS to complete the desired user experience.

Local development

To develop and test the app on your computer, once your environment is set up, make sure you have the needed dependencies:

pip install --upgrade -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

Add the following block to main.py. It will only execute when you run your app manually:

import os

# ...

if __name__ == "__main__":
    os.environ["FLASK_ENV"] = "development"
    app.run(host="localhost", port=8080, debug=True)
Enter fullscreen mode Exit fullscreen mode

Run your app:

python main.py
Enter fullscreen mode Exit fullscreen mode

Flask starts a local web server:

 * Serving Flask app 'main' (lazy loading)
 * Environment: development
 * Debug mode: on
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 718-408-327
 * Running on http://localhost:8080/ (Press CTRL+C to quit)
Enter fullscreen mode Exit fullscreen mode

Note: In this mode, you’re using a development web server (one that is not suited for production). You’ll next set up the deployment to serve your app with Gunicorn, a production-grade server.

You're all set. Open localhost:8080 in your browser, test, refine, and iterate.

🚀 Deployment

Once your app is ready for prime time, you can define how it will be served with this single line in a file named Procfile:

web: gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app
Enter fullscreen mode Exit fullscreen mode

At this stage, here are the files found in a typical project:

.
├── main.py
├── Procfile
├── requirements.txt
└── static
    ├── favicon.ico
    ├── index.html
    ├── scripts.js
    └── styles.css
Enter fullscreen mode Exit fullscreen mode

That's it, you can now deploy your app from the source folder:

SERVICE="coloring-page-generator"
SOURCE="."

gcloud run deploy $SERVICE --source $SOURCE --allow-unauthenticated
Enter fullscreen mode Exit fullscreen mode

⚙️ Under the hood

The command line output details all the different steps:

This command is equivalent to running `gcloud builds submit --pack image=[IMAGE] SOURCE` and `gcloud run deploy SERVICE --image [IMAGE]`

Building using Buildpacks and deploying container to Cloud Run service [SERVICE] in project [PROJECT_ID] region [REGION]
OK Building and deploying... Done.
  OK Creating Container Repository...
  OK Uploading sources...
  OK Building Container... Logs are available at […].
  OK Creating Revision...
  OK Routing traffic...   
  OK Setting IAM Policy...
Done.
Service [SERVICE] revision [SERVICE-REVISION] has been deployed and is serving 100 percent of traffic.
Service URL: https://SERVICE-PROJECTHASH-REGIONID.a.run.app
Enter fullscreen mode Exit fullscreen mode

Cloud Build is indirectly called to containerize your app. One of its core components is Google Cloud Buildpacks, which automatically builds a production-ready container image from your source code. Here are the main steps:

  • Cloud Build fetches the source code.
  • Buildpacks autodetects the app language (Python, in this case) and uses the corresponding secure base image.
  • Buildpacks installs the app dependencies (defined in requirements.txt for Python).
  • Buildpacks configures the service entrypoint (defined in Procfile for Python).
  • Cloud Build pushes the container image to Artifact Registry.
  • Cloud Run creates a new revision of the service based on this container image.
  • Cloud Run routes production traffic to it.

Notes:

  • Buildpacks currently supports the following runtimes: Go, Java, .NET, Node.js, and Python.
  • The base image is actively maintained by Google, scanned for security vulnerabilities, and patched against known issues. This means that, when you deploy an update, your service is based on an image that is as secure as possible.
  • If you need to build your own container image, for example with a custom runtime, you can add your own Dockerfile and Buildpacks will use it instead.

💫 Updates

More testing from real-life users shows some issues.

First, the app does not handle pictures taken with digital cameras in non-native orientations. You can fix this using the EXIF orientation data:

-from PIL import Image
+from PIL import Image, ImageOps
...
def generate_coloring_page(input: PilImage) -> PilImage:
    # Convert to grayscale if needed
    if input.mode != "L":
        input = input.convert("L")
+   # Transpose if taken in non-native orientation (rotated digital camera)
+   NATIVE_ORIENTATION = 1
+   if input.getexif().get(0x0112, NATIVE_ORIENTATION) != NATIVE_ORIENTATION:
+       input = ImageOps.exif_transpose(input)
    np_image = np.asarray(input)
    ...
Enter fullscreen mode Exit fullscreen mode

In addition, the app is too sensitive to details in the input image. Textures in paintings, or noise in pictures, can generate many edges in the processed image. You can improve the processing algorithm by adding a denoising step upfront:

...
def generate_coloring_page(input: PilImage) -> PilImage:
    ...
+   # Remove some noise to keep the most visible edges
+   np_image = skimage.restoration.denoise_tv_chambolle(np_image, weight=0.05)
    # Detect the edges
    np_image = skimage.filters.sobel(np_image)
    ...
Enter fullscreen mode Exit fullscreen mode

This additional step makes the coloring page cleaner and reduces the quantity of ink used if you print it:

La nascita di Venere by Botticelli, with and without denoising

Redeploy, and the app is automatically updated:

gcloud run deploy $SERVICE --source $SOURCE
Enter fullscreen mode Exit fullscreen mode

🎉 It's alive

The app is visible as a service in Cloud Run:

screenshot

The service dashboard gives you an overview of app usage:

screenshot

That's it; your image processing app is in production!

Animated Demo

🤯 It's serverless

There are many benefits to using Cloud Run in this architecture:

  • Your app is available 24/7.
  • The environment is fully managed: you can focus on your code and not worry about the infrastructure.
  • Your app is automatically available through HTTPS.
  • You can map your app to a custom domain.
  • Cloud Run scales the number of instances automatically and the billing includes only the resources used when your code runs.
  • If your app is not used, Cloud Run scales down to zero.
  • If your app gets more traffic (imagine it makes the news), Cloud Run scales up to the number of instances needed.
  • You can control performance and cost by fine-tuning many settings: CPU, memory, concurrency, minimum instances, maximum instances, and more.
  • Every month, the free tier offers the first 50 vCPU-hours, 100 GiB-hours, and 2 million requests for no cost.

💾 Source code

The project includes just seven files and less than 200 lines of Python + JavaScript code.

You can reuse this demo as a base to build your own image processing app:

🖖 More

Image
What is Algorithm

programming, algorithms, webdev, tutorial
Image
An Easy-To-Follow Guide To Asbestos Defense Attorney
Image
Fluent 2 UI for Flutter

flutter, dart, fluent2, design
Image
Venn Diagram Generator

codepen
Image
Meble, które ożywią Twoje wnętrze: Trendy i inspiracje na 2023 rok
Image
How to Add 100% Free Support Chat to Your Application with Tawk.to

chatgpt, nextjs, typescript, coding
Image
Top 10 Cybersecurity Threats You Need to Know About This Year 2024

cybersecurity, discuss, danger, security
Image
Daily Drawing Challenge: A Creative Journey with Tolgee

webdev, opensource, hacktoberfest, nextjs
Image
pelayanan dan spesifikasi yang ada selalu pada agen slots online dapat dipercaya
Image
7 Things You've Never Learned About Pedestrian Accident Lawyer
Image
Ten Accident Lawyers Myths You Shouldn't Share On Twitter
Image
Monday.com Salesforce Integration Guide

blog, integrations, salesforce
Image
The Ultimate Guide to Protecting Windows 11
Image
A Peek At The Secrets Of Mesothelioma From Asbestos Exposure
Image
trik untung besar di permainan slots online
Image
My Third PR for Hacktoberfest
Image
Cat Flaps For French Doors
Image
Deep Dive Prompts

promptengineering, learning, llm
Image
cari situs slots online paling beri keuntungan
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .