Introduction

Accessibility is a vital aspect of web development. One key component of web accessibility is providing alt text for images. Alt text (alternative text) is a textual description of an image that helps screen readers convey the image's content to users who cannot see it.

Thinking of the right words for alt text can be tricky, especially when you're unsure of what to call an object. In addition, describing multiple images can quickly become tedious. Amazon Rekognition is a powerful image analysis service that can help automate some of the process.

This guide will walk you through how to set up and use Rekognition to quickly generate alt text for your images, saving time and improving accessibility.

What is Amazon Rekognition?

Amazon Rekognition is a powerful machine learning service offered by AWS that can analyze images and videos to identify objects, scenes, activities, and people. It can also recognize text, faces, and other visual elements in images.

When you use Rekognition, it generates labels for an image, such as "Person," "Dog," or "Car," which describe the objects or scenes it detects. These labels can be a great starting point for alt text. However, it's important to remember that Rekognition is not a replacement for human judgment. It is essential to ensure that alt text is accurate, informative, and user-friendly.

Practical Example of When to Use Rekognition

Let’s say you manage an online store with hundreds of product images. Manually writing alt text for each product image can be time-consuming. Using Rekognition, you can automatically generate labels useful to write alt text for your product images in minutes, without having to spend a lot of time thinking about what to describe each image.

Prerequisites

Before you begin, ensure you have the following:

AWS Account.
AWS CLI.
IAM user with an access key ID and secret key set up
Basic Terminal Knowledge

Step-by-Step Guide: How to Generate Alt Text Using Rekognition

Step 1: Set Up AWS CLI

The first step is to set up your AWS credentials in the CLI.

Open your terminal and run aws configure.
Enter your AWS Access Key ID, AWS Secret Access Key, and the default region (e.g., us-east-1).

This sets up your AWS CLI with the necessary credentials to interact with AWS services.

Step 2: Prepare Your Image

You can use images stored locally on your computer or images stored in an AWS S3 bucket.

For local images: You need to either encode the image to base64 format or upload it to S3. For simplicity, we'll focus on using the base64 method here.
For images in an S3 bucket: You can refer directly to the image in the S3 bucket by providing the bucket name and image file name.

Step 3: Run the Rekognition Command

Once your AWS CLI is set up and your image is ready, you can use the Rekognition detect-labels command to generate labels.

For Local Images (Base64 Encoding):

Run the following command, replacing /path/to/your/image.jpg with the actual path to your image file:

aws rekognition detect-labels --image "Bytes=$(base64 /path/to/your/image.jpg | tr -d '\n')" --max-labels 10 --min-confidence 80

Here’s what each part does:

Bytes=$(base64 ...): Encodes the image to base64 so it can be passed to Rekognition.
--max-labels 10: Limits the result to 10 labels.
--min-confidence 80: Filters out labels with a confidence lower than 80%.

The command will return a JSON output with labels detected in the image. For example:

{
  "Labels": [
    {
      "Name": "Woman",
      "Confidence": 99.5
    },
    {
      "Name": "Baby",
      "Confidence": 95.1
    }
  ]
}

For Images in an S3 Bucket:

If your image is stored in an S3 bucket, use the following command, replacing <your-bucket-name> and <image-name> with your S3 bucket and image names:

aws rekognition detect-labels --image "S3Object={Bucket=<your-bucket-name>,Name=<image-name>}" --max-labels 10 --min-confidence 80

Upon following either of the steps above, you’ll receive a JSON response with detected labels. You can use these labels as the starting point of your alt text. In this case, the alt text could be: "A woman carrying a baby."

Step 4: Automate the Process (Optional)

If you have many images, you can automate the process to make it more efficient.

Batch Processing with S3: You can use AWS Lambda to automatically trigger Rekognition whenever an image is uploaded to your S3 bucket.
Using a Shell Script: You can also write a shell script to process multiple images in a folder.

For example:

#!/bin/bash
for image in /path/to/images/*.jpg; do
    aws rekognition detect-labels --image "Bytes=$(base64 $image | tr -d '\n')" --max-labels 10 --min-confidence 80
done

Considerations

While Rekognition is a powerful tool that can be helpful for generating alt text, it’s important to keep the following in mind:

Use Rekognition as a starting point: Rekognition can help you identify key objects and scenes in an image. It may not always generate the most precise labels for your images. It’s a good idea to review the generated labels and adjust them if necessary.
Context: The labels generated by Rekognition are generic, so you may want to adjust them to fit the specific context of your image.
Limitations: Rekognition generates labels based on visual content, but it doesn’t provide nuanced descriptions like a human might. You may need to fine-tune the alt text to provide a more detailed description when needed.

Conclusion

Rekognition can be a valuable tool for assisting with image alt text, but it should always be used as part of a human-centered workflow. By following these steps, you can automate the process of generating alt text for your images. This saves time, improves accessibility, and ensures that your website is more inclusive for all users.

In addition, make your alt text brief and to the point, and use a screen reader to test it, ensuring it is accessible.

How to Automatically Generate Alt Text for Images Using Amazon Rekognition