The Issue

Last week was a busy week with many changes that I can't talk about. Looking back, it's been over a week since I wrote anything. What's going on? I just optimized the images for my blog! As you know, statistics show that images account for a large amount of data transmitted over the Internet when accessing websites. Images convey a lot of messages, making them more vivid and visually appealing than plain text. Images even stimulate curiosity and encourage users to click on them.

If you've been a long-time reader of 2coffee.dev, you'll notice that the homepage used to consist of only text, with a headline and a snippet of the article content. Putting them together in a list. In the modern age, having an image representing the article is not uncommon. Combining images and headlines will pique the reader's curiosity more than just text alone. Recently, I upgraded to add the ability to display images next to each article. The homepage is now less "boring" than before.

In the beginning, finding a place to store images was quite difficult. I once set up a simple image server using Node.js, but it was a burden to maintain. There were many things to consider, and I didn't want to spend too much time on that. By chance, I found the website imgur.com - which allowed me to upload images without many restrictions, and at that time, the image loading speed from imgur was quite good, so using it as a "host" was not too much of a concern.

The uploaded images had URLs like https://i.imgur.com/Z4ERHfq.jpg, and I enclosed them in the ![] markdown tag to display the images. Everything seemed to work well, and I was satisfied with this solution for a long time. But in reality, imgur is not a CDN, and they have also clarified that they are not a CDN service, so sometimes, the image loading speed from imgur is unstable. Specifically, there are times when there are interruptions or users from different regions take longer to load, and the "loading" icon keeps spinning in the browser, making the speed experience very poor.

I constantly searched for free CDN service providers. There were a few prominent names like Cloudimage, Imgix, Bunny... but they came with many limitations in terms of storage and features, and in another aspect, they were more difficult to use or more complex than what I needed.

A few months ago, when I learned about Cloudflare's R2 service - a type of Object Storage similar to Amazon S3 that allows you to use 10GB of storage for free with unlimited bandwidth. The limit only lies in the number of requests, for example, you will have 1 million free requests to R2 within a month, a number that is neither large nor small, but certainly 1 million is more than enough for this blog.

Immediately, I wrote some code to use R2 as my new image storage. The image loading speed from R2 is relatively fast. Because R2 is part of Cloudflare, and Cloudflare also offers a CDN service for images, the speed from R2 is likely to be one step behind the CDN, but it is definitely faster than imgur. I also wrote a CLI to facilitate image uploading, just run a command, the image is uploaded, the image URL is returned and copied to the clipboard.

At this point, more challenges started to arise. Although the optimized and compressed images uploaded to R2 aimed to reduce the file size while maintaining good display quality, the old images from imgur did not go through the same optimization process. All links to the external thumbnail images were the original images from the banners in the articles, so if accessed for the first time, it's not surprising that the total data size of the entire website could reach several hundred MB. This is a number that greatly affects the speed, especially for those with slow or unstable internet connections or during peak hours...

Doing nothing is not an option, but once I start, I have to face the headaches. This is the nth time I have optimized the images for this website, the optimization process did not happen immediately but rather sporadically throughout the operation, but in summary, there are two major milestones that I will continue to tell you about.

First Image Display Optimization

When the image size is large and the network speed is slow, the area containing the image will display a dark mark due to setting a black background, which doesn't look very appealing.

There are many techniques to solve this problem, for example, creating a loading spinner to indicate that the image is being loaded, using a small-sized image called a placeholder to "reserve space"... and the method I chose is to use a placeholder image.

This image needs to be as small as possible, and it's creative to choose the logo itself, just place it on an image with a background color that matches the logo, and immediately there is an image to use, the size of the image at this point is only a few KB. The first time it's loaded, the placeholder image will be displayed first, waiting for the original image to finish loading, then the lazysizes library will automatically replace the old image with the new image.

As for the tool to upload images to R2, I cleverly used the @squoosh/cli library to convert images to the compressed webp format, while reducing the image quality to 80% to save space while still maintaining good display quality. The remaining task is to insert the image links into the article. At this point, I was almost confident in my ability to display images.

Second Image Display Optimization

Although the low-quality placeholder image helped to improve the smoothness of image display to some extent, users still had to download a large amount of image data from the first load. In some ways, this is quite wasteful.

Each article has a large representative image called a banner. This image contributes to conveying messages, making the article more vivid, and has a few other benefits. If you use Chrome, on the browser's homepage, in a new browser window, scroll down and you will see a list of suggested articles that Google recommends based on the content you have recently accessed. As they introduce in the article Discover and your website - Search Central, there is no manual way to get your article on this list, it is completely automatic based on Google's suggestions, one of which is that the article needs to have a representative image with a minimum size of 1200x900 pixels.

Typically, an image that meets this standard, when optimized at 75% quality with the webp format, will be around 100KB. Roughly, if there are 6 articles on the homepage, the total data size to be loaded is 600KB, a number that I consider quite "wasteful".

If you pay close attention, you will notice that the thumbnail images outside the homepage are much smaller than the original size inside the article details. Therefore, if the thumbnail image size can be adjusted to the necessary level, a large amount of data can be saved. Although the purpose is to reduce the download size, display quality and user experience still need to be considered, no one wants to see a blurry and patched image. At this point, it is necessary to strike a balance between file size and display quality.

The idea at that time was to resize the banner image in the article to a more fitting size within the thumbnail image display area on the homepage. If the banner size requires 1200x900, then the thumbnail only needs to be 700x525, and if it is possible to replace the previous placeholder image, which was the website logo, with a low-quality image of the original image, it will bring a new experience that is much more reasonable.

blur is a technique to blur images while reducing the file size surprisingly. blur works by enlarging the pixels of the image, which reduces the details of the image, and the number of colors also decreases, thus saving storage space. Sharp is a popular image processing library in Node.js, and it supports the blur function. After going through the blur function, the image size at this point is only a few KB, which is reasonable for an image placeholder in the article.

Another issue that needs to be addressed is that not all articles have banner images because, as mentioned earlier, the use of images was not determined. To create a placeholder, I needed to add banners to all missing articles. But I couldn't sit down and manually create images for over 170 previous articles, and if you count the English version as well, the number c ould exceed 340 articles, which presents a significant time challenge.

It was time for an automated solution. After a few hours of thinking, I came up with the "hardship first, happiness later" approach - that is, to rewrite all the articles.

The initial idea was that if I could save all the original images in all the articles, classify them into folders that match the article's URL (because each article has a unique URL). Then the banner images would have the form article-url, and other images in the article would have similar forms, for example, article-url_image_1, article-url_image_2... No need to worry about what format the original images are because after creating an image formatting tool, all of them will be converted to the .webp extension.

This image formatting tool is quite simple, using the Sharp library. When given an input image, it will produce 4 output images. Among them, 2 images have a size of 1200x900 to create the blur placeholder and the banner image, and 2 smaller images are used for display outside the homepage. Then it continues to upload the 4 newly created images to R2 and returns the links to each image.

Storing the original images is also an advantage, assuming that in the future, I come up with a better image optimization method, I can write a new tool to process images and produce different images with unchanged URLs, minimizing changes in the articles. The next challenge was to spend time rewriting over 300 articles. Replace the previous image links with the new format, make some code changes, and voila... we have a new image loading effect that you are experiencing.

Conclusion

Optimization is a long-term problem that requires time and resources. There are many ways to solve a problem, for example, in my case, I chose to modify the links to the images in over 300 articles, along with creating smaller-sized images as placeholders to speed up page loading and enhance the user experience. So if you were in my shoes, do you have a better way? Leave a comment below the article!