John Vester
Finding your niche in the technology landscape is exciting and fulfilling at the same time. When your application or service is made available to customers, you feel that same thrill of completing some level of education, winning a game, or producing some artistic gem for others to enjoy.
But what happens when customers love your creation too much?
In the video link above, the product provided by Vertabiz became an instant success. At first this was great! But seconds later, they realized they lacked the capacity to meet customer demand.
While the commercial in the link above was related to shipping and logistics, the same problem exists at the service level. Knowing when to scale up (and to scale down) to stay ahead of demand not only saves the customer experience, but can yield financial rewards as well.
In this article, I am going to explore scaling options within the Heroku ecosystem.
How Do I Know When I Need To Scale?
In the Vertabiz video (above), the team realized a need to scale up their capacity when they saw the number of orders exceed their expectations. For a smooth customer experience with web applications, capacity should be added before existing services reach 100% utilization.
The concept of knowing "when to scale" is just as important as building an architecture that is scalable. After all, scaling up too quickly will consume more costs than required and scaling up after the fact will impact performance and potentially lose customers. The same costly scenario exists when demand decreases and there is no need to employ additional capacity.
Heroku provides the following options to help gain insight on knowing when capacity needs to change for your application or service:
- referencing the Metrics tab in the Heroku Dashboard provides a graphical representation of historical demand for the application, which can justify changes to the current capacity
- using the loader.io add-on, which can help understand where additional capacity is required by your application
- employing a Monitor add-on from the Heroku product marketplace to keep an eye on performance
Scaling Vertically vs Horizontally
Depending on the design of the application, it is possible to scale two different ways to meet consumer demand:
- Vertical scaling - increase to a larger Dyno type. Think of this as upgrading to a more powerful system to process requests.
- Horizontal scaling - increases the number of Dynos for the same Dyno type.
In most cases, simply adding instances horizontally does not fix the problem; that is, unless the application architecture is prepared for that scenario as noted in the "processes" aspect of the Twelve-Factor App. This is where load balancers or API gateways are often introduced.
With Heroku, routing of requests is part of the ecosystem — just another way Heroku saves time and resources. You can read more about this here:
Request Distribution with Heroku
Manually Scaling with Heroku
The manual approach to auto-scaling with Heroku simply means someone is going to utilize the Heroku dashboard or the Heroku command-line interface (CLI) to address a demand-related need for a given application or service.
Using Heroku Web Dashboard
In the example above, if I wanted to increase the processing power of a Dyno configuration (scaling vertically) for one of my services running in Heroku, the following steps would be utilized:
- Navigate to the Resources tab on the Heroku dashboard:
- Single-click the Change Dyno Type button and make the appropriate selection.
- Follow any remaining instructions on the screen.
- Save the request.
- Heroku will take care of the rest for you.
If I wanted to scale my existing Heroku (Professional) application horizontally using the same user interface, the pencil icon in step number one would be selected. The ensuing screen would allow the number of Dynos to be increased as needed.
Using Heroku CLI
Using the CLI is another option to manually scale Heroku applications or services. However, this choice does rely on your knowledge of what the desired result will be.
As an example, to increase the number of Dyno instances, the following CLI can be executed:
heroku ps:scale web=3
To change the Dyno type, the following CLI can be executed:
heroku ps:type worker=standard-2x
Keep in mind that changes to the Dyno type do not alter the number of instances.
Of course, savvy DevOps engineers can use external processes to execute the Heroku CLI to automate Dyno scaling in Heroku by having a scheduled process execute CLI commands.
Autoscaling with Heroku
If you are running Heroku at the Performance-tier or have Dynos running in Private Spaces (dedicated environments running on an isolated network), autoscaling is built into the level of service provided by Heroku.
For those situations, the Resources tab in Heroku Desktop includes an Enable Autoscaling button, which introduces the following dialog:
Since this feature is limited to more advanced tiers of the Heroku ecosystem, the following link provides excellent documentation on this feature:
Other Autoscaling Options
For applications running on other tiers in Heroku, there are several add-on products available in the Heroku Marketplace:
I have worked with a few of these add-ons, which are broken down into the following autoscaling categories:
- Scheduled mode - scheduled based autoscaling
- Smart mode - autoscaling based on request/response time
Scheduled Autoscaling Mode
In the scheduled mode of autoscaling, changes to the number of Dynos running in Heroku are known to the engineering team supporting the application or service. There are two subcategories of the approach:
- User Interface - add-on leverages a dedicated UI for scaling Dynos
- Scheduler - add-on utilizes CLI to achieve auto-scaling
User Interface
The user interface subcategory contains Heroku Marketplace add-ons which are focused at setting schedules for applications or services running in Heroku. Those options include a purpose-driven user interface to make things easier for the end-user.
The following add-ons allow for Dynos to be autoscaled on a scheduled basis:
It is important to note that AutoIdle is a "scale down to zero" option only and is recommended for non-production environments. The primary goal is to save money for non-production Dynos that are not currently in use.
Scheduler
The scheduler-based add-ons can be employed to pass in CLI commands and can be used to scale up and down as needed.
This approach leverages the following Heroku CLI commands (noted in the Using Heroku CLI section above):
heroku ps:scale web=3
heroku ps:type worker=standard-2x
The following add-on products allow for CLI commands to be executed on a scheduled basis:
These solutions often lean on the basic Cron functionality running on underlying systems in the Heroku ecosystem.
Smart Autoscaling Mode
The limitation of the Scheduled Autoscaling Mode is that the demand pattern for the application or service is fully known. Meaning, the higher and lower demand levels can be predicted. This may not always be known — the Vertabiz video in the introduction is an example scenario.
The smart mode of autoscaling alters the number of Dynos based upon the current request/response time being realized in your application or service. The following add-on products offer this functionality:
- Adept Scale
- Rails Autoscale (limited to Ruby only)
Using the Smart autoscaling option ensures unexpected changes in demand will be handled without any human intervention. This is helpful in those scenarios where demand exceeds any expected levels from the options noted in this document thus far.
Conclusion
In my publications, I have been referring to the following mission statement I proffer all IT professionals:
“Focus your time on delivering features/functionality which extends the value of your intellectual property. Leverage frameworks, products, and services for everything else.”
- J. Vester
The need to leverage autoscaling solutions is a requirement when your application or service reaches the point where capacity needs to be scaled up and down to maintain positive consumer satisfaction levels. In my 30+ years in the IT industry, I have witnessed how wonderful solutions lose their appeal because of negative performance realities.
Currently, the following instance-count ranges exist for each Dyno type in Heroku:
While autoscaling incurs a cost, properly implemented levels will net an overall savings from manual attempts at scaling your application or service. Using the options above, the probability of losing customers along the way is greatly reduced.
Have a really great day!
