API Gateway and Lambda Throttling with Terraform. Part 2

Alexey Vidanov - Oct 14 - - Dev Community

In the previous post we covered the basics of setting up throttling for your API Gateway and Lambda functions. In this follow-up, we’ll take it to the next level, adding budget controls, time-based throttling adjustments, and AWS WAF security integration to safeguard your API while optimizing both performance and cost-efficiency.

1. Budget and Billing Alerts

Managing cloud expenses is essential to avoiding surprises, especially when usage spikes unexpectedly. AWS Budgets allows you to set up monthly spending limits and receive alerts before overspending occurs.

Example: Setting up Budget Alerts 

resource "aws_budgets_budget" "monthly_budget" {
  name         = "monthly-budget-${var.environment}"
  budget_type  = "COST"
  time_unit    = "MONTHLY"
  limit_amount = "1000"
  limit_unit   = "USD"

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 80
    threshold_type             = "PERCENTAGE"
    notification_type          = "FORECASTED"
    subscriber_email_addresses = ["alert@example.com"]
  }

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 100
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = ["alert@example.com"]
  }
}
Enter fullscreen mode Exit fullscreen mode

You can also configure alerts to trigger responses in other AWS services, such as SNS topics or CloudWatch Alarms, to better manage your Lambda functions during traffic spikes.

# SNS Topic for Alerts
resource "aws_sns_topic" "budget_alerts_topic" {
  name = "budget-alerts-topic-${var.environment}"
}

# Subscribe an email address to the SNS topic
resource "aws_sns_topic_subscription" "email_subscription" {
  topic_arn = aws_sns_topic.alarms.arn
  protocol  = "email"
  endpoint  = var.alarms_email
}

# Example CloudWatch Alarm for Lambda Throttling (to control costs)
resource "aws_cloudwatch_metric_alarm" "lambda_throttles" {
  alarm_name          = "lambda-throttles-${aws_lambda_function.example.function_name}"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 2
  metric_name         = "Throttles"
  namespace           = "AWS/Lambda"
  period              = 300
  statistic           = "Sum"
  threshold           = 5
  alarm_description   = "Alert for Lambda throttling due to budget constraints"
  alarm_actions       = [aws_sns_topic.budget_alerts_topic.arn]

  dimensions = {
    FunctionName = aws_lambda_function.example.function_name
  }
}
Enter fullscreen mode Exit fullscreen mode

2. Time-Based Throttling Adjustments

For high-traffic applications, one-size-fits-all throttling can be inefficient. You can dynamically adjust API Gateway throttling limits during peak and off-peak hours using AWS EventBridge and Lambda.

Example: Time-Based Throttling with Lambda and EventBridge

Below is the Python code that dynamically adjusts API Gateway throttling limits:

import boto3
import os

def handler(event, context):
    api_gateway = boto3.client('apigateway')

    api_id = os.environ['API_ID']
    stage = os.environ['STAGE']
    burst_limit = int(os.environ['BURST_LIMIT'])
    rate_limit = int(os.environ['RATE_LIMIT'])

    # Update method settings with new throttling limits
    api_gateway.update_stage(
        restApiId=api_id,
        stageName=stage,
        patchOperations=[
            {
                'op': 'replace',
                'path': '/*/*/throttling/burstLimit',
                'value': str(burst_limit)
            },
            {
                'op': 'replace',
                'path': '/*/*/throttling/rateLimit',
                'value': str(rate_limit)
            }
        ]
    )

    return {
        'statusCode': 200,
        'body': f'Throttling updated: Burst={burst_limit}, Rate={rate_limit}'
    }
Enter fullscreen mode Exit fullscreen mode

Lambda Creation in Terraform

To automate the creation of the Lambda function for adjusting throttling limits, use the following Terraform configuration:

variable "api_throttling_configs" {
  type = map(object({
    burst_limit = number
    rate_limit  = number
  }))
  default = {
    peak = {
      burst_limit = 5000
      rate_limit  = 2000
    }
    off_peak = {
      burst_limit = 1000
      rate_limit  = 500
    }
  }
}


resource "aws_lambda_function" "adjust_throttling_peak" {
  function_name = "adjust_throttling_peak_${var.environment}"
  handler       = "adjust_throttling.handler"
  runtime       = "python3.11"
  role          = aws_iam_role.lambda_exec.arn
  filename      = "adjust_throttling_peak.zip"

  environment {
    variables = {
      API_ID      = aws_api_gateway_rest_api.example.id
      STAGE       = aws_api_gateway_stage.example.stage_name
      BURST_LIMIT = var.api_throttling_configs.peak.burst_limit
      RATE_LIMIT  = var.api_throttling_configs.peak.rate_limit
    }
  }
}

resource "aws_lambda_function" "adjust_throttling_off_peak" {
  function_name = "adjust_throttling_off_peak_${var.environment}"
  handler       = "adjust_throttling.handler"
  runtime       = "python3.11"
  role          = aws_iam_role.lambda_exec.arn
  filename      = "adjust_throttling_off_peak.zip"

  environment {
    variables = {
      API_ID      = aws_api_gateway_rest_api.example.id
      STAGE       = aws_api_gateway_stage.example.stage_name
      BURST_LIMIT = var.api_throttling_configs.off_peak.burst_limit
      RATE_LIMIT  = var.api_throttling_configs.off_peak.rate_limit
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Here's a more understandable version:

Automating Time-Based Adjustments

To automatically adjust settings based on the time of day (for example, peak hours vs. off-peak hours), you can use EventBridge Rules to trigger Lambda functions at specific times. This way, your system can automatically adjust to handle more traffic during peak times and reduce capacity during off-peak hours.

In this example, we set up rules that run at 8:00 AM (peak hours) and 8:00 PM (off-peak hours). Here's how it works:

  • Peak Hours Rule: This triggers at 8:00 AM to increase capacity during busy times.
  • Off-Peak Hours Rule: This triggers at 8:00 PM to reduce capacity during slower times.

Here’s how to set it up:

# Define a rule for peak hours (8:00 AM)
resource "aws_cloudwatch_event_rule" "peak_hours_rule" {
  name                = "peak-hours-rule"
  schedule_expression = "cron(0 8 * * ? *)"  # Adjust to your peak hours
}

# Define a rule for off-peak hours (8:00 PM)
resource "aws_cloudwatch_event_rule" "off_peak_hours_rule" {
  name                = "off-peak-hours-rule"
  schedule_expression = "cron(0 20 * * ? *)"  # Adjust to your off-peak hours
}

# Set the target for peak hours (a Lambda function to handle peak traffic)
resource "aws_cloudwatch_event_target" "peak_hours_target" {
  rule      = aws_cloudwatch_event_rule.peak_hours_rule.name
  target_id = "adjust_throttling_peak"
  arn       = aws_lambda_function.adjust_throttling_peak.arn
}

# Set the target for off-peak hours (a Lambda function to handle lower traffic)
resource "aws_cloudwatch_event_target" "off_peak_hours_target" {
  rule      = aws_cloudwatch_event_rule.off_peak_hours_rule.name
  target_id = "adjust_throttling_off_peak"
  arn       = aws_lambda_function.adjust_throttling_off_peak.arn
}
Enter fullscreen mode Exit fullscreen mode

This setup automatically changes your system’s capacity based on time. At 8:00 AM, the system prepares for heavy traffic, and at 8:00 PM, it scales down for lighter usage.

3. Adding AWS WAF for Extra Protection

Integrating AWS WAF (Web Application Firewall) into your API Gateway helps protect it from malicious traffic, such as bots or potential DDoS (Distributed Denial of Service) attacks. It acts as a security layer that filters out harmful requests before they reach your API.

Example: Blocking IPs and Controlling Bots with WAF

This example shows how to set up a WAF rule that protects your API Gateway by blocking bot traffic:

resource "aws_wafv2_web_acl" "api_gateway_acl" {
  name  = "api-gateway-waf-acl"
  scope = "REGIONAL"  # Use "REGIONAL" to protect your regional API Gateway

  # Default action to allow all requests unless a rule blocks them
  default_action {
    allow {}
  }

  # Enable visibility so you can monitor activity using CloudWatch metrics
  visibility_config {
    cloudwatch_metrics_enabled = true
    metric_name                = "apiGatewayAcl"
    sampled_requests_enabled   = true
  }

  # Bot Protection Rule to block bad bot traffic
  rule {
    name     = "BotProtectionRule"  # Name of the rule
    priority = 2  # Order of execution for this rule (lower number = higher priority)

    # No override action means the default allow/block settings apply
    override_action {
      none {}
    }

    # Managed rule group to use AWS's pre-built bot control rules
    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesBotControlRuleSet"
        vendor_name = "AWS"  # AWS provides these bot protection rules
      }
    }

    # Enable visibility for this specific rule as well
    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "botProtection"
      sampled_requests_enabled   = true
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

This setup adds protection to your API by detecting and blocking bot traffic, while also giving you the ability to monitor suspicious activity via CloudWatch metrics.

Conclusion

By incorporating Budget Alerts, Time-Based Throttling Adjustments, and WAF Integration, you're ensuring that your API is not only efficient and cost-effective but also secure. These advanced practices complement each other, providing a well-rounded solution for managing traffic, costs, and security in a scalable AWS environment.

Next Steps

  1. Monitoring and Fine-Tuning: After deploying these configurations, it's crucial to monitor their performance. Use CloudWatch metrics and logs to verify that throttling adjustments and security rules are working as expected.

  2. Performance Testing: Perform load testing during both peak and off-peak hours to ensure your dynamic throttling setup aligns with your traffic patterns.

  3. Further Optimization: Explore other AWS services like AWS Shield for advanced DDoS protection and AWS X-Ray for tracing API Gateway requests to identify potential bottlenecks.

By continually monitoring and adjusting, you’ll be able to maintain an API infrastructure that scales with traffic, optimizes costs, and maintains high levels of security.

Image
10 Things You Learned In Kindergarden That Will Aid You In Obtaining Accident Lawyer Boston
Image
The Three Greatest Moments In Asbestos Lawsuit Settlement Amount History
Image
5 Things That Everyone Is Misinformed About About Gas Boiler Engineer Near Me
Image
5 Qualities That People Are Looking For In Every Accident Lawyers
Image
Use Dishwasher Repair Services in North Miami Beach

dishwasher, repair, services, dryer
Image
Dog dental snacks are an excellent way
Image
Pusat bantuan layanan bank woori saudara +62 812 1411 9209

webdev, ai, api, database
Image
upvc Door Panels With Cat Flap
Image
Where to buy Flash Btc

wheretobuyflashbtc, flashbtc, flashusdt, flashbtcgeneratorsoftware
Image
The Sage Advice On Personal Injury Lawyer From A Five-Year-Old
Image
10 Facts About Gas Safe Registered Engineer Near Me That Will Instantly Put You In The Best Mood
Image
15 Terms That Everyone In The Lawyer Injury Accident Industry Should Know
Image
15 Unquestionably Reasons To Love Asbestos Disease Mesothelioma
Image
10 Undeniable Reasons People Hate Lawyers For Accidents At Work
Image
Gas Central Heating Engineers Strategies From The Top In The Industry
Image
10 Accident Lawyers In My Area Meetups You Should Attend
Image
10 Places To Find Coffee Filter Maker
Image
Wall Electric Fireplace Insert Explained In Fewer Than 140 Characters
Image
How To Become A Prosperous Electric Fireplace With Wall Mount If You're Not Business-Savvy
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .