How to set up Amazon RDS Proxy for Amazon Aurora (Serverless) database cluster and connect AWS Lambda function to it

Vadym Kazulkin - Apr 22 - - Dev Community

Introduction

In this article we'd like to explore how to connect Lamdba function to the Aurora database through the RDS Proxy using Java JDBC capabilities. You can explore RDS Proxy capabilities in the Using Amazon RDS Proxy article. We will use Aurora Serverless v2 PostgreSQL database in this article but the same will also work Amazon Aurora PostgreSQL-Compatible Edition, Amazon Aurora MySQL-Compatible Edition, Amazon RDS for PostgreSQL, Amazon RDS for MySQL, Amazon RDS for MariaDB, and Amazon RDS for SQL Server as well. And it's easy to re-write Lambda function using different programming language as the basic concepts remain the same.

Solution

What we'd like to achieve is described in the following architecture diagram:

Image description

We'll focus on the part where Lambda connects through RDS Proxy to Aurora database. As the code sample we will use what we have created as part of the Data API for Amazon Aurora Serverless v2 with AWS SDK for Java series, and the project can be found on my GitHub repository. The relevant part of the Infrastructure as a Code can be found in this AWS SAM template.

First of all, we need to create VPC to put Lambda, RDS Proxy and Aurora Cluster to, so that Lambda is able to talk to RDS Proxy which in turn connects to Aurora Cluster. For that, please define your own VPC Id and Subnet list in the "Parameter" section of the SAM template like this :



  VpcId:
    Type: String
    Default: vpc-950cd6fd
    Description: VpcId of your existing Virtual Private Cloud (VPC)
  Subnets:
    Type: CommaDelimitedList  
    Default: subnet-0787be4d, subnet-88dc46e0
    Description: The list of SubnetIds, for at least two Availability Zones in the
      region in your Virtual Private Cloud (VPC)
Enter fullscreen mode Exit fullscreen mode

Then we need to define Lambda Security Group which references the VPC Id:



  LambdaSecurityGroup:
      Type: AWS::EC2::SecurityGroup
      Properties:
        GroupDescription: SecurityGroup for Serverless Functions
        VpcId: 
          Ref: VpcId
Enter fullscreen mode Exit fullscreen mode

and then VPC Security Group itself:



  VPCSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for RDS DB Instance.
      VpcId: 
         Ref: VpcId 
      SecurityGroupEgress: 
        - CidrIp: '0.0.0.0/0'
          Description: lambda RDS access over 5432
          FromPort: 5432
          IpProtocol: TCP
          ToPort: 5432
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: '5432'
          ToPort: '5432'
          SourceSecurityGroupId:
               Ref: LambdaSecurityGroup
Enter fullscreen mode Exit fullscreen mode

VPC Security Group defines Security Group engress and igress roules which enable Lambda to talk to RDS Proxy via the port number 5432 (default port for PostgreSQL) and references the defined Lambda Security Group.

Now let's create Aurora Serverless v2 PostgreSQL database cluster. But before the let's create SecretsManager to store the database user and password:



  DBSecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Ref UserSecret
      Description: RDS database auto-generated user password
      GenerateSecretString:
        SecretStringTemplate: !Sub '{"username": "${DBMasterUserName}"}'
        GenerateStringKey: "password"
        PasswordLength: 30
        ExcludeCharacters: '"@/\'
Enter fullscreen mode Exit fullscreen mode

Then we need to define DB Subnet Group:



  DBSubnetGroup:
    Type: AWS::RDS::DBSubnetGroup
    Properties:
      DBSubnetGroupDescription: Subnets available for the RDS DB Instance
      SubnetIds:
       Ref: Subnets
Enter fullscreen mode Exit fullscreen mode

and then create the Aurora Cluster with the engine "aurora-postgresql" itself:



 AuroraServerlessV2Cluster:
    Type: 'AWS::RDS::DBCluster'
    DeletionPolicy: Delete
    Properties:
      DBClusterIdentifier: !Ref DBClusterName
      Engine: aurora-postgresql
      Port: 5432
      EnableHttpEndpoint: true
      MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:username}}' ]]
      MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:password}}' ]]
      DatabaseName: !Ref DatabaseName
      ServerlessV2ScalingConfiguration:
        MinCapacity: 0.5
        MaxCapacity: 1
      DBSubnetGroupName:
        Ref: DBSubnetGroup
      VpcSecurityGroupIds:
      - !Ref VPCSecurityGroup
Enter fullscreen mode Exit fullscreen mode

In the last lines we reference already created DB Subnet Group and Vpc Security Group Ids. With ServerlessV2ScalingConfiguration we define the scaling behaviour of our Aurora Serverless V2 Cluster.

After it we need to create database instance of the Aurora (Serverless v2) cluster :



  AuroraServerlessV2Instance:
    Type: 'AWS::RDS::DBInstance'
    Properties:
      Engine: aurora-postgresql
      DBInstanceClass: db.serverless
      DBClusterIdentifier: !Ref AuroraServerlessV2Cluster
      MonitoringInterval: 1
      MonitoringRoleArn: !GetAtt EnhancedMonitoringRole.Arn
      PubliclyAccessible: false
      EnablePerformanceInsights: true
      PerformanceInsightsRetentionPeriod: 7
Enter fullscreen mode Exit fullscreen mode

Now let's take care of the creation of the RDS Proxy itself :



  RDSProxy:
    Type: AWS::RDS::DBProxy
    Properties:
      Auth:
        - { AuthScheme: SECRETS, SecretArn: !Ref DBSecret}
      DBProxyName: 'rds-proxy'
      RoleArn: !GetAtt RDSProxyRole.Arn
      EngineFamily: 'POSTGRESQL'
      IdleClientTimeout: 120
      RequireTLS: true
      DebugLogging: false
      VpcSecurityGroupIds:
      - !Ref VPCSecurityGroup
      VpcSubnetIds: !Ref  Subnets
Enter fullscreen mode Exit fullscreen mode

We use EngineFamily 'POSTGRESQL' and AuthScheme connected to the created SecretsManager secret for authentication in our scenario. There are also other authentication possibilities like IAM authentication offered. We also connect RDS Proxy to the already created Vpc Security Group and Vpc Subnet Id.

We also need to create RDS Proxy Role to allow RDS Proxy to fetch secrets from the SecretsManager:



  RDSProxyRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Action: [ 'sts:AssumeRole' ]
            Effect: Allow
            Principal:
              Service: [ rds.amazonaws.com ]
      Policies:
        - PolicyName: DBProxyPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                  - secretsmanager:GetSecretValue
                Effect: Allow
                Resource:
                  - !Ref DBSecret
Enter fullscreen mode Exit fullscreen mode

As the final step we need to create DB Proxy Target Group:



  ProxyTargetGroup:
    Type: AWS::RDS::DBProxyTargetGroup
    Properties:
      DBProxyName: !Ref RDSProxy
      DBClusterIdentifiers: [ !Ref AuroraServerlessV2Cluster]
      TargetGroupName: default
      ConnectionPoolConfigurationInfo:
        MaxConnectionsPercent: 5
        MaxIdleConnectionsPercent: 4
        ConnectionBorrowTimeout: 120
Enter fullscreen mode Exit fullscreen mode

DB Proxy Target Group connects RDS Proxy with the Aurora V2 Cluster and defines Connection Pool Configuration. Please refer to Configure Connection Settings documentation for the extended explanation of this configuration.

Now let's connect Lambda function "GetProductByIdViaAuroraServerlessV2WithRDSProxy" to the RDS Proxy.
The relevant part is this one:



      Policies:
        - Statement:
            - Sid: AllowDbConnect
              Effect: Allow
              Action:
                - rds-db:connect
              Resource:
                - !Sub arn:aws:rds-db:${AWS::Region}:${AWS::AccountId}:dbuser:!Select [6, !Split [":", !GetAtt RDSProxy.DBProxyArn]]/*

        - VPCAccessPolicy: {}
      VpcConfig:
        SecurityGroupIds:
          - Fn::GetAtt: LambdaSecurityGroup.GroupId
        SubnetIds: !Ref  Subnets
Enter fullscreen mode Exit fullscreen mode

In the VPCAccessPolicy part we reference already created Security Groups and Subnets. In the Policies part we allow Lambda to connect to our RDS Proxy instance by defining the exact resource ARN by knowing how the resource schema name works but extracting the appropriate RDSProxy ARN part from it. For the exact explanation please visit Creating and using an IAM policy for IAM database access.

The word of caution: only for the demonstration purpose I passed the database name and password as the Lambda environment variables to connect to RDS Proxy which introduces the security risk. 



      Environment:
        Variables:
          DB_USER_PASSWORD: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:password}}' ]]
          DB_USER_NAME: !Join ['', ['{{resolve:secretsmanager:', !Ref DBSecret, ':SecretString:username}}' ]]
          RDS_PROXY_ENDPOINT: !GetAtt RDSProxy.Endpoint
Enter fullscreen mode Exit fullscreen mode

The proper solution is to use the stored database name and password in Amazon Secret Manager and then retrieve the in the Lambda function itself.

That it on the Infrastructure as a Code level.

Everything else will be done in the Lambda handler itself to connect to the RDS Proxy. In my case I wrote GetProductByIdViaAuroraServerlessV2RDSProxyHandler in Java and used JDBC for the connection with the PostgreSQL database driver in the pom.xml like this.



     <dependency>
       <groupId>org.postgresql</groupId>
       <artifactId>postgresql</artifactId>
       <version>42.5.4</version>
    </dependency>
Enter fullscreen mode Exit fullscreen mode

Each programming languge in which Lambda function can be implemented offers its own capabilities to connect to the database.

Conclusion

In this article we explored how to connect Lamdba function to the Aurora database through the RDS Proxy and demonstrated the example of the corresponding Infrastructure as a Code part and the Lambda function written in Java which uses JDBC to connect to RDS Proxy.

Image
OOP - JavaScript Challenges

webdev, javascript, beginners, learning
Image
Django Request/Response Life Cycle

python, django, softwaredevelopment, webdev
Image
A Step-By Step Guide For Choosing Your Self Propelled Wheelchair Uk
Image
20 Things You Need To Know About Asbestos Lawsuit Settlement
Image
5 Reasons To Be An Online Asbestos Cancer Lawsuit Lawyer Mesothelioma And 5 Reasons You Shouldn't
Image
10 Things Everybody Hates About Asbestos Cancer Lawsuit
Image
📜 Mastering Code Quality in ReadCraft: Formatting, Linting, and Automation Adventure 🚀

opensource, flake8, python, black
Image
Five Tools That Everyone Within The Built In Electric Fireplace Wall Industry Should Be Making Use Of
Image
Is Ruby on Rails Worth Learning in 2024? A Quick Insight

ruby, rails, programming
Image
The Most Convincing Evidence That You Need Good Accident Lawyers
Image
The Most Prevalent Issues In Average Settlement For Asbestos Exposure
Image
ASIA PASCIFIC 3D PRINTING IN CONSTRUCTION: A Game-Changer for the Industry
Image
How to Train Your Cat to Use a Cat Flap
Image
Gevelreiniging Zaandam? Gevelrenovatie Prijzen 2024!
Image
It's The Next Big Thing In Mental Health Psychiatrist Near Me
Image
15 Interesting Hobbies That Will Make You Better At Accident Lawyer Dallas
Image
10 Misconceptions Your Boss Shares About Asbestos Attorney Mesothelioma
Image
Menembus Batas: Strategi Jitu Judi Bola untuk Menang Besar
Image
Iterable - JavaScript Challenges

webdev, javascript, beginners, learning
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .