Vadym KazulkinIntroduction
In the part 13 of our series we measured the warm starts of the Lambda function with Corretto Java 21 runtime without SnapStart enabled, with SnapStart enabled and also applied DynamoDB invocation priming optimization with different memory settings.
In this article we'll also provide measurements for the warm execution times for this use case for Java 17 (and compare them with Java 21), as using different Lambda memory settings not only affect the cold start times but also the warm execution times and Lambda costs. I'll also refer to and build on the measurements in my previous article Measuring cold starts and deployment time with Java 17 using different Lambda memory settings.
Measuring warm starts with Java 17 with and without SnapStart enabled using different Lambda memory settings
In our experiment we'll re-use the application introduced in part 8 for this. Here is the code for the sample application. There are basically 2 Lambda functions which both respond to the API Gateway requests and retrieve product by id received from the API Gateway from DynamoDB. One Lambda function GetProductByIdWithPureJava17Lambda can be used with and without SnapStart and the second one GetProductByIdWithPureJava17LambdaAndPriming uses SnapStart and DynamoDB request invocation priming. We'll measure cold and warm starts using the following memory settings in MBs : 256, 512, 768, 1024, 1536 and 2048.
I also put the cold starts measured in the part 19 into the tables to see both cold and warm starts in one place. The results of the experiment below were based on reproducing more than 100 cold and approximately 100.000 warm starts for the duration of our experiment which ran for approximately 1 hour. Here is the code for the sample application. For it (and experiments from my previous article) I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman.
Abbreviation c is for the cold start and w is for the warm start.
Cold (c) and warm (w) start times without SnapStart in ms:
| RAM | c p50 | c p75 | c p90 | c p99 | c p99.9 | c max | w p50 | w p75 | w p90 | w p99 | w p99.9 | w max |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 256 MB | 7309.66 | 7432.6 | 7575.51 | 7662.51 | 7782.25 | 7962.01 | 7.16 | 15.58 | 27.16 | 73.82 | 5364.09 | 5914.32 |
| 512 MB | 4213.37 | 4256.07 | 4325.17 | 4496.15 | 4661.23 | 4786.71 | 6.01 | 6.93 | 9.99 | 33.92 | 81.20 | 2731.29 |
| 768 MB | 3310.08 | 3414.9 | 3551.82 | 4271.48 | 4421.09 | 4594.42 | 6.21 | 7.27 | 9.38 | 26.31 | 1530.71 | 1833.73 |
| 1024 MB | 2880.53 | 2918.79 | 2974.45 | 3337.29 | 3515.86 | 3651.65 | 6.11 | 7.05 | 8.94 | 23.54 | 62.99 | 1272.96 |
| 1536 MB | 2390.15 | 2434.52 | 2464.46 | 2668.95 | 2812.15 | 2987.04 | 6.01 | 6.83 | 8.39 | 20.09 | 51.24 | 839.92 |
| 2048 MB | 2198.02 | 2272.5 | 2397.97 | 2757.06 | 2892.65 | 3005.31 | 6.61 | 7.87 | 9.99 | 24.69 | 600.02 | 895.15 |
Cold (c) and warm (w) start times with SnapStart without Priming in ms:
| RAM | c p50 | c p75 | c p90 | c p99 | c p99.9 | c max | w p50 | w p75 | w p90 | w p99 | w p99.9 | w max |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 256 MB | 4972.83 | 5227.66 | 5754.12 | 7551.76 | 7559.31 | 7562.5 | 6.40 | 13.09 | 21.75 | 72.66 | 242.79 | 4973.05 |
| 512 MB | 2550.59 | 2604.69 | 2765.67 | 2942.48 | 3108.76 | 3110.69 | 5.82 | 6.83 | 9.68 | 33.92 | 85.16 | 2052.3 |
| 768 MB | 1801.28 | 1887.92 | 2251.03 | 2604.69 | 2681.29 | 2681.34 | 6.01 | 6.93 | 9.08 | 27.16 | 93.67 | 1550.8 |
| 1024 MB | 1521.33 | 1578.64 | 1918.35 | 2113.65 | 2115.77 | 2117.42 | 6.01 | 7.05 | 8.94 | 23.92 | 101.41 | 1077.45 |
| 1536 MB | 1204.06 | 1325.32 | 1507.70 | 1817.56 | 1821.19 | 1821.6 | 5.92 | 6.83 | 8.39 | 22.45 | 609.62 | 896.53 |
| 2048 MB | 1129.45 | 1286.17 | 1583.38 | 1819.37 | 1998.60 | 2000.17 | 6.30 | 7.51 | 9.38 | 25.09 | 472.89 | 863.68 |
Cold (c) and warm (w) start times with SnapStart and with DynamoDB invocation Priming in ms:
| RAM | c p50 | c p75 | c p90 | c p99 | c p99.9 | c max | w p50 | w p75 | w p90 | w p99 | w p99.9 | w max |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 256 MB | 1126.07 | 1183.77 | 1397.42 | 1608.90 | 1621.82 | 1622.23 | 6.72 | 13.72 | 24.30 | 71.52 | 511.95 | 753.45 |
| 512 MB | 831.84 | 881.49 | 1136.24 | 1348.03 | 1386.29 | 1387.52 | 5.82 | 6.72 | 10.15 | 32.86 | 254.63 | 444.51 |
| 768 MB | 819.46 | 891.23 | 1141.93 | 1243.19 | 1808.50 | 1809.34 | 5.73 | 6.61 | 8.52 | 24.30 | 83.82 | 2073.35 |
| 1024 MB | 692.79 | 758.00 | 1003.80 | 1204.06 | 1216.15 | 1216.88 | 6.21 | 7.27 | 9.38 | 25.09 | 103.03 | 256.65 |
| 1536 MB | 713.17 | 773.31 | 995.80 | 1124.94 | 1372.50 | 1372.73 | 6.01 | 7.05 | 8.94 | 23.92 | 103.03 | 262.89 |
| 2048 MB | 797.64 | 858.87 | 1080.86 | 1296.49 | 1376.62 | 1377.05 | 6.41 | 7.51 | 9.38 | 25.89 | 115.14 | 241.28 |
Conclusions
In this article we measured the warm start time of the Lambda function without SnapStart with SnapStart and for the latter with additional priming of DynamoDB invocation using different Lambda memory settings.
In case of not enabling SnapStart for the Lambda function we observed that increasing memory reduces the warm execution time for our use case especially for p>90. As adding more memory to the Lambda function is also a cost factor, the sweet spot between cold and warm start time and cost is around 1024 MB memory setting for the Lambda function for our use case. You can use AWS Lambda Power Tuning for very nice visualisations.
We made similar observations in the case of enabling SnapStart for the Lambda function without and with additionally using priming of DynamoDB invocation request.
In case of applying priming of the DynamoDB invocation it's still the same observation and there is only little to no improvement of the warm execution times using more than 1024 MB memory even for p99, p99.9 and maximal values.
Comparing the measurements with the same measurement done in part 13 but for Java 21 we observe that the overall warm start times are lower with Java 21 compared to Java 17 for nearly all percentiles but especially visible for the higher ones.
