Kenichiro WadaRegarding the detection and stopping of Lambda loops introduced in 2023, in October 2024, the long-awaited (I think) detection and stopping of loops with S3 triggers has been supported!
During the 2023 update, I wrote verification articles in the following post.
As a continuation of that, or rather, since S3 triggers were not a target for detection and stopping at that time, and I wrote that I would be happy if they were updated in the future,I re-verified the S3 triggers after the update. (Two months late... sorry)
I will also update the past articles later.
For the runtime of the verification Lambda, I am using the one I created last time.
https://github.com/Kenichiro-Wada/aws-lambda-recursion-detection
It works as is, but the following have been addressed:
- Updates to various modules,
- Node.js runtime version upgrade (v18 -> v22)
- Added a 5-second wait in the internal process (to prevent it from stopping immediately)
- Updated README (mentioning detection and stopping)
Verification of Loop Detection and Notification with S3 Trigger
The verification of detection was done with the following configuration.
To confirm that it was okay, I also checked other detection patterns, but since there were no particular changes in behavior, I will omit them.
If you are interested, please refer to the previous article.
The result is... it stops properly (well, it has to stop, right? lol)
It has also been executed 16 times according to the metrics.
Before the update, it looked like this. ※From the previous verification.
Searching for messages output within Lambda in Cloudwatch Logs showed 16 entries, indicating that it was limited to 16 executions.
The metric for detection and stopping is "Recursive invocations dropped" and one stop has been recorded.
Notifications are also displayed on the AWS Health Dashboard as usual.
Additional Verification Needed
In this loop case, there is no flow to trigger a notification, so I think it might be possible to notify by setting a DLQ for Lambda.
I will try a slightly modified version of the flow. Also, I will additionally verify what happens when S3 and SQS are combined and update the article.
What Not to Forget (I Write This Every Time)
Although the long-awaited implementation for S3 triggers has been made, as I mentioned in a previous article, even though it has been implemented, we should not rely on it.
To begin with,
we should avoid loops.
I will not forget this. With S3 support, I think all cases that might cause an infinite loop have been addressed, but I still believe that avoiding loops in the first place is the most important.
In Case of Intentional Looping...
In cases where looping is intentional, stopping at 16 times becomes a problem, so in such cases,it seems that it can be disabled using the PutFunctionRecursionConfig API.
(I think it used to be raising a support case...) I might try it later...
The End
Detection & stopping is just a guardrail, I think (it's also mentioned in the guide).
As mentioned above, unless it's intentional,
it's important to avoid loops in the first place, and I will be careful not to do so.
This article is being translated from Japanese to English using Amazon Bedrock and Amazon Nova Pro.
Additionally, the title image is being generated using Amazon Bedrock and Amazon Nova Canvas.
