Question #592
A data analytics company uses an AWS Lambda function written in Python to process large CSV files uploaded to an Amazon S3 bucket. The Lambda function transforms the data, stores the results in a second S3 bucket, and updates metadata in an Amazon DynamoDB table. The Lambda function is triggered when new CSV files are uploaded. Recently, CSV file sizes have increased substantially, causing frequent Lambda timeouts even at the maximum duration. A solutions architect must redesign the architecture to prevent timeouts without managing infrastructure.
Package the application code into a Docker container image and upload it to Amazon Elastic Container Registry (Amazon ECR).
Create an Amazon ECS task definition with AWS Fargate compatibility. Configure the task to use the ECR image. Modify the Lambda function to trigger the ECS task via the AWS SDK when new files arrive in S3.
Use AWS Step Functions to orchestrate multiple Lambda invocations in parallel. Increase the Lambda memory allocation to the maximum.
Create an Amazon ECS task definition with EC2 compatibility. Configure the task to use the ECR image. Modify the Lambda function to trigger the ECS task via the AWS SDK when new files arrive in S3.
Migrate the CSV files to Amazon FSx for Lustre and process them using an Amazon EC2 Auto Scaling group. Update the Lambda function to trigger EC2 instances.
Explanation
AWS Lambda has a maximum execution timeout of 15 minutes, which causes failures for large CSV files. Options A and B address this by moving processing to Amazon ECS with AWS Fargate, which is serverless and supports longer-running tasks. Packaging the code into a Docker container (A) enables deployment to ECS Fargate (B), which Lambda triggers via the SDK. This avoids infrastructure management and eliminates timeouts.
Other options are incorrect:
- C: Step Functions with parallel Lambdas still face individual Lambda timeouts.
- D: ECS with EC2 requires managing infrastructure.
- E: EC2 Auto Scaling involves server management.
Key points: Lambda's 15-minute timeout, Fargate's serverless long-running capability, and avoiding infrastructure management.
Answer
The correct answer is: AB