Question #609
A financial application is streaming transaction logs containing sensitive financial information through an Amazon Kinesis Data Firehose delivery stream. A developer must redact specific sensitive fields from the logs before storing the processed data in an Amazon S3 bucket. What should the developer do to fulfill this requirement?
Configure Kinesis Data Firehose to use an AWS Lambda function for data transformation. Implement the Lambda function to redact the sensitive fields. Set up the delivery stream to send the transformed data to an S3 bucket.
Deploy an AWS Glue ETL job to process the data from the delivery stream. Configure the Glue job to redact the sensitive fields and output the results to the S3 bucket. Set the Glue job as the destination for Firehose.
Use an Amazon EC2 instance configured as the Firehose destination. Install a custom application on the EC2 instance to redact the sensitive fields and then forward the data to the S3 bucket.
Set up an Amazon Kinesis Data Analytics application to process the incoming data stream. Use SQL queries to redact the sensitive fields and direct the output to the S3 bucket through the delivery stream.
Explanation
Answer A is correct because Amazon Kinesis Data Firehose allows direct integration with AWS Lambda for data transformation. The Lambda function can process and redact sensitive fields in real-time before delivering the data to S3. This approach is serverless, scalable, and requires minimal operational overhead.
Other options are incorrect because:
- B: AWS Glue ETL jobs are designed for batch processing, not real-time streaming. Firehose cannot directly use Glue as a destination.
- C: Using EC2 adds management complexity and latency. Firehose can directly integrate with Lambda and S3 without needing intermediate infrastructure.
- D: Kinesis Data Analytics requires additional setup (e.g., Kinesis Data Streams) and SQL-based processing, which is less efficient for field-level redaction compared to Lambda.
Key Points:
1. Kinesis Data Firehose supports Lambda transformations for real-time data processing.
2. Lambda is ideal for lightweight, on-demand data redaction.
3. Avoid solutions requiring batch processing (Glue) or infrastructure management (EC2) when serverless options exist.
Answer
The correct answer is: A