Alright, let's give this text a quick polish and sprinkle some sass on it!
If you're like me and you're obsessed with cloud-based architectures, then you've gotta check out the "This is My Architecture" series from AWS. Every week, they feature a new design that'll make your head spin with excitement. Today, we're gonna talk about Biogen's awesome scalable centralized logging architecture for multi-account use.
Check out this snazzy diagram:
Now, let's break down the design decisions:
First, we use Amazon Kinesis Firehose to capture data from CloudWatch, CloudTrail, and VPC flow logs. This helps us create logs in the correct format.
Then, we use Lambda (for each individual account) to classify or split the information based on what data we would need for troubleshooting and what to store for later use.
Next, we use S3 as a centralized backup because it's scalable, encrypted, and has life cycle policies across the logs. Plus, different accounts may want to use different visualization tools and dashboards, so having a centralized storage helps with that design.
Lambda then pushes the data from S3 into ElasticSearch.
And finally, using the application keyname in ElasticSearch, we can search and troubleshoot the application. For example, we can find out which EC2 instances belong to a particular account and then use that information to check the logs.
The best part? It's mostly serverless, so the architecture scales on demand and doesn't require much maintenance.
Boom, there you have it! A slick logging architecture that'll make all your developer dreams come true.
Reference: This is My Architecture, AWS