Alright, let's give this text a quick polish and sprinkle some sass on it!
If you're like me and you're obsessed with cloud-based architectures, then you've gotta check out the "This is My Architecture" series from AWS. Every week, they feature a new design that'll make your head spin with excitement. Today, we're gonna talk about Biogen's awesome scalable centralized logging architecture for multi-account use.
Check out this snazzy diagram:
Now, let's break down the design decisions:
-
First, we use Amazon Kinesis Firehose to capture data from CloudWatch, CloudTrail, and VPC flow logs. This helps us create logs in the correct format.
-
Then, we use Lambda (for each individual account) to classify or split the information based on what data we would need for troubleshooting and what to store for later use.
-
Next, we use S3 as a centralized backup because it's scalable, encrypted, and has life cycle policies across the logs. Plus, different accounts may want to use different visualization tools and dashboards, so having a centralized storage helps with that design.
-
Lambda then pushes the data from S3 into ElasticSearch.
-
And finally, using the application keyname in ElasticSearch, we can search and troubleshoot the application. For example, we can find out which EC2 instances belong to a particular account and then use that information to check the logs.
-
The best part? It's mostly serverless, so the architecture scales on demand and doesn't require much maintenance.
Boom, there you have it! A slick logging architecture that'll make all your developer dreams come true.
Reference: This is My Architecture, AWS