Analyze VPC Flow log using EK

Tools/Task:

Create Flow Log in VPC
IAM Role for VPC Flow Log
Cloudwatch Log Group
ElasticSearch
IAM Lambda role

Create Flow Log on VPC

Open VPC
Click on Create Flow Log

3. For Role — Click on Set up Permissions, it will create a role and then select it

4. For Destination Log Group — this is the cloud watch log group name

The name of the CloudWatch Logs log group to which the flow log will be published. A log stream will be created in this log group for each network interface being monitored.

Flow Log is created successfully.

You can check the flow logs being generated

ElasticSearch Service

Create domain

All default setting then

Click Confirm

It will take around 10 minutes to create a elasticsearch service

Now go to cloudwatch

Click on Actions -> Stream to Amazon ElasticSearchService.

Select this Account and your ES cluster

CloudWatch Logs uses Lambda to deliver log data to Amazon ES. You must specify an IAM role that grants Lambda permission to make calls to Amazon ES. You can choose an existing role or create an IAM role that automatically has the required permissions. To deliver log data to another account, you must specify the Elasticsearch Domain ARN and Elasticsearch Endpoint of other account and ensure permissions are granted to be able to publish to that ARN.

Select — Click on Create new role

{
“Version”: “2012–10–17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“logs:CreateLogGroup”,
“logs:CreateLogStream”,
“logs:PutLogEvents”
],
“Resource”: [
“arn:aws:logs:*:*:*”
]
},
{
“Effect”: “Allow”,
“Action”: “es:ESHttpPost”,
“Resource”: “arn:aws:es:*:*:*”
}
]
}