Send Amazon Cloudwatch Logs to Loggly

You can push Amazon Cloudwatch Logs (CWL) to Loggly using an Amazon Lambda Blueprint. Please note these instructions are for Cloudwatch Logs, which are different from Cloudwatch metrics. Follow the instructions below.

AWS Setup

For this step you need to install the AWS Command Line Interface (AWS CLI). If you don’t have the AWS CLI installed, click here to learn more about its installation and usage.

NOTE: Always use the latest version of AWS CLI to get all the updated features. You can update the AWS CLI using the command below:

pip install --upgrade awscli

If you are new to AWS, follow this link to learn how to get AWS Access Key, Access ID and Region information.

1. Encrypt Loggly Customer Token

Please make sure that the AWS CLI is configured and is using the correct AWS Access Key ID, AWS Secret Access Key and Default region name using ‘aws configure’ command.

aws configure

Create a KMS key using the steps mentioned in the link with alias name: logglyCustomerToken
Encrypt the Loggly Customer Token using the AWS CLI. Please retain the quotes around the TOKEN below.

aws kms encrypt --key-id alias/logglyCustomerToken --plaintext "TOKEN"

Replace:

• TOKEN: your customer token from the source setup page

Sample Output:

You should see something like the output below. If you are getting any error, please make sure the user has access to the KMS key.

2.?Create a Lamba Role

Sign in to your AWS account and open the IAM console – https://console.aws.amazon.com/iam/
In your IAM console:

– Create a new Role say, ‘CloudWatch-Full-Access-Role‘.

– Select Role type as ‘Lambda‘ from the AWS Service Roles.

– Attach policy ‘CloudWatchFullAccess‘ and save.

3. Create Lambda Function from the blueprint

Go to AWS Lambda Console. Click the “Create function” button.

Select the “cloudwatch-logs-to-loggly”?Loggly blueprint

4. Configure triggers

Configure the triggers to?call your Lambda function as below.

Log Group: Select your log group whose logs you want to send to Loggly.
Filter Name: Provide your filter name.
Filter Pattern: This is not a mandatory field. You can keep it empty.
Enable trigger: Check this option to enable the trigger. You could also come back to this setting later to enable it.
Click on Next button.

5. Configure function and Add Environment Variables:

Configure Environment Variables as below:

kmsEncrptedCustomerToken: Copy the base-64 encoded, encrypted token from step 1's CLI output (CiphertextBlob attribute) and paste it in kmsEncryptionCustomerToken key. 

logglyTags: Enter logglyTags as per your requirement

logglyHostName: Enter logs-01.loggly.com

Important: Select the correct KMS key to encrypt. Not selecting the right key here will not make this function work.

6. Configure?Lambda function handler and role

Role: Choose an existing role. Select the role created in Step 2. 
Memory (MB): set memory to 512. You can increase it as needed.
Timeout: Set time out to 1 min.You can increase it as needed.
KMS key: Select logglyCustomerToken from the dropdown

7.?Adding CloudWatch-Full-Access-Role to the list of IAM users and roles who can use the key:

You need to add CloudWatch-Full-Access-Role to the list of IAM users and roles who can use this key to encrypt and decrypt data from within the applications and when using AWS services integrated with KMS. Head to the IAM and add the CloudWatch-Full-Access-Role role as shown below.

Important: If you miss this step you will not be able to successfully send Cloudwatch logs to Loggly.

8.?Test your function

?

8. Verify Events

Search Loggly events with the tag as cloudwatch2loggly over the past 20 minutes. It may take few minutes to index the events. If it doesn’t work, see the troubleshooting section below.

tag:CloudWatch2Loggly

Advanced Amazon CloudWatch Options

1. 1. • AWS KMS Key – Set up you KMS key and assign roles to it.
      • Cloudwatch Metrics – Send Cloudwatch Metrics to Loggly.
      • Loggly Libraries Catalog – See more logging options here.
      • Set a log group during setup in Cloudwatch Logs Agent

Troubleshooting

If you don’t see any data show up in the verification step, check for these common problems.

1. • Make sure you’ve included your own customer token
   • Make sure you are using the latest version of AWS CLI
   • Make sure you have configured the same roles as mentioned above
   • Create new role from template(s) only
   • Go to your Lambda function in AWS Console and click on View logs in Cloudwatch in the Monitoring tab to view logs.
   • Search or post your own Amazon Cloudwatch logging questions in the community forum.