Securing Data transmission to Log Analytics v2 with Azure Automation

In this post I will show you a way to securely send data to Log Analytics v2 using Azure Automation and Log Ingestion API.

Context

Log Analytics is really useful to create your own dashboard with custom data.

You can for instance use a remediation script to collect data from your Intune devices and send them to Log Analytics.

For this, you need to use the Log Ingestion API.

The process can be found in my post here.

By default, MS recommands to create a app registration to use the Log Ingestion API.

The issue is that you need to provide info about the app in your script: appid, secret.

In this post we want to avoid this and we will see a way to do this.

The solution

It works as below:

1. An Azure Automation with managed identity is created.

2. A remediation script is used to collect data on your devices.

3. The script sends data to the Automation runbook through a webhook.

4. The automation runbook sends data to Log Analytics through the managed identity.

As mentioned it's a way to secure it, you can do this with Azure function (I will soon publish the same post using Azure function).

Get script

Click on the below GitHub picture to get the scripts.

You will find:

- Assign_permissions.ps1: script to add permissions to the MI

- Runbook.ps1: the Azure Automation runbook

- Remediation scripts folder sample

Prerequisites

YOu need first to add prerequisites to use the Log Ingestion API aka Log Analytics v2.

See my post here to get more info.

Creating automation account

Purpose: here we will create the automation account that will be used to run the script.

1. Go to Azure

2. Go to Automation accounts

3. Click on Create

4. Type a name

5. Choose a Subscription, resource group and region

6. Click on Create

7. Click on Go to resource

Setting Managed Identity

Purpose: the managed identity is used to authenticate to your tenant, this way we don't need to provide credentials.

It allows you to avoid the credentials part.

When you configure the managed identity, a new Azure Enterprise application will be created.

This one will be used to authenticate to our tenant and do API calls.

For that we will proceed as below:

1. Go to your automation account

2. Go to Identity

3. Go to System assigned

4. Select On

5. Click on Save

6. Click on Yes

7. When it's configured a new enterprise application will be created with the same name than the automation account

Adding permissions

Purpose: in order to be able to check if the device exists and is managed, we need the following permission.

We will proceed as below:

1. Edit the Assign_permissions.ps1 script

2. Modify variable $TenantID and $DisplayNameOfMSI

3. Run script with global admin rights

Creating the Runbook

Purpose: here we will create the script that run the KQL query remotely.

1. Click on Create a runbook

2. Type a name

3. In Runbook type, select PowerShell

4. Click on Create

5. Click on Edit on the runbook

6. Copy the Runbook.ps1 script content

7. Click on Save

8. Click on Publish

Adding a webhook

1. Go to your Runbook

2. Click on Add webhook

3. Click on Create new webhook

4. Type a name

5. Set it to Enabled

6. Copy the webhook URL

7. Click on OK

Adding DCR role

Now we need to give the application permission to use the DCR. 

For that proceed as below:

1. Go to your DCR

2. Go to Access Control (IAM) 

3. Click on Add role assignment

4. Check Monitoring Metrics Publisher 

5. Click on Next

6. Check User, group, or service principal 

7. Click on Select members

8. Search the function app managed identity

9. Click on Select

10. Click on Review + assign 

Secure the process

In remediation script

Here we work on the script that sends data to Log Analytics (through the automation runbook).

The process is easy:

1. Collect data in an array

2. Adds table name and DCR ID to the array

3. Converts content to JSON

4. Sends data to the runbook through the webhook

In the Remediation sample script folder you can find two scripts to collect data from your devices:

- Send_drivers.ps1

- Send_services.ps1

Let's give a look to the first one.

We will proceed as below:

1. Provides information about DCR, Table and webhook in variables $DCR, $Table and $webhookURI.

2. Collects required info (here drivers):

3. Converts content to JSON:

4. Sends data to the runbook using the webhook

Here you may noticed that in the header I added a password in the variable $Secure_header.

It will be used by the runbook to check if it has been correctly called from a secure script.

In the runbook we will need to compare the info of course.

Ok, ok it's not enough secure, so now let's work on the azure automation runbook side.

In the runbook

By default, to get info send from the webhook, the code to use is:

In the script that sends data, we added a password. To check it in the runbook we will proceed as below:

If the password provided in the remediation script is not the same the runbook script will be stopped.

Now to make it a bit more secure, we need to be sure that the source from which the data has been sent is valid.

For that we will check is the source device is compliant and managed.