Azure Log Analytics Data Export

data-export-overview

One of the new features which has been introduced in preview now is Azure Monitor Log Analytics data export. This allows (As it’s being collected), data from selected tables in your Log Analytics workspace can be continuously exported to an Azure storage account hourly or to Azure Event Hubs in near-real-time.

So instead of defining that certain services or defining Event Hub integration for instance on a resource level you can define this for all services that are now reporting into Log Analytics.

Many organizations are today using Log Analytics to collect data from Azure services which might be diagnostics data such as Audit logs and such but even performance metrics from the different services. In combination with this many companies might also have a existing SIEM solution on-premises such as Splunk or Q-Radar. With this capability you can still collect all the data into log analytics (even use Azure Sentinel as the first responder when it comes to automation) but export data for long-term or security related events down to your existing SIEM solution and also for long-term retention storage. So as an example I can configure Log Analytics to have a short retention period let’s say 30 days but then continosly export data to Event Hub which is then collected by my SIEM solution.

Of course within a Log Analytics workspace it will create a lot of predefined Tables which can be configured for export. Not every table is supported at this time, but you can reference this document updated list of supported tables (https://docs.microsoft.com/nb-no/azure/azure-monitor/platform/logs-data-export#supported-tables)

To configure the export you need to either use REST API or use AZ Cli and note that the table that you want to use as part of the export needs to be created in Log Analytics before you configure the export.

So if you have a newly created Log Analytics Workspace and event hub you will not be able to for instance setup an integration with SecurityEvents, unless you have configured a connector which will start to export that log data into Log Analytics Workspace.

The CLI command to setup the integration between Log Analytics and Event Hub, just replace the bold text with your own resources, also that if you want to export additional tables you need to use enter more tables after the first one just using spaces.

az monitor log-analytics workspace data-export create --resource-group
test-export-rg
 --workspace-name la-test-wrg --name ruleexport1 --tables Heartbeat --destination
/subscriptions/subid/resourceGroups/rg/providers/Microsoft.EventHub/namespaces/
eventhubnamespace/eventhubs/logexport


Then if successfull you will get this type of message indicating that it has created the rule successfull.

You also have the option to create an export rule which applies for all supported tables, but this of course will also contain all performance based tables as well. This can be done using the same command and appending a –export-all-tables true. if –export-all-tables is true, then the tables argument should not be provided.

 

 

 

 

 

 

 

You May Also Like

About the Author: Marius Sandbu

Leave a Reply

Your email address will not be published. Required fields are marked *