Azure Event Hubs is a great platform for planning real time and actionable telemetry operations to audit your services in the Microsoft Azure ecosystem. The support for streaming diagnostic logs into Event Hubs was released in preview recently, and is made available in various services in Azure including App Service Gateways, Network Security Groups, Logic Apps, Data Lake, Search and Key Vault. These are the only the set of services where the support is currently available, and the footprint will only increase in days to come.

Telemetry of the diagnostic log using Event Hubs allows services to stream usage and makes it possible to apply corrective measures or compensation in real time. A great use for this feature could be with Logic Apps in which a live stream of real-time audit trail from a Logic App workflow could mean real-time analytics of the telemetry data and subsequent corrective or follow-up action. Streaming data to Event Hubs also allows you to perform live analysis of the data using Azure Stream Analytics and Power BI. In Stream Analytics you can directly create a query to fetch the service health hot path data from Event Hub and store the output into a Power BI dataset table. “Stream Analytics & Power BI“?explains the steps to configure a real time stream analytics dashboard using Power BI.

There are various means by which you can enable diagnostic streaming for your Logic App instance. One of the easier ways is to enable it through the Azure Portal. Let’s assume that our Logic App is a simple request-response service as shown below:

If you now navigate to your Logic App settings, you find the option under Diagnostic Settings after you enable the status to ON. The following picture illustrates this:

You can specify what to log. If you check “Export to Event Hubs”, you will have to provide the service bus namespace. The other option to create this is to execute the following command using the Azure Command Line Interface:

azure insights diagnostic set --resourceId  --serviceBusRuleId  --enabled true

You can analyze the JSON output received by the Event Hub instance using the Properties attribute. The event details will be available under this property.