Auditing and Security Monitoring of Azure Active Directory

In a cloud-based world, both for end-users and operator access to management layers the identity provider plays an important part to provide authentication/authorization/accounting and secure authentication mechanisms.

Many organizations today use Azure Active Directory to provide these identity mechanisms for both end-users such as accessing Office 365 but also other SaaS based applications. Many also use it together with B2C to provide consumers access to their services and use Azure AD as a mechanism to provide secure operator access to Azure-based services.

However, many organizations fail to configure basic audit policies for Azure AD and less so monitoring for abnormal user activity. Therefore, I wanted to write a bit about how to audit Azure Active Directory and how what kind of data is collected. Lastly, go into how you can monitor for abnormal traffic and activity against Azure Active Directory.

Let’s start with an overview of Azure Active Directory to give a bit of insight into the authentication flow and activities that are logged.

When a user authenticates to Azure Active Directory to access a certain service, such as Office 365 or other SaaS application. There will be an authentication workflow attempt to Azure AD. The first part is to evaluate if the user is within the specified tenant or if not will route the users to the right directory tenant. Next, it will evaluate the group memberships and password. All authentication attempts regardless of if pass/failed will get logged. Then there are of course conditional access policies which will be evaluated to determine what kind of settings should be in place before the user is given access or if it should be blocked.

Now within Azure Active Directory there are some different audit logs that are thereby default stored in a local Azure AD database. A log entry for Azure AD Sign-in logs will look something like this.

Example: Log entry

NOTE: An Azure AD Sign-in Activity also relates to two other attributes which are device information and conditional access policies. 

“userDisplayName”: “John Doe”,
“userPrincipalName”: “[email protected]”,
“userId”: “2das4c20-fd85-40c5-a6ac-c88baf8ca396”, (User GUID)
“appId”: “2012a258-227b-4e31-a9cf-717495945fc2” (App GUID)
“appDisplayName”: “Microsoft Azure PowerShell”, (App Displayed in Azure)
“ipAddress”: “8.8.8.8”, (IP Address of the user that tried to sign-in)
“clientAppUsed”: “Mobile Apps and Desktop clients”, (Identifies the legacy client used for sign-in activity)
“conditionalAccessStatus”: “notApplied”, (If any Conditional Access policies applied)
“isInteractive”: true, (Indicates if a sign-in is interactive or not.)
“riskDetail”: “hidden”, (Provides the ‘reason’ behind a specific state of a risky user, sign-in or a risk event)
“riskLevelAggregated”: “hidden”,
“riskLevelDuringSignIn”: “hidden”,
“riskState”: “none”,
“riskEventTypes”: [],
“riskEventTypes_v2”: [],
“resourceDisplayName”: “Windows Azure Active Directory”,
“resourceId”: “00000002-0000-0000-c000-000000000000”,
“status”: { (Sign-in status. Includes the error code and description of the error (in case of a sign-in failure))
“errorCode”: 0, (Error code if any, 0 indicates successfull authentication attempt
“failureReason”: “Other.”,
“additionalDetails”: null
},
“deviceDetail”: {
“deviceId”: “”,
“displayName”: “”,
“operatingSystem”: “Windows 10”,
“browser”: “”,
“isCompliant”: false,
“isManaged”: false,
“trustType”: “”
},
“location”: {
“city”: “Amsterdam”,
“state”: “Noord-Holland”,
“countryOrRegion”: “NL”,
“geoCoordinates”: {
“altitude”: null,
“latitude”: 52.35,
“longitude”: 4.917
}
},
“appliedConditionalAccessPolicies”: [
{
“id”: “b9a01886-57fe-47af-861b-bc979bcdbb2b”,
“displayName”: “Example Policy”,
“enforcedGrantControls”: [],
“enforcedSessionControls”: [],
“result”: “notEnabled”

This log entry indicates that a specific user logged into Azure using Azure PowerShell client and didn’t have any Conditional Access policies that were applied. The most important things to consider is

1: If successful authenticated if the traffic originates from known locations and device?
2: If failed authentication, what is the reason why it failed?

Since these events are logged as well, it means that we can check through the activity. However, it should be noted that by default all these logs will be stored within the Azure AD database and retention of these logs are dependent on what kind of licenses you have for Azure AD (P1 or P2) you however also have the option to define diagnostics, exporting the logs out of Azure AD and into a Log Analytics Workspace.

NOTE: in order to export Sign-in data, your organization needs Azure AD P1 or P2 license.

The different log sources will log some different sets of data.

• AuditLogs
• SigninLogs
• NonInteractiveUserSignInLogs
• ServicePrincipalSigninLogs
• ManagedIdentitySigninLogs
• ProvisioningLogs
• ADFSSigninLogs

If you move the logs to Log Analytics with Sentinel we also have the option to create analytics queries that can automatically trigger an alert in case of suspicious behavior.

NOTE: You also have the option to export data to an Azure Storage Account and Azure Event Hub as well. You can also define multiple export flow to having data sent to multiple workspaces for instance.

So what should we be looking for when it comes to abnormal behavior in Azure AD logs?

First of you need to understand the different EventID’s that are being logged, NOTE: I’ve created a list of some of the eventID’s that Azure AD uses here –> https://github.com/msandbu/azuread/blob/main/azuread.md which can be used as reference list.

Best practices when it comes to monitoring Azure AD

Once you have configured diagnostics and have the log data in place, what should you be looking for in the data sets?

Log Analytics and Azure Sentinel use Kusto queries to analyze the data. Kusto is a read-only query language, but also comes with a surprisingly good visualizer as part of the portal.

Luckily, there is a good list of different kusto queries that we can use here (which is a GitHub repo maintained by Microsoft) –>

Azure-Sentinel/Hunting Queries/SigninLogs at master · Azure/Azure-Sentinel · GitHub

There are of course some different rules that we can create. Hunting rules are used to explore (but not run on a regular basis) and the second one is analytics rules which are running in a predefined schedule. One example of hunting rules which I typically use is a simple one to determine the logon locations from our sign-ins.

SigninLogs 
| summarize count() by Location

Another good example is showing failed logon attempts to Azure AD where the error code is 50126 or 50020.

 SigninLogs
 // 50126 - Invalid username or password, or invalid on-premises username or password.
 // 50020? - The user doesn't exist in the tenant.
 | where ResultType in ( "50126" , "50020")
 | extend OS = DeviceDetail.operatingSystem, Browser = DeviceDetail.browser
 | extend StatusCode = tostring(Status.errorCode), StatusDetails = tostring(Status.additionalDetails)
 | extend State = tostring(LocationDetails.state), City = tostring(LocationDetails.city)
 | summarize StartTime = min(TimeGenerated), EndTime = max(TimeGenerated), IPAddresses = makeset(IPAddress), DistinctIPCount = dcount(IPAddress), 
 makeset(OS), makeset(Browser), makeset(City), AttemptCount = count() 
 by UserDisplayName, UserPrincipalName, AppDisplayName, ResultType, ResultDescription, StatusCode, StatusDetails, Location, State
 | extend timestamp = StartTime, AccountCustomEntity = UserPrincipalName
 | sort by AttemptCount

Another example of this is to look at users that failed MFA authentication attempts.

SigninLogs
| where TimeGenerated >= ago(31d)
| where ResultType == "50074"

Another example is looking into external guest users and changes made to Conditional Access Policy.

AuditLogs
| where Category == "UserManagement"
| where OperationName == "Invite external user" or OperationName == "Redeem external user invite"

AuditLogs | where Category == "Policy" |
 project ActivityDateTime, ActivityDisplayName , TargetResources[0].displayName, InitiatedBy.user.userPrincipalName

Moving on, I’ll be adding a lot more Kusto queries that can be useful for monitoring Azure AD activities here –> msandbu/azuread (github.com)