Automate Audit Monitoring with Office 365 Alert Policies

Activity alerts aren’t new to Office 365 and have been available since mid-2016, but if you implement Alert Policies you’re applying much more intelligence to check audit entries.

By itself, Activity Alerts watches the flow of events into the Office 365 audit log and fires when users perform a selected activity. You can create an alert for any activity recorded in the audit log, in order to do this, you must enable auditing for your tenant and you have to enable mailbox auditing too.

Activity alerts deliver the benefit of not having to check the audit log for specific events for administrators.

Alert Policies

An alert policy can trigger alerts for every instance of a certain activity, such as when an administrator grants another user higher permissions, or can look for patterns of events, such as when certain number of activities occur within a specific time frame. An example would be, a user who downloads 100 files from a SharePoint library with an hour. That user could be perceived as a hard-worker, or it could also be a sign that someone is grabbing valuable information or intellectual property that they plan to take with them to another job.

Office 365 can also set the threshold for a trigger by analyzing up to a week’s worth of activities to understand what usually happens within a tenant. If something then happens that is outside the expected norm, Office 365 notifies the recipients set in the policy. Being able to create alert policies based on events, simple aggregation, or baseline thresholds are “huge” for administrators.

Default Policies

To help administrators understand how alert policies function, Office 365 creates the six default alert policies described in the table below. You can amend the recipients for alerts generated by default alert policies but you cannot change the other settings.

Creating Custom Alert Policies

Like any default policies, the default alert policies cover some generic situations that might or might not apply to your tenant, which is why you can define your own policies. To begin, go to the Alerts section of the Security and Compliance Center.

Alert Policy Settings

To create a new policy, click New alert policy and then enter the name of the policy, a free-form description, a category for the alert, which you can use to sort alerts in the View alerts page. The categories are:

• Data governance
• Data loss prevention
• Threat management
• Anything else

You assign the alerts fired by a policy to be Low, Medium, or High. Obviously, the more destructive a condition is, the higher its severity should be. After defining the basic elements of a policy, the next step is to set out what the policy will do, as shown in Figure 2.

Alert policies do not yet cover all the activities recorded in the Office 365 audit log because the intention is that alert policies can deliver near real-time notifications about problem and not all workloads feed events into the audit log that quickly. You can only select a single activity per policy, like the “Granted mailbox permission” activity selected here.

For most activities, you can define conditions that must exist before Office 365 fires an alert. For example, a user downloads a file to a computer with a specific IP address. It is also possible to configure alerts to fire every time a user performs the activity or when users download files from a specific site. We are interested in the activities of three users, so they are conditions for the policy.

The Threshold defines how often an activity must occur within a period before a problem condition exists. The threshold can also be unusual activity, which is when Office 365 compares activity against the baseline for the tenant. Setting too low a threshold usually results in many notifications, which can hide real problems (the lowest threshold is 3 activities). Setting too high means that Office 365 might not send notifications for situations when administrators need to act. It is easy to tune a threshold after noting how many notifications administrators must process to reach a point where notifications arrive when real problems exist. You can also set a daily limit for an alert policy so that a policy can only ever generate a certain number of notifications in a single day.

Handling Alerts

When Office 365 recognizes that the threshold and conditions set in an alert policy occur, it triggers an email alert (Figure 3) to the recipient list defined in the policy.

The alert message has an Investigate button. The link takes you to a page to view the details of the triggered alert. You can then resolve or suppress the alert. When you resolve an alert, you record a status for the alert (Active, Investigating, Resolved, Dismissed) and any comments to justify the status. The new status and comments and your name become part of the history of the alert.

If you suppress an alert, you tell Office 365 how long you want to suppress email notifications for similar alerts and select from 1 day, 7 days, 30 days, or 365 days. Office 365 continues to trigger new alerts during the suppression period but you will not receive email about them.

Dashboard

The Alerts dashboard has widgets for alert trends (number of incidents over the last two weeks), alerts by severity, recent alerts, alert policies, and other alerts. These widgets give an overview of what is happening in the tenant. More details of recent alerts are available through the View alerts choice.

Automation is Goodness

Automated checking of audit events is all goodness. No one wants to have to check events manually, especially at the volume that a busy tenant generates. Alert policies are not a reason by themselves to upgrade to Office 365 E5 and many will be happy with activity alerts, but it is nice to have the option.

This article was originally posted here