One might hope that Microsoft would provide straightforward and coherent file activity events in the Windows event log. The file event log is important for all the usual reasons – compliance, forensics, monitoring privileged users, and detecting ransomware and other malware attacks while they’re happening. A log of file activities seems so simple and easy, right? All that’s needed is a timestamp, user name, file name, operation (create, read, modify, rename, delete, etc.), and a result (success or failure).
But this is Microsoft. And they never, ever do anything that’s nice and easy.
The Case of “Delete”
Let’s start with delete, the simplest scenario. Google “Windows file delete event” and the first answer, as usual, will be from Randy Franklin Smith’s Windows Event Encyclopedia: “4660: An object was deleted”.
To a Windows event newbie, it would appear that our research effort is complete. Unfortunately, this delete event is missing one critical piece of information: the file name. Why? If you’re detecting ransomware, one would want to know which files are being encrypted. Likewise, when being alerted on privileged user access to sensitive files, one would expect to know exactly which files were accessed.
The hard part is: how do you determine the file name associated with an event? Even identifying something as simple as a file name isn’t easy because Windows event information is spread out across multiple log entries. For example, you’d have to correlate the delete event 4660 with the “access object” event 4663. In practice, you’d create a search for matching 4660 and 4663 events, and then combine information from both events to derive a more user-friendly log entry.
Let’s continue discussing the correlations needed to implement file activity monitoring using the Windows event log. The actual implementation depends on the tools used to collect the event: a database query, a program, or a dedicated correlation engine – a program tailored for such activity.
The Windows File Activity Audit Flow
Windows does not log file activity in a way that you’d expect. Instead, it logs granular file operations that require further processing. So let’s deep dive into how Windows logs file operations.
The diagram below outlines how Windows logs each file operation using multiple micro-operations logs:The delete operation is a unique case in that there is a fourth event, 4660, mentioned above. The sequence is identified by the “Handle ID” event property which is unique to this sequence (at least until a reboot).
The event that provides the most information is 4663, identifying that an attempt was made to access an object. However, the name is misleading because Windows only issues the event when the operation is complete. In reality, there might be multiple 4663 events for a single handle, logging smaller operations that make up the overall action. For example, a rename involves a read, delete, and a write operation. Also, one 4663 event might include multiple values in the “Accesses” property which lists access rights exercised to perform the operation. Those “Accesses” serve as our guideline for the operations themselves. More on that later.
The following table provides more information about each event:
|Event ID||Name||Description||Data It Provides|
|4656||A handle to an object was
|Logs the start of every file
activity but does not guarantee that it succeeded
|The name of the file|
|4663||An attempt was made to access an object||Logs the specific micro
operations performed as part of the activity
|What exactly was done|
|4660||An object was deleted||Logs a delete operation||The only way to verify an
activity is actually a delete
|4658||The handle to an object was
|Logs the end of a file
|How much time it took|
One step you would not want to skip is setting Windows to log those events, which is not the default. You’ll find a good tutorial on how to do that here.
You get the idea: you’re dealing with lots of different low-level events related to a higher-level file actions. Let’s take up the problem of correlating them to produce user-friendly information.
To identify the actual action, we need to decode the exercised permissions as reported in the “Accesses” event property. Unfortunately, this is not a one-to-one mapping. Each file action is made up of many smaller operations that Windows performs and those smaller operations are the ones logged.
The more important “Accesses” property values are:
- “WriteData” implies that a file was created or modified unless a “delete” access was recorded for the same handle.
- “ReadData” will be logged as part of practically any action. It implies “Access” if no “Delete” or “WriteData” were encountered for the same handle and for the same file name around the same time.
- “Delete” may signify many things: delete, rename (same folder), move (to a different folder) or recycled, which is essentially a move to the recycle bin. Event 4660 with the same handle differentiate between delete or recycled for which a 4660 event is issued and a rename or move for which it is not.
Consider this only as a starting point. The analysis above is somewhat simplified, and real-world implementation will require more research. Some areas for further research are:
- Differentiating delete from recycle and move from rename.
- Analyzing attributes access (with or without other access operations).
- Handling event 4659 which is similar to 4660 but is logged on a request to delete a locked file on the next reboot rather than deleting them now.
- Researching reports that events come out of order and the “request handle event” (4656) may not be the first in the sequence.
You may want to review this PowerShell Script which reads Windows events and generates from them meaningful file activity report to get a somewhat less simplified analysis. A word of warning: it is not for the faint-hearted!
Windows Event Log Limitations for File Access Monitoring
While the Windows file activity events seem comprehensive, there are things that cannot be determined using only the event log. A few examples are:
- Create vs. modify: the only way to know if this is a new file or a modified file is to have knowledge of the prior state, i.e. whether the file existed before.
- Missing information on failures: in a case of an operation that was rejected due to insufficient permissions, the only event issued is 4656. Without a sequence, most of the processing described in this article is not possible.
- Cut & paste: while one would assume “cut and paste” would be similar to a move operation, in practice, the behavior seems to be similar to a delete operation followed by a create operation with no relations whatsoever between the two operations.
Collecting Windows file activity is a massive event flow and the Microsoft event structure, generating many events for a single file action, does not help. Such collection will require more network bandwidth to transfer events and more storage to keep them. Furthermore, the sophisticated logic required may need a powerful processing unit and a lot of memory.
To reduce the overhead, you may want to:
- Carefully select which files you monitor based on the scenario you plan to implement. For example, you may want to track only system files or shares that include sensitive data.
- Limit collection of unneeded events at the source. If your collection infrastructure uses Microsoft Event Forwarding, you can build sophisticated filters based on event IDs and event properties. In our case, filter only events 4656, 4660, 4663 and optionally 4658 and only for the “Accesses” values needed.
- Limit event storage and event sizes as raw Windows events are sizable.
An Alternative Approach
Real world experience shows that there are not many organizations that succeed in utilizing Windows Event Log for file activity monitoring. An alternative approach for implementing this important security and compliance measure would be to use a lightweight agent on each monitored Windows system, with a focus on file servers. Such an agent, similar to the Varonis agent alongside DatAdvantage, would record file activity with a minimal server and network overhead, enabling better threat detection and forensics.