Varonis announces strategic partnership with Microsoft to accelerate the secure adoption of Copilot.

Learn more

Practical PowerShell for IT Security, Part V: Security Scripting Platform Gets a Makeover

A few months ago, I began a mission to prove that PowerShell can be used as a security monitoring tool. I left off with this post, which had PowerShell code...
Michael Buckbee
11 min read
Published August 22, 2017
Last updated June 16, 2023

A few months ago, I began a mission to prove that PowerShell can be used as a security monitoring tool. I left off with this post, which had PowerShell code to collect file system events, perform some basic analysis, and then present the results in graphical format. My Security Scripting Platform (SSP) may not be a minimally viable product, but it was, I think, useful as simple monitoring tool for a single file directory.

After finishing the project, I knew there were areas for improvement. The event handling was clunky, the passing of information between various parts of the SSP platform was anything but straightforward, and the information being displayed using the very primitive Out-GridViewwas really a glorified table.

Get the Free PowerShell and Active Directory Essentials Video Course

I'd recommend this for both new and advanced PowerShell users. Building an AD tool is a great learning experience.

New and Improved

I took up the challenge of making SSP a bit more viable. My first task was to streamline event handling. I had initially worked it out so that file event messages were picked up by a handler in my Register-EngineEvent scriptblock and sent to an internal queue and then finally forwarded to the main piece of code, the classification software.

I regained my sanity, and realized I could just directly forward the messages with Register-EngineEvent -forward from within the event handling scriptblock, removing an unnecessary layer of queuing craziness.

You can see the meaner, leaner version below.

  1. #Count events, detect bursts, forward to main interface
  2.  
  3. $cur = Get-Date
  4. $Global:Count=0
  5. $Global:baseline = @{"Monday" = @(1,1,1); "Tuesday" = @(1,.5,1);"Wednesday" = @(4,4,4);"Thursday" = @(7,12,4); "Friday" = @(5,4,6); "Saturday"=@(2,1,1); "Sunday"= @(2,4,2)}
  6. $Global:cnts = @(0,0,0)
  7. $Global:burst = $false
  8. $Global:evarray = New-Object System.Collections.ArrayList
  9.  
  10. $action = {
  11. $Global:Count++
  12. $d=(Get-Date).DayofWeek
  13. $i= [math]::floor((Get-Date).Hour/8)
  14.  
  15. $Global:cnts[$i]++
  16.  
  17.  
  18. #event auditing!
  19.  
  20. $rawtime = $EventArgs.NewEvent.TargetInstance.LastAccessed.Substring(8,6)
  21. $filename = $EventArgs.NewEvent.TargetInstance.Name
  22. $etime= [datetime]::ParseExact($rawtime,"HHmmss",$null)
  23.  
  24.  
  25. $msg="$($etime)): Access of file $($filename)"
  26. $msg|Out-File C:\Users\Administrator\Documents\events.log -Append
  27.  
  28. New-Event -SourceIdentifier Delta -MessageData "Access" -EventArguments $filename #notify
  29.  
  30. $Global:evarray.Add(@($filename,$etime))
  31. if(!$Global:burst) {
  32. $Global:start=$etime
  33. $Global:burst=$true
  34. }
  35. else {
  36. if($Global:start.AddMinutes(15) -gt $etime ) {
  37. $Global:Count++
  38. #File behavior analytics
  39. $sfactor=2*[math]::sqrt( $Global:baseline["$($d)"][$i])
  40.  
  41. if ($Global:Count -gt $Global:baseline["$($d)"][$i] + 2*$sfactor) { #at 95% level of poisson
  42.  
  43.  
  44. "$($etime): Burst of $($Global:Count) accesses"| Out-File C:\Users\Administrator\Documents\events.log -Append
  45. $Global:Count=0
  46. $Global:burst =$false
  47. New-Event -SourceIdentifier Delta -MessageData "Burst" -EventArguments $Global:evarray #notify on burst
  48.  
  49. $Global:evarray= [System.Collections.ArrayList] @()
  50. }
  51. }
  52. else { $Global:burst =$false; $Global:Count=0; $Global:evarray= [System.Collections.ArrayList] @()}
  53. }
  54. }
  55.  
  56. Register-EngineEvent -SourceIdentifier Delta -Forward
  57. Register-WmiEvent -Query "SELECT * FROM __InstanceModificationEvent WITHIN 5 WHERE TargetInstance ISA 'CIM_DataFile' and TargetInstance.Path = '\\Users\\Administrator\\' and targetInstance.Drive = 'C:' and (targetInstance.Extension = 'txt' or targetInstance.Extension = 'doc' or targetInstance.Extension = 'rtf') and targetInstance.LastAccessed > '$($cur)' " -sourceIdentifier "Accessor" -Action $action
  58. Write-Host "starting engine ..."
  59.  
  60. while ($true) {
  61.  
  62. Wait-Event -SourceIdentifier Access # just hang on this so I don't exit
  63.  
  64. }
#Count events, detect bursts, forward to main interface

$cur = Get-Date
$Global:Count=0
$Global:baseline = @{"Monday" = @(1,1,1); "Tuesday" = @(1,.5,1);"Wednesday" = @(4,4,4);"Thursday" = @(7,12,4); "Friday" = @(5,4,6); "Saturday"=@(2,1,1); "Sunday"= @(2,4,2)}
$Global:cnts =     @(0,0,0)
$Global:burst =    $false
$Global:evarray =  New-Object System.Collections.ArrayList

$action = { 
    $Global:Count++  
    $d=(Get-Date).DayofWeek
    $i= [math]::floor((Get-Date).Hour/8) 

   $Global:cnts[$i]++ 
   

   #event auditing!
    
   $rawtime =  $EventArgs.NewEvent.TargetInstance.LastAccessed.Substring(8,6)
   $filename = $EventArgs.NewEvent.TargetInstance.Name
   $etime= [datetime]::ParseExact($rawtime,"HHmmss",$null)
  

   $msg="$($etime)): Access of file $($filename)"
   $msg|Out-File C:\Users\Administrator\Documents\events.log -Append
  
   New-Event -SourceIdentifier Delta -MessageData "Access" -EventArguments $filename  #notify 
   
   $Global:evarray.Add(@($filename,$etime))
   if(!$Global:burst) {
      $Global:start=$etime
      $Global:burst=$true            
   }
   else { 
     if($Global:start.AddMinutes(15) -gt $etime ) { 
        $Global:Count++
        #File behavior analytics
        $sfactor=2*[math]::sqrt( $Global:baseline["$($d)"][$i])
       
        if ($Global:Count -gt $Global:baseline["$($d)"][$i] + 2*$sfactor) {  #at 95% level of poisson
         
         
          "$($etime): Burst of $($Global:Count) accesses"| Out-File C:\Users\Administrator\Documents\events.log -Append 
          $Global:Count=0
          $Global:burst =$false
          New-Event -SourceIdentifier Delta -MessageData "Burst" -EventArguments $Global:evarray #notify on burst
          
          $Global:evarray= [System.Collections.ArrayList] @()
        }
     }
     else { $Global:burst =$false; $Global:Count=0; $Global:evarray= [System.Collections.ArrayList]  @()}
   }     
} 

Register-EngineEvent -SourceIdentifier Delta -Forward 
Register-WmiEvent -Query "SELECT * FROM __InstanceModificationEvent WITHIN 5 WHERE TargetInstance ISA 'CIM_DataFile' and TargetInstance.Path = '\\Users\\Administrator\\' and targetInstance.Drive = 'C:' and (targetInstance.Extension = 'txt' or targetInstance.Extension = 'doc' or targetInstance.Extension = 'rtf') and targetInstance.LastAccessed > '$($cur)' " -sourceIdentifier "Accessor" -Action $action   
Write-Host "starting engine ..."

while ($true) {

   Wait-Event -SourceIdentifier Access # just hang on this so I don't exit    
  
}

 

Then I took on the main piece of code, where I classify files based on whether they have social security numbers and other sensitive keywords. As events come in from the handler, the file reclassification is triggered. This code then periodically displays some of the results of the classification.

In this latest version, I removed the “real-time” classification and focused on cleaning up the PowerShell code and improving the graphics — more on that below.

I took a wrong turn in the original version by relying on a PowerShell data locking module to synchronize data access from concurrent tasks, which I used for some of the grunt work. On further testing, the freebie module that implements the Lock-Object cmdlet didn’t seem to work.

As every junior system programmer knows, it’s easier to synchronize with messages than with low-level locks. I reworked the code to take the messages from the event handler above, and send them directly to a main message processing loop. In short: I was able to deal with asynchronous events in a synchronous manner.

.Net Framework Charts and PowerShell

My great discovery in the last month was that I could embed Microsoft-style charts inside PowerShell. In other word, the bar, line, scatter and other charts that are available in Excel and Word can be controlled programmatically in PowerShell. As a newbie PowerShell programmer, this was exciting to me. You can read more about .Net Framework Controls in this post.

It’s a great idea, and it meant I could also replace the messy Out-GridViewcode.

But the problem, I quickly learned, is that you also have to deal with some of the interactive programming involved with Microsoft forms. I just wanted to display my .Net charts while not having to code the low-level details. Is there a lazy way out?

After much struggle, I came to see that the easiest way to do this is to launch each chart in its own runspace as a separate task. (Nerd Note: this is how I avoided coding message handling for all the charts since each runs separately as modal dialogs.)

I also benefited from this freebie PowerShell module that wraps the messy .Net chart controls. Thanks Marius!

I already had set up a tasking system earlier to scan and classify each file in the directory I was monitoring, so it was just a matter of reusing this tasking code to launch graphs.

I created a pie chart for showing relative concentration of sensitive data, a bar chart for a breakdown of files by sensitive data types, and, the one I’m most proud is a classic event stair-step chart for file access burst conditions — a possible sign of an attack.

My amazing dashboard. Not bad for PowerShell with .Net charts.

For those who are curious about the main chunk of code doing all the work of my SSP, here it is for your entertainment:

  1. $scan = { #file content scanner
  2. $name=$args[0]
  3. function scan {
  4. Param (
  5. [parameter(position=1)]
  6. [string] $Name
  7. )
  8. $classify =@{"Top Secret"=[regex]'[tT]op [sS]ecret'; "Sensitive"=[regex]'([Cc]onfidential)|([sS]nowflake)'; "Numbers"=[regex]'[0-9]{3}-[0-9]{2}-[0-9]{3}' }
  9.  
  10. $data = Get-Content $Name
  11.  
  12. $cnts= @()
  13.  
  14. if($data.Length -eq 0) { return $cnts}
  15.  
  16. foreach ($key in $classify.Keys) {
  17.  
  18. $m=$classify[$key].matches($data)
  19.  
  20. if($m.Count -gt 0) {
  21. $cnts+= @($key,$m.Count)
  22. }
  23. }
  24. $cnts
  25. }
  26. scan $name
  27. }
  28.  
  29.  
  30.  
  31.  
  32. #launch a .net chart
  33. function nchart ($r, $d, $t,$g,$a) {
  34.  
  35. $task= {
  36. Param($d,$t,$g,$a)
  37.  
  38. Import-Module C:\Users\Administrator\Documents\charts.psm1
  39. $chart = New-Chart -Dataset $d -Title $t -Type $g -Axis $a
  40. Show-Chart $chart
  41.  
  42. }
  43. $Task = [powershell]::Create().AddScript($task).AddArgument($d).AddArgument($t).AddArgument($g).AddArgument($a)
  44. $Task.RunspacePool = $r
  45. $Task.BeginInvoke()
  46.  
  47. }
  48.  
  49. Register-EngineEvent -SourceIdentifier Delta -Action {
  50.  
  51. if($event.MessageData -eq "Burst") { #just look at bursts
  52. New-Event -SourceIdentifier File -MessageData $event.MessageData -EventArguments $event.SourceArgs
  53. }
  54.  
  55.  
  56. Remove-Event -SourceIdentifier Delta
  57. }
  58.  
  59.  
  60.  
  61.  
  62. $list=Get-WmiObject -Query "SELECT * From CIM_DataFile where Path = '\\Users\\Administrator\\' and Drive = 'C:' and (Extension = 'txt' or Extension = 'doc' or Extension = 'rtf')"
  63.  
  64.  
  65. #long list --let's multithread
  66.  
  67. #runspace
  68. $RunspacePool = [RunspaceFactory]::CreateRunspacePool(1,5)
  69. $RunspacePool.Open()
  70. $Tasks = @()
  71.  
  72.  
  73.  
  74.  
  75. foreach ($item in $list) {
  76.  
  77. $Task = [powershell]::Create().AddScript($scan).AddArgument($item.Name)
  78. $Task.RunspacePool = $RunspacePool
  79.  
  80. $status= $Task.BeginInvoke()
  81. $Tasks += @($status,$Task,$item.Name)
  82. }
  83.  
  84.  
  85.  
  86.  
  87. #wait
  88. while ($Tasks.isCompleted -contains $false){
  89.  
  90. }
  91.  
  92.  
  93. #Analytics, count number of sensitive content for each file
  94. $obj = @{}
  95. $tdcnt=0
  96. $sfcnt=0
  97. $nfcnt=0
  98.  
  99.  
  100. for ($i=0; $i -lt $Tasks.Count; $i=$i+3) {
  101. $match=$Tasks[$i+1].EndInvoke($Tasks[$i])
  102.  
  103. if ($match.Count -gt 0) {
  104. $s = ([string]$Tasks[$i+2]).LastIndexOf("\")+1
  105.  
  106. $obj.Add($Tasks[$i+2].Substring($s),$match)
  107. for( $j=0; $j -lt $match.Count; $j=$j+2) {
  108. switch -wildcard ($match[$j]) {
  109. 'Top*' { $tdcnt+= 1 }
  110.  
  111. 'Sens*' { $sfcnt+= 1}
  112.  
  113. 'Numb*' { $nfcnt+=1}
  114.  
  115. }
  116.  
  117. }
  118. }
  119. $Tasks[$i+1].Dispose()
  120.  
  121. }
  122.  
  123.  
  124. #Display Initial Dashboard
  125. #Pie chart of sensitive files based on total counts of senstive dat
  126. $piedata= @{}
  127. foreach ( $key in $obj.Keys) {
  128. $senscnt =0
  129. for($k=1; $k -lt $obj[$key].Count;$k=$k+2) {
  130. $senscnt+= $obj[$key][$k]
  131.  
  132. }
  133. $piedata.Add($key, $senscnt)
  134.  
  135. }
  136.  
  137.  
  138. nchart $RunspacePool $piedata "Files with Sensitive Content" "Pie" $false
  139.  
  140. #Bar Chart of Total Files, Sensitive vs Total
  141. $bardata = @{"Total Files" = $Tasks.Count}
  142. $bardata.Add("Files w. Top Secret",$tdcnt)
  143. $bardata.Add("Files w. Sensitive", $sfcnt)
  144. $bardata.Add("Files w. SS Numbers",$nfcnt)
  145.  
  146.  
  147. nchart $RunspacePool $bardata "Sensitive Files" "Bar" $false
  148.  
  149.  
  150. #run event handler as a seperate job
  151. Start-Job -Name EventHandler -ScriptBlock({C:\Users\Administrator\Documents\evhandler.ps1})
  152.  
  153.  
  154. while ($true) { #main message handling loop
  155.  
  156. [System.Management.Automation.PSEventArgs] $args = Wait-Event -SourceIdentifier File # wait on event
  157. Remove-Event -SourceIdentifier File
  158. #Write-Host $args.SourceArgs
  159. if ($args.MessageData -eq "Burst") {
  160. #Display Bursty event
  161. $dt=$args.SourceArgs
  162. #time in seconds
  163. [datetime]$sevent =$dt[0][1]
  164.  
  165. $xyarray = [ordered]@{}
  166. $xyarray.Add(0,1)
  167. for($j=1;$j -lt $dt.Count;$j=$j+1) {
  168. [timespan]$diff = $dt[$j][1] - $sevent
  169. $xyarray.Add($diff.Seconds,$j+1)
  170. }
  171. nchart $RunspacePool $xyarray "Burst Event" "StepLine" $true
  172. }
  173.  
  174.  
  175. }#while
  176.  
  177. Write-Host "Done!"
$scan = {  #file content scanner
$name=$args[0]
function scan {
   Param (
      [parameter(position=1)]
      [string] $Name
   )
      $classify =@{"Top Secret"=[regex]'[tT]op [sS]ecret'; "Sensitive"=[regex]'([Cc]onfidential)|([sS]nowflake)'; "Numbers"=[regex]'[0-9]{3}-[0-9]{2}-[0-9]{3}' }
     
      $data = Get-Content $Name
      
      $cnts= @()
      
      if($data.Length -eq 0) { return $cnts} 
      
      foreach ($key in $classify.Keys) {
       
        $m=$classify[$key].matches($data)           
           
        if($m.Count -gt 0) {
           $cnts+= @($key,$m.Count)  
        }
      }   
 $cnts   
}
scan $name
}




#launch a .net chart 
function nchart ($r, $d, $t,$g,$a) {

$task= {
Param($d,$t,$g,$a)

Import-Module C:\Users\Administrator\Documents\charts.psm1
$chart = New-Chart -Dataset $d -Title $t -Type $g -Axis $a
Show-Chart $chart

}
$Task = [powershell]::Create().AddScript($task).AddArgument($d).AddArgument($t).AddArgument($g).AddArgument($a)
$Task.RunspacePool = $r
$Task.BeginInvoke()

}

Register-EngineEvent -SourceIdentifier Delta -Action {
      
      if($event.MessageData -eq "Burst") { #just look at bursts
        New-Event -SourceIdentifier File -MessageData $event.MessageData -EventArguments $event.SourceArgs 
      }
      
      
      Remove-Event -SourceIdentifier Delta
}




$list=Get-WmiObject -Query "SELECT * From CIM_DataFile where Path = '\\Users\\Administrator\\' and Drive = 'C:' and (Extension = 'txt' or Extension = 'doc' or Extension = 'rtf')"  


#long list --let's multithread

#runspace
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1,5)
$RunspacePool.Open()
$Tasks = @()




foreach ($item in $list) {
  
  $Task = [powershell]::Create().AddScript($scan).AddArgument($item.Name)
  $Task.RunspacePool = $RunspacePool
  
  $status= $Task.BeginInvoke()
  $Tasks += @($status,$Task,$item.Name)
}




#wait
while ($Tasks.isCompleted -contains $false){
  
}


#Analytics, count number of sensitive content for each file
$obj = @{}
$tdcnt=0
$sfcnt=0
$nfcnt=0


for ($i=0; $i -lt $Tasks.Count; $i=$i+3) {
   $match=$Tasks[$i+1].EndInvoke($Tasks[$i]) 
  
   if ($match.Count -gt 0) {   
      $s = ([string]$Tasks[$i+2]).LastIndexOf("\")+1
      
      $obj.Add($Tasks[$i+2].Substring($s),$match)
       for( $j=0; $j -lt $match.Count; $j=$j+2) {      
         switch -wildcard ($match[$j]) {
             'Top*'  { $tdcnt+= 1 }
                      
             'Sens*' { $sfcnt+= 1}                      
                      
             'Numb*' { $nfcnt+=1} 
                                              
      }         
            
       }
   }    
   $Tasks[$i+1].Dispose()
   
}


#Display Initial Dashboard
#Pie chart of sensitive files based on total counts of senstive dat
$piedata= @{}
foreach ( $key in $obj.Keys) {
   $senscnt =0
   for($k=1; $k -lt $obj[$key].Count;$k=$k+2) {
     $senscnt+= $obj[$key][$k]

   }
   $piedata.Add($key, $senscnt) 

}


nchart $RunspacePool $piedata "Files with Sensitive Content" "Pie" $false

#Bar Chart of Total Files, Sensitive  vs Total
$bardata = @{"Total Files" = $Tasks.Count}
$bardata.Add("Files w. Top Secret",$tdcnt)
$bardata.Add("Files w. Sensitive", $sfcnt)
$bardata.Add("Files w. SS Numbers",$nfcnt)


nchart $RunspacePool $bardata "Sensitive Files" "Bar" $false


#run event handler as a seperate job
Start-Job -Name EventHandler -ScriptBlock({C:\Users\Administrator\Documents\evhandler.ps1})


while ($true) { #main message handling loop
   
       [System.Management.Automation.PSEventArgs] $args = Wait-Event -SourceIdentifier File  # wait on event
        Remove-Event -SourceIdentifier File
        #Write-Host $args.SourceArgs      
        if ($args.MessageData -eq "Burst") {
        #Display Bursty event
         $dt=$args.SourceArgs
         #time in seconds
         [datetime]$sevent =$dt[0][1]
         
         $xyarray = [ordered]@{}
         $xyarray.Add(0,1)
         for($j=1;$j -lt $dt.Count;$j=$j+1) {
               [timespan]$diff = $dt[$j][1] - $sevent
               $xyarray.Add($diff.Seconds,$j+1) 
          }
          nchart $RunspacePool $xyarray "Burst Event" "StepLine" $true 
        }        
        
   
}#while

Write-Host "Done!"

 

Lessons Learned

Of course, with any mission the point is the journey not the actual goal, right? The key thing I learned is that you can use PowerShell to do security monitoring. For a single directory, on a small system. And only using it sparingly.

While I plan on improving what I just presented by adding real-time graphics, I’m under no illusion that my final software would be anything more than a toy project.

File event monitoring, analysis, and graphical display of information for an entire system is very, very hard to do on your own. You can, perhaps, recode my solution using C++, but you’ll still have to deal with the lags and hiccups of processing low-level events in the application space. To do this right, you need to have hooks deep in the OS — for starters — and then do far more serious analysis of file events than is performed in my primitive analytics code. That ain’t easy!

I usually end up these DIY posts by saying “you know where this is going.” I won’t disappoint you.

You know where this is going. Our own enterprise-class solution is a true data security platform or DSP – it handles classification, analytics, threat detection, and more for entire IT systems.

By all means, try to roll your own, perhaps based on this project, to learn the difficulties and appreciate what a DSP is actually doing.

Have questions? Feel free to contact us!

Next Steps

If you’re interested in learning more practical, security focused PowerShell, you can unlock the full 3 hour video course on PowerShell and Active Directory Essentials with the code cmdlet.

What you should do now

Below are three ways we can help you begin your journey to reducing data risk at your company:

  1. Schedule a demo session with us, where we can show you around, answer your questions, and help you see if Varonis is right for you.
  2. Download our free report and learn the risks associated with SaaS data exposure.
  3. Share this blog post with someone you know who'd enjoy reading it. Share it with them via email, LinkedIn, Reddit, or Facebook.

Try Varonis free.

Get a detailed data risk report based on your company’s data.
Deploys in minutes.

Keep reading

Varonis tackles hundreds of use cases, making it the ultimate platform to stop data breaches and ensure compliance.

practical-powershell-for-it-security,-part-iii:-classification-on-a-budget
Practical PowerShell for IT Security, Part III: Classification on a Budget
Last time, with a few lines of PowerShell code, I launched an entire new software category, File Access Analytics (FAA). My 15-minutes of fame is almost over, but I was...
practical-powershell-for-it-security,-part-ii:-file-access-analytics-(faa)
Practical PowerShell for IT Security, Part II: File Access Analytics (FAA)
In working on this series, I almost feel that with PowerShell we have technology that somehow time-traveled back from the future. Remember on Star Trek – the original of course...
practical-powershell-for-it-security,-part-iv: -security-scripting-platform-(ssp)
Practical PowerShell for IT Security, Part IV:  Security Scripting Platform (SSP)
In the previous post in this series, I suggested that it may be possible to unify my separate scripts — one for event handling, the other for classification — into...
practical-powershell-for-it-security,-part-i:-file-event-monitoring
Practical PowerShell for IT Security, Part I: File Event Monitoring
Back when I was writing the ultimate penetration testing series to help humankind deal with hackers, I came across some interesting PowerShell cmdlets and techniques. I made the remarkable discovery...