Tuesday, August 03, 2010

Automatically closing SCOM alerts w/ powershell

You may have seen my post earlier about Syslog Monitoring Walkthrough with Systems Center Operations Manager 2007.  Well, I went on vacation and my colleagues got a little overzealous in what we directed at scom.  I came back to over 200k warnings from one device that was sending alerts at waaaay too low a threshhold.  These were going into a view that wasn’t normally checked (it seems) and now we have a problem.  The console crashes when trying to select all of them (surprise) and I do not want to do this by hand.  Powershell to the rescue?
After opening the OpsMgr PS console, we can get started.  Setting the alert to closed (ResolutionState =255) should be trivial.  We mustn’t forget to update the alert after we set the state.  The interesting part is that we need to get our collection of alerts with as little impact as possible.
I could just do:
$colAgents = get-alert
but that would crush my database and take forever.  To narrow this down a bit, I am going to select by source (as basically all of these came from the same source.) 
We could run a get-alert | where {SOMETHING} but that would have to return all the alerts and then parse them which would be very heavy.  The SCOM cmdlets have a –criteria to do the filtering in the db, lets use that.
Source seems to translate, at an object level, to MonitoringObjectDisplayName so I set my criteria, do the query and loop through the results like so:
$criteria = “ResolutionState = 0 AND MonitoringObjectDisplayName = ‘SERVERNAME’”

$colSysLogAlerts = get-alert -criteria $criteria

foreach($Alert in $colSysLogAlerts) {

      $Alert.ResolutionState = 255

      write-host “Closing $($Alert.id)”

      $Alert.Update("Closed by Powershell")

}

Powershell to the rescue indeed.

analytics