Delayed Write Failed – Event ID 50 – Solved

Saw this issue though OpsMgr 2012 R2 console, telling me that all Windows Server 2012 and 2012 R2 servers where reporting this issue every evening when the backup kicked in.

There are a lot of blog posts on the web telling you to disabling the cache on the disks and so on, but this is a default setting i did not want to change. And in my opinion that is a workaround, not a fix.

So working with this issue led me to an known error from VMware: KB2006849

Telling me that this has been solved in a patch from the 27th of january 2015 and later versions.

Version: ESXi 5.5 Patch 4
Release: 2015-01-27
Build: 2403361

To verify which version you are running, then look in the vSphere Client under one of your hosts:

VMware version

Then compare that number with this website:Build number history

If you have a build version before 2403361 (like the above example) – then you need to update your ESX hosts and the VMware tools on the virtual machines.

This solves the issue.

Event id numbers which leads to this issue:

Event ID 50, 57, 137, 140, 157 and 12289

Best regards.

Mads

How to monitor Unix, Linux, Debian, CentOS, RHEL, Ubuntu with OpsMgr 2012 R2

This is a quick guide collecting the information I needed to install an OpsMgr agent on RHEL 7 and CentOS 6 operating systems. There can be some variations from the different systems, but this should give you an indication on what is needed and save you some hours.

To install the agent the firewall needs to be opened, a local service account needs to be created and a security settings need to be set on the Linux server. After this it can be implemented in OpsMgr 2012 with the discovery wizard.

Prepare OpsMgr 2012 R2 for the Linux implementation

Kevin Holman preparation guide

Find the latest OpsMgr Management Pack: Google System Senter 2012 management pack for Unix ]

Import the Management PackHow to import and Operations Manager Management Pack

Version 2015/08 includes support for the following operating systems:

AIX 5.3, AIX 6.1, and AIX 7 operating systems.

HP-UX 11iv2 and HP-UX 11iv3 operating systems.

Red Hat Enterprise Linux Server 4, Red Hat Enterprise Linux Server 5, Red Hat Enterprise Linux Server 6, and Red Hat Enterprise Linux 7 operating systems.

Solaris 9, Solaris 10, and Solaris 11 operating systems.

SUSE Linux Enterprise Server 9, SUSE Linux Enterprise Server 10 SP1, SUSE Linux Enterprise Server 11, and SUSE Linux Enterprise Server 12 operating systems.

CentOS 5, CentOS 6, and CentOS 7 operating systems

Debian GNU/Linux 5, Debian GNU/Linux 6, Debian GNU/Linux 7, and Debian GNU/Linux 8 operating systems

Oracle Linux 5, Oracle Linux 6, and Oracle Linux 7 operating systems

Ubuntu Linux Server 10.04 and Ubuntu Linux Server 12.04, and Ubuntu Linux Server 14.04 operating systems

Information on ports and firewall requirements

Default discovery and management occurs over TCP 1270,

Troubleshooting, and diagnostics discovery occur over SSH, TCP 22.

Discovery and deployment over SSH, default TCP 22

  • Secure Shell (SSH) – Used for installing, upgrading, and removing agents.
  • Web Services for Management (WS-Management) – Used for all monitoring operations and include the discovery of agents that were already installed.

Installing SCOM agent – requirements

https://technet.microsoft.com/en-us/library/hh230690.aspx

Configure a Low-Privileged Account for sudo elevation:

To create a low-privileged user

1.Log on to the UNIX or Linux computer as root.

2.Add the user:

# useradd opsmgrsvc

3.Add a password and confirm the password:

# passwd opsmgrsvc

# (define password here)

You can now configure sudo elevation

To configure sudo elevation for the low-privileged user

1.Log on to the UNIX or Linux computer as root.

2.Use the visudo program to edit the sudo configuration in a vi text editor. Run the following command:

# visudo

3.Find the following line:

root ALL=(ALL) ALL

4.Insert the following line after it:

Opsmgrsvc ALL=(ALL) NOPASSWD: ALL

5.Insert the following line after “Defaults requiretty”

Defaults:opsmgrsvc !requiretty

6.Save the file and exit visudo:

Press ESC + : (colon) followed by wq!, and then press Enter.

7.Test the configuration by entering in the following two commands. The result should be a listing of the directory without being prompted for a password:

# su – opsmgrsvc

# sudo ls /etc

Configuring the firewall

RHEL 7 had a firewall enabled, and it was necessary to run the following command:

# iptables -I INPUT -p tcp -m tcp –dport 1270 -j ACCEPT

# firewall-cmd –runtime-to-permanent

Configuring sudo Elevation for UNIX and Linux Monitoring with System Center 2012 – Operations Manager

If you would like to have more granular control of the service account permissions, you can read the below post.

http://social.technet.microsoft.com/wiki/contents/articles/7375.configuring-sudo-elevation-for-unix-and-linux-monitoring-with-system-center-2012-operations-manager.aspx

Installing the agent

When the firewall ports are opened, and the service account is put in place, then the OpsMgr agent can be installed with the Discovery Wizard.

Management Pack view in OpsMgr 2012 console:

OpsMgr Linux MP

References:

Accessing UNIX and Linux Computers in Operations Manager

https://technet.microsoft.com/en-us/library/hh212886.aspx

How to Configure sudo Elevation and SSH Keys

https://technet.microsoft.com/en-us/library/hh230690.aspx

Configuring sudo Elevation for UNIX and Linux Monitoring with System Center 2012 – Operations Manager

http://social.technet.microsoft.com/wiki/contents/articles/7375.configuring-sudo-elevation-for-unix-and-linux-monitoring-with-system-center-2012-operations-manager.aspx

Accessing UNIX and Linux Computers in Operations Manager

https://technet.microsoft.com/en-us/library/hh212886.aspx

How to Configure sudo Elevation and SSH Keys

https://technet.microsoft.com/en-us/library/hh230690.aspx

Credentials You Must Have to Access UNIX and Linux Computers

https://technet.microsoft.com/en-us/library/hh476947.aspx

Agent and Agentless Monitoring

https://technet.microsoft.com/library/hh487284.aspx

Understanding SCOM 2012 Alerts and Monitors and how to reactivate a closed Monitor

If you would like to know a bit more about the differences between a SCOM “Rule” and a “Monitor” and why Alerts can be closed and Monitors should not, then read this great article from Cameron Fuller. It describes nicely how to react on Alerts and Monitors in SCOM / OpsMgr 2012 R2 

An alert can typically be closed if the state has not changed for a longer period of time, otherwise there would be a repeat count on the alert if it were still an issue.

Monitors will typically close by them self, if not you would have to reset the health state to close it automaticly.

If you by accident close a Monitor, it will not reappear before the health state changes. Therefore, if you are running out of disk space, the monitor will only reappear when the issue have been resolved and then reappears.

This script can reset the closed monitors, which has been copied from this great article, with a small fix since the script was missing a terminator.

$Alertname=@();
$State=@();
$Displayname=@();
# Import Operations Manager Module and create Connection
Import-Module OperationsManager;
New-SCOMManagementGroupConnection EURSCOMACS01;
$alerts=get-scomalert -Criteria “Severity!=0 AND IsMonitorAlert=1 AND ResolutionState=255” | where {$_.LastModified -ge ((get-date).AddMinutes(-5)).ToUniversalTime()}
if ($alerts -is [object])
{
foreach ($alert in $alerts)
{
$monitoringobject = Get-SCOMClassinstance -id $alert.MonitoringObjectId
# Reset Monitor
If (($monitoringobject.HealthState -eq “Error”) -or ($monitoringobject.HealthState -eq “Warning”))
{
$monitoringobject.ResetMonitoringState()
$State+=$monitoringobject.HealthState
$Displayname+=$monitoringobject.displayname
$Alertname+=$alert.Name
}
}
}

I have verified that it works, but use at your own risk.

/Mads