File Server Cluster with Enterprise Vault FSA Agent installed reports a cluster node lockout or deadlock requiring a hard reboot

book

Article ID: 100014405

calendar_today

Updated On:

Description

Error Message

Resource Monitor (resrcmon.exe) reports an error in cluster event log:

Cluster resource monitor detected a deadlock possibly caused by invoking the 'Offline' entry point of resource DLL 'EnterpriseVaultFSA_M.dll'

Memory dump analysis of resource monitor points to EnterpriseVaultFSA_M.DLL as cause of cluster node lockout.

This activity is logged in the FSA Cluster log on EV server:  

[Install Drive]\Program Files (x86)\Enterprise Vault\Utilities\FSA Cluster - on 64 bit Operating System

[Install Drive]\Program Files\Enterprise Vault\Utilities\FSA Cluster - on 32 bit Operating System

[2014/06/15 21:33:42:684] MSCS.cpp(2850):DeleteFileFromRemote for node File01 [0][2014/06/15 21:33:42:918] MSCS.cpp(2850):DeleteFileFromRemote for node File01 [0]
 
 
When the FSA Cluster wizard is launched via the VAC to configure the clustered File Server target for High Availability (HA), therefore adding the FSA cluster resources to the group, the EnterpriseVaultFSA_M.dll is copied from Enterprise Vault server to the respective cluster nodes. The wizard copies the latest version of this file corresponding to the EV version running on the server.
 
 
CopyResFileToClusNodes: Is64bit = 1 [0][2014/06/15 22:07:03:617] MSCS.cpp(2328):Calling CopyFileW, source: 'D:\Program Files (x86)\Enterprise Vault\Utilities\FSA Cluster\x64\EnterpriseVaultFSA_M.dll', destination : '\\File01\C$\Windows\cluster\EnterpriseVaultFSA_M.dll' [0][2014/06/15 22:07:03:882] MSCS.cpp(2328):Calling CopyFileW, source: 'D:\Program Files (x86)\Enterprise Vault\Utilities\FSA Cluster\x64\EnterpriseVaultFSA_M.dll', destination : '\\File01\C$\Windows\cluster\EnterpriseVaultFSA_M.dll'

Cause

When the FSA Cluster Configuration wizard runs to remove the resources from the clustered File Server groups, the EnterpriseVaultFSA_M.dll is correctly deleted from the cluster nodes.

On the cluster nodes, the dll is typically located in the following directory:
      C:\windows\cluster

This issue can occur if the EnterpriseVaultFSA_M.dll is either manually deleted or was not properly removed / updated on the File Server.

Resolution

Upgrading the Enterprise Vault FSA Cluster resource within the clustered File Server targets requires the removal of the FSA cluster resources prior to installing a new version of the FSA Agent.

Once the FSA Agent has been correctly installed on the cluster nodes, the FSA Cluster wizard should be used to reconfigure the FSA Cluster resource. 

For more details on upgrading Enterprise Vault and configuring FSA with clustered File Servers, please refer to the documentation listed in the Related Articles section below. 
 

Applies To

Windows Failover Clustering - WFC

Issue/Introduction

A lockout or deadlock situation is reported on a node participating in File Server Cluster where the Enterprise Vault FSA Agent is installed. A hard reboot is required as failover does not happen.
Powered by Wolken Software