eDiscovery Platform Worker Nodes will not come online or enable after upgrade or install of Software.

book

Article ID: 100070485

calendar_today

Updated On:

Description

Error Message

Unavailable

NOTE:  All images and log entries in this article are from a Veritas support lab.

Worker node appliances are in red color under the Appliances tab with Role Unavailable.  Below is an example (See Figure 1).

Figure 1.

In the master server log file, the following log message will be present:
2024-08-24 21:38:28,734 INFO  [utilitynodes.resourcemanager.UNResourceMonitor] (UNResourceMonitorThread:[])  Error pinging node: eDP-App2.eDP.lab:2595 ESA Application service is not up and running.
com.teneo.esa.admin.service.ServiceNotInitializedException: [#71000] Admin Services Error : service not initialized - HealthCheckManager.HealthCheckManager
    at com.teneo.esa.admin.service.ServiceResolver._getMBean(ServiceResolver.java:312) ~[testclasses/:?]
    at com.teneo.esa.admin.service.ServiceResolver.getMBean(ServiceResolver.java:165) ~[testclasses/:?]
    at com.teneo.esa.utilitynodes.resourcemanager.UNResourceMonitor.performBasicHealthCheck(UNResourceMonitor.java:226) [testclasses/:?]
    at com.teneo.esa.utilitynodes.resourcemanager.UNResourceMonitor.monitorAllResources(UNResourceMonitor.java:200) [testclasses/:?]
    at com.teneo.esa.utilitynodes.resourcemanager.UNResourceMonitor.monitorResources(UNResourceMonitor.java:179) [testclasses/:?]
    at com.teneo.esa.utilitynodes.resourcemanager.UNResourceMonitor.run(UNResourceMonitor.java:93) [testclasses/:?]
    at java.lang.Thread.run(Thread.java:750) [?:1.8.0_412]
Caused by: com.teneo.esa.admin.jmx.MBeanNotFoundException: [#71002] JMX Error: No object with the name teneo:instance=HealthCheckManager,name=HealthCheckManager is registered in the server at eDP-App2.eDP.lab:2595 35f0be (client f573c0)
    at com.teneo.esa.admin.jmx.RmiClientImpl.lookupObjectId(RmiClientImpl.java:522) ~[testclasses/:?]
    at com.teneo.esa.admin.jmx.RmiClientImpl.getProxyInternal(RmiClientImpl.java:404) ~[testclasses/:?]
    at com.teneo.esa.admin.jmx.RmiClientImpl.getProxy(RmiClientImpl.java:301) ~[testclasses/:?]
    at com.teneo.esa.admin.service.ServiceResolver$MBeanValidator.(ServiceResolver.java:330) ~[testclasses/:?]
    at com.teneo.esa.admin.service.ServiceResolver._getMBean(ServiceResolver.java:309) ~[testclasses/:?]
    ... 6 more

 

On the worker nodes, there's a warning message like the following in the server log:
2024-08-24 09:54:27,670 WARN  [admin.jmx.Connection] (clusterupdateservice.clusterupdateservice:[]) [#71000] Admin Services Error : Unable to connect to eDP-App1.eDP.lab:2595 after 30 of 60 attempts - Will retry in 8 secs

When adding an appliance to the distributed architecture (DA) cluster, a popup message shows (See Figure 2):
Verify Connection Failed
No response received. 
-Check your Host Address.
-Check your Clearwell Application Port.
-Check that this appliance is on-line.

Figure 2.

Cause

During the eDiscovery Platform (eDP) installation/upgrade process, the installer will ask to manage the firewall.  If the person who is installing the software answers no, then the installer will not add/modify the inbound allowed rules on the node.  If during an upgrade, and the person who is installing answers no, then the firewall inbound rules will not be modified to the new java path location.  Therefore, the worker nodes cannot communicate back to the master server.  The firewall should have all expected inbound rules or should be disabled to prevent unexpected product behavior.

Resolution

  1. On the master eDP server, disable the firewall temporarily.
  2. In a browser, login to a master eDP server.
  3. Select System>Appliances tab to see if the worker nodes come online and are enabled as expected.  If the worker nodes come online, then this is a firewall configuration issue.  There are 4 inbound firewall rules named "Webserver JVM" which will allow cwjave.exe on the worker nodes to communicate back to the master.
  4. If the eDP inbound firewall rules do not exist, then it is best to keep the firewall disabled until all the eDP necessary rules are created and configured.  These can be exported/imported from another working eDP server with the same versions of eDP and java or created manually.  Windows Defender Firewall logging for dropped connections can also be useful to monitor if the firewall is interfering with eDP processes.  If needing assistance creating or modifying the firewall rules, create a Veritas support case and reference this tech note.
  5. If the eDP inbound firewall rules exist, then the paths for cwjava.exe may be incorrect.  This is likely because a previous version of eDP was installed and during an upgrade, the option to allow the installer to manage the firewall settings was not selected.  Therefore, the path to cwjava.exe points to an older version.  For example, the path may be C:\jdk-8u412-windows-x64\jdk1.8.0_402\bin\cwjava.exe, but after the upgrade the path should be C:\jdk-8u412-windows-x64\jdk1.8.0_412\bin\cwjava.exe.  In this case, change the path from the 4 "Webserver JVM" inbound rules to the correct path, then go to the Appliances tab on the master to confirm if the worker node then comes online.

Issue/Introduction

Appliances tab shows worker nodes in red and under Roles column shows as Unavailable.