Index has stuck by [Maximum item wait time] timeouts, which causes huge number of index waiting items.

book

Article ID: 100015703

calendar_today

Updated On:

Description

Error Message

The major symptom of this problem is the accumulation of index awaiting items, and can cause Events 41021 which warn the huge number of index waiting items.

Example of Event 41021:
There are 115866 archived items in Vault Store 'xxxxxxxxx02 VaultStore3' that  are waiting to be indexed.

There may be a problem with Enterprise Vault indexing.

You can use the System Status feature in the Administration Console to help you resolve this issue.


There are many causes of this. The problem in this Tech Note can be identified by collecting DTrace for EVIndexVolumesProcessor. In the EVIndexVolumesProcessor DTrace log, administrator may find a lot of "Item kept timing out" DTrace log lines. Administrator may also find exceptions below:

{FETCHER.EN_US} Exception: Item kept timing out Info: worker thread has detected an error.

{FETCHER.EN_US} Exception: Item kept timing out Info:Could not fetch an item

Cause

This is caused by [Maximum item wait time] timeout which is 15 minutes default.
Enterprise Vault Index Service treats [Maximum item wait time] timeout as a recoverable error because this is most likely to be caused by a temporary high work load or some other temporary environmental reasons. In case Enterprise Vault Index Service meets this timeout, the same item will be queued to retry index again, and this works in most cases.
But sometimes, the timeout caused by the item's permanent characteristic. That means the timeout will occur infinitely. Then Enterprise Vault Index Service stuck to that particular item and will not proceed to the next index item.

Resolution

Temporary Workaround:

Administrator can extend the [Maximum item wait time] timeout to longer value, such as 30 minutes or 60 minutes. Veritas support can not advise the best value for individual customer's environment. Administrator need to decide the value suitable for his/her own environment.

To change [Maximum item wait time] timeout, follow steps below:
  1. Open Enterprise Vault Admin Console.
  2. Browse left pane tree to the server which is running Enterprise Vault Index Service.
  3. Right click on the server and open [Properties]
  4. Open [Advanced] tab, choose [Indexing] from the drop down.
  5. Search configuration item [Maximum item wait time] from the list.
  6. Change the value accordingly.
  7. Restart Enterprise Vault Indexing Service.


Permanent Solution:
This issue has been addressed in the following release:
 
Enterprise Vault 11.0.1 Cumulative Hotfix 3 Release
https://www.veritas.com/docs/000025726

Enterprise Vault 12

The following link contains information about Enterprise Vault 12
https://www.veritas.com/docs/000099905

Note: For information on how to obtain Enterprise Vault 12 see: How to obtain the license key and installation download for Veritas products

Issue/Introduction

Huge number of index waiting items.

Additional Information

ETrack: 3772585