Enterprise Vault (EV) is unable to retrieve items that have been migrated to Secondary Storage

book

Article ID: 100051035

calendar_today

Updated On:

Description

Error Message

Event ID: 6882
Source: Migrator Server
Reason:           The system cannot find the file specified.  (0x80070002)
Vault:              Sent Items
Vault Id:           XXXX
Item Id:            XXXX
Reference:          Vault/RI
Extended Item Info: 
Supplementary Info:
[time stamp]    ID: 6838
Failed to recall a Saveset from its Collection.    
Reason: The system cannot find the file specified.  (0x80070002)  
Relative Saveset Filename: XXXX.DVSSP  
Relative Collection Filename: XXX\Collection1405843.CAB  
PartitionEntryID: XXX  
PartitionGUID: XXX  
Partition Root Path: XXX  
Secondary Location:   
Migrated File Id:    
Reference: NSDR/LCF       

 

Dtrace on Server performing the retrieval would show the following:

EV:L        {CMigratorHelper::AccessMigrator} (Entry)
EV:M        CMigratorHelper::AccessMigrator : Entered : PartitionEntryId = XXX
EV:M        CMigratorHelper::AccessMigrator : Creating MigratorCallback object
EV:M        CMigratorHelper::AccessMigrator : Initializing MigratorCallback object
EV:M        CMigratorHelper::AccessMigrator : IStorageSystemCallbackPrimer.QueryInterface for IMigratorCallback
EV:M        CMigratorHelper::AccessMigrator : Obtaining Migrator ProgId for Partition XXX
EV:M        CMigratorHelper::AccessMigrator : PartitionEntryId = XXX, Migrator ProgId = EnterpriseVault.MigratorToStreamerBridge
EV:M        CMigratorHelper::AccessMigrator : Creating Migrator object with ProgId EnterpriseVault.MigratorToStreamerBridge
EV:L        {CMigratorHelper::AccessMigrator} (Exit) Status: [Success]
EV:M        CMigratorHelper::ExtractFromCab : PartitionEntryId = XXX, Migrator ProgId = EnterpriseVault.MigratorToStreamerBridge, SecondaryLocation = (null)
EV:L        {CMigratorHelper::RequestOfflineFile} (Entry)
EV:L        {CVaultStorePartitionCache::ReadEntry:#101} Successfully read partition info for Vault Store [XXXXXXXXXXXXXXX]
EV:L        {CPartitionCache::ReadEntry:#76} Vault Store ID 4, Partition ID 0 available in cache
EV:M        CMigratorHelper::RequestOfflineFile : Calling IStorageSystemCallbackPrimer.UserWaitTimeout property
EV:M        CMigratorHelper::RequestOfflineFile : Calling IStorageSystemCallbackPrimer.StorageOffline property
EV:L        CMigratorCallback::get_StorageOffline - Getting computername for EntryId: XXX
EV:L        {CStorageProxyBase::GetStorageComputer} (Entry)
EV:L        {CStorageOnlineOpnsInstanceHelper::GetCachedComputerInfo} (Entry)
EV:L        {CStorageOnlineOpnsInstanceHelper::GetCachedComputerInfo:#149} Input ComputerNameOrEntryId: [XXX] - Found ComputerName: [XXXXXXXXXXXXXXXXXx], SOO AdditionalInstanceCount: [4]
EV:L        {CStorageOnlineOpnsInstanceHelper::GetCachedComputerInfo} (Exit) Status: [Success]
EV:L        {CStorageProxyBase::GetStorageComputer} (Exit) Status: [Success]
EV:L        CMigratorCallback::get_StorageOffline - Connecting to computer: XXXXXXXXXXXXXXXXXx
EV:L        CMigratorCallback::get_StorageOffline - Found EVStgOffline in cache
EV:M        CMigratorHelper::RequestOfflineFile : Calling IStorageOffline.RequestFile method
EV:M        CMigratorHelper::RequestOfflineFile : IStorageOffline.RequestFile return code - hr=0x0
EV:L        {CMigratorHelper::RequestOfflineFile} (Exit) Status: [Success]
EV:L        {CCabinet::ExtractFromCab} (Entry)
EV:H        {CCabinet::ExtractFromCab:#273} Failed to extract files XXXX\XXXXXXXXXXXXXXX~03~XXXX~00~1.DVSSP| from Collection file XXXXXX.ARCHCAB. Extract Operation = 0, hr = 0x80070002, FDI Error =
EV:H        {CCabinet::ExtractFromCab} (Exit) Status: [The system cannot find the file specified.  (0x80070002)]
EV:H        {CMigratorHelper::ExtractFromCab} (Exit) Status: [The system cannot find the file specified.  (0x80070002)]

 

 

 

Cause

This issue may happen in environments where Vault Store (VS) Partitions are configured on Network Shares with 'Sharing' configured at a Vault Store Group level, causing savesets and their associated SISParts to be distributed across EV Servers, and retrievals happening in bulk (Vault Cache Sync, Exports, etc.).

This issue is known to happen due to the SMB Redirector Caches. When a file cannot be found by an SMB2 client in the first attempt, its reference is stored in a local 'File Not Found Cache' for a specific span of time (5 sec by default). if there are subsequent attempts to access that file within this span, the SMB2 client reports that it cannot be found even though the physical file is present at the target Network Path.

 

Resolution

Create the following Registry Key on the EV Server throwing the above errors:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Lanmanworkstation\Parameters
[DWORD] 
FileNotFoundCacheLifetime
Value 0

Warning: Incorrect use of the Windows registry editor may prevent the operating system from functioning properly. Great care should be taken when making changes to a Windows registry. Registry modifications should only be carried-out by persons experienced in the use of the registry editor application. It is recommended that a complete backup of the registry and workstation be made prior to making any registry changes.

 

About SMB2 Redirector Caches

Issue/Introduction

MigratorServer, StorageonlineOPNS, and other processes throw Event ID 6882 with error "The system cannot find the file specified" for SISParts that have been collected and migrated to Secondary Storage. This issue is typically reported when the item is being recalled from 1 EV Server, but the SISParts are stored on another EV Server. These errors are seen even if the SISParts are successfully downloaded on the Primary Partition by the EVStgOfflineOPNS process.