Error Message

The Lotus Domino Mailbox Provisioning Task, the Lotus Domino Mailbox Archiving Task or both could fail logging the error "Unable to initialize Names and Address Book (NAB) session for Domino server: 'Failed to create Notes Session (mutex)'" with Event ID 41050.

Before the Event ID 41050 there could be several others errors logged by the StorageCrawler such as:

Event :  29026
Source :  Enterprise Vault
Category : Storage Crawler
Computer : EVServer.domain.local
Description: StorageCrawler FetchAheadThread thread has been forcefully terminated. No action is necessary.

Type :        Error
Date :        1/8/2019
Time :        7:16:12 PM
Event :        28998
Source :        Enterprise Vault 
Category :    Storage Crawler
User :        N/A
Computer :    EVServer.domain.local
Description:
Retrieval of a saveset failed.

Reason: Abnormal error occurred      Error:        Reference:    Info:                                       For more information, see Help and Support Center at http://entced.symantec.com/entt?product=ev&language=english&version=11.0.1.0&build=11.0.1.3706&error=V-437-6727  (0xc0041a47)
Saveset Id: 201901086983142~201807040625410000~Z~C08182432D2292C6AC74849B0724AEA1
Vault Id: 101611C4D8FDD0B40B2528DAEB677A5EC1110000.domain.local
Index Sequence Number: 2627
Internal Reference: Fetching Saveset for indexing

Event :  6727
Source :  Enterprise Vault
Category : Storage Crawler
Computer : EVServer.domain.local
Description: Abnormal error occurred
Error:     Catastrophic failure  (0x8000ffff)
Reference: GIM/gim
Info:      126FFA0F8AD68C74DA276F5E57830F0261110000evsite1 (User Name/DOMAIN)
            
V-437-6727


Using the Sysinternals Handle utility and running the following command will result in the following:

Command:  handle /accepteula -a ev~nrh~init-mutex
Result:         6/4/2015 4:22:46 PM StorageCrawler.exe pid : 14940  type: Mutant         450: \BaseNamedObjects\ EV~NRH~INIT-MUTEX

A memory dump of the process id reported as mutant from the 'Handle' output will show as follows:

0:000> kPn1000
 # ChildEBP RetAddr 
00 003ef814 76d43bd7 ntdll!ZwDelayExecution+0x15
01 003ef87c 76d444a7 KERNELBASE!SleepEx+0x65
02 003ef88c 6b53e42a KERNELBASE!Sleep+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
03 003ef89c 6b66d6c0 nnotes!OSDelayThread+0x2a
04 003efad0 0043705f nnotes!OSTerm+0x380
05 003efadc 00436f93 StorageCrawler!NotesRuntimeHelper::TerminateInternal(void)+0x1f [v:\lotus\common\notesruntimehelper.cpp @ 509]
06 003efb2c 0041d181 StorageCrawler!NotesRuntimeHelper::Terminate(void)+0xc3 [v:\lotus\common\notesruntimehelper.cpp @ 483]
07 003efb5c 0041e2fc StorageCrawler!CStorageCrawlerModule::PostMessageLoop(void)+0x171 [v:\storage\storagecrawler\storagecrawler.cpp @ 350]
08 003efb74 0041c10e StorageCrawler!ATL::CAtlExeModuleT::Run(
int nShowCmd = 0n10)+0x5c [v:\ms sdk\atl server\inc\atlbase.h @ 3554]
09 003efbb0 0041e0fe StorageCrawler!CStorageCrawlerModule::Run(
int nShowCmd = 0n10)+0x8e [v:\storage\storagecrawler\storagecrawler.cpp @ 65]
0a 003efbd0 0041d66c StorageCrawler!ATL::CAtlExeModuleT::WinMain(
int nShowCmd = 0n10)+0x6e [v:\ms sdk\atl server\inc\atlbase.h @ 3364]
0b 003efbe0 00439bb7 StorageCrawler!wWinMain(
struct HINSTANCE__ * __formal = 0x003f0000,
struct HINSTANCE__ * __formal = 0x00000000,
wchar_t * __formal = 0x00511eea "-Embedding",
int nShowCmd = 0n10)+0x1c [v:\storage\storagecrawler\storagecrawler.cpp @ 530]
0c 003efc74 772233aa StorageCrawler!__tmainCRTStartup(void)+0x150 [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c @ 578]
0d 003efc80 77798fc2 kernel32!BaseThreadInitThunk+0xe
0e 003efcc0 77798f95 ntdll!__RtlUserThreadStart+0x70
0f 003efcd8 00000000 ntdll!_RtlUserThreadStart+0x1b


A dtrace of storagecrawler for the affected thread will show that the thread hangs as follows:
 

15210902     16:21:13.884         [14940]        (StorageCrawler)        <14944>        EV:M        CFetchAheadThread::Shutdown 2 threads are still alive after 901 seconds

15213640     16:21:14.898         [14940]        (StorageCrawler)        <14944>        EV~W        Event ID: 29026 StorageCrawler FetchAheadThread thread has been forcefully terminated. No action is necessary. | |Process Id: 14940 |Thread Id: 15092 |

15213641     16:21:14.898         [14940]        (StorageCrawler)        <14944>        EV:M        CFetchAheadThread::Shutdown - Terminating a thread after 15 minutes

15223795     16:21:15.912         [14940]        (StorageCrawler)        <14944>        EV~W        Event ID: 29026 StorageCrawler FetchAheadThread thread has been forcefully terminated. No action is necessary. | |Process Id: 14940 |Thread Id: 15100 |

15223796     16:21:15.912         [14940]        (StorageCrawler)        <14944>        EV:M        CFetchAheadThread::Shutdown - Terminating a thread after 15 minutes

15223825     16:21:16.052         [14940]        (StorageCrawler)        <14944>        EV:M        CFetchAheadThread::Shutdown...Done

15223827     16:21:16.052         [14940]        (StorageCrawler)        <14944>        EV:L        {NotesRuntimeHelper::Terminate} (Entry)

15223828     16:21:16.052         [14940]        (StorageCrawler)        <14944>        EV:M        NotesRuntimeHelper::AcquireInitMutex - Creating mutex

15223830     16:21:16.052         [14940]        (StorageCrawler)        <14944>        EV:M        NotesRuntimeHelper::AcquireInitMutex - Acquiring mutex

15223831     16:21:16.052         [14940]        (StorageCrawler)        <14944>        EV:M        NotesRuntimeHelper::AcquireInitMutex - Acquired mutex

15223832     16:21:16.052         [14940]        (StorageCrawler)        <14944>        EV:M        NotesRuntimeHelper::TerminateInternal - calling NotesTerm()