The Enterprise Vault FSA agent installs a driver on the Windows File Server. During the restore process, the EV driver and the service will communicate to confirm if any recalls are occurring, even if the file server volume was not added to the Enterprise Vault Admin Console. The high amount of might cause a bottleneck between the driver and the service, causing this behavior.
Disable the Enterprise Vault Placeholder service before starting the restore process from backup.
Alternatively, the server could be placed in backup mode so that request for the restored items is not processed. Please refer to the following documentation.