Upgrade fails with 'Timeout expired. The timeout period elapsed prior to completion of the operation or the server not responding'

book

Article ID: 100018995

calendar_today

Updated On:

Description

Error Message

Timeout expired. The timeout period elapsed prior to completion of the operation or the server not responding.

Cause

The timeout during database upgrade processing can occur if any of several conditions occur.  Those conditions are, but are not limited to:

  1. Searches that are returning large numbers of hits are in progress.
  2. Many searches that are in a pending status that have a total hit count in the hundreds of thousands or greater.
  3. Exports or, for Discovery Accelerator, Production runs that are in progress.
  4. Insufficient available space for the transaction log growth needed for the upgrade processing.
  5. Stale table indexes and / or statistics.

 

Resolution

Clean up and prepare the Accelerator customer database for upgrade using the following actions.

STEP 1: Verify there are no active searches or productions.
Run the following scripts against the Accelerator customer database

SELECT tc.[Name] AS 'Case Name', tis.[Name] AS 'Search Name', tis.StatusID, tis.RunDate
FROM tblIntSearches tis
JOIN tblCase tc ON tc.CaseID = tis.CaseID
WHERE tis.StatusID IN (2, 5, 6)
/*
-- 2 = Searching
-- 5 = Rejecting
-- 6 = Deleting
The status 2, 5 and 6 are active; Allow these activities to complete.
*/

SELECT tc.[Name] AS 'Case Name', tp.[Name] AS 'Export Name',
tp.StatusID, tp.CreateDate AS 'Create Date'
FROM tblProduction tp
JOIN tblCase tc ON tc.CaseID = tc.CaseID = tp.CaseID
WHERE tp.StatusID IN (50, 51, 56, 57, 59, 190, 191)
/*
-- 50 = Preparing
-- 51 = Processing
-- 56 = Processing
-- 57 = Processing
-- 59 = Waiting to retry
-- 190 = To Be Stopped
-- 191 = Stopping
Allow these activities to complete.
*/

STEP 2:  Clean up completed searches.
Clean up all searches that are in any pending state (pending acceptance or pending acceptance with errors) and either accept or reject the search.
Run the following script against the Accelerator customer database to help identify the searches that require end user action.

SELECT tc.[Name] AS 'Case Name', tis.[Name] AS 'Search Name',
StatusID AS 'Search Status' FROM tblIntSearches tis
JOIN tblCase tc ON tc.CaseID = tis.CaseID
WHERE StatusID IN (3, 10)
/*
-- 3 = Pending Acceptance
-- 10= Pending Acceptance (with errors)
*/

STEP 3: Optimize database before upgrade
Before the upgrade, run maintenance on the Accelerator databases.  Veritas recommends the following maintenance be performed:
 
  • Stop the Enterprise Vault Accelerator Manager Service (EVAMS)
  • Backup database
  • Shrink databases (Note that this is an optional step designed to keep the physical database size intact but to reorganize the pages within the database so they are all toward the front of the database.  If done, this action should be followed by a transaction log backup and MUST be completed prior to rebuilding the indexes and updating the statistics.)
  • Rebuild Indexes
  • Update Statistics

STEP 4: Prepare Transaction log physical drive space
Ensure the hard drive containing the transaction log has at least 5 times the database size of available free space as the upgrade process generates all transactions needed to upgrade the database, then commits those transactions.  The transaction log can grow very large due to this action.

STEP 5: Prepare Transaction log growth mechanism
Ensure the transaction logs are configured for unlimited growth in 500 MB increments.  For large customer databases when the transaction log files grow by a percentage of the existing log size, a timeout could occur while the process waits for the next expansion increment to be created (i.e., if the log is 100GB and is configured to auto grow by 10%, the system will allocate 10GB for the next increment).

STEP 6: Increase Connection Times and Max Pool Size in Customer properties
Access the Customer properties through the EVBAAdmin site and increase the timeout values as follows:
 
  • Connection Time Out - Default is 500, increase in increments of 500 as needed (recommend setting to 1000 or 2500)
  • Connection Life Time - Default is 200, increase in increments of 100 as needed (recommend setting to 500 if upgrade times out after increasing the Connection Time Out value)
  • Max Pool Size - Default is 300, increase in increments of 100 as needed (recommend setting to 500 if upgrade times out after increasing the Connection Time Out value)

STEP 7: Upgrading multiple customer databases.
For multiple customer databases, upgrade each of database as described in steps 1 through 6.  Upgrade one database at a time to save on transaction file drive space and SQL processing load.

STEP 8: Optimize database after upgrade
After the upgrade is complete run maintenance on the Accelerator databases.  Veritas recommends the following maintenance be performed before and after the upgrade as well as creating a weekly maintenance schedule:
 
  • Stop the Enterprise Vault Accelerator Manager Service (EVAMS)
  • Backup database
  • Shrink databases (Note that this is an optional step designed to keep the physical database size intact but to reorganize the pages within the database so they are all toward the front of the database.  If done, this action should be followed by a transaction log backup and MUST be completed prior to rebuilding the indexes and updating the statistics.)
    Rebuild Indexes
  • Update Statistics

Issue/Introduction

Upgrade fails with 'Timeout expired. The timeout period elapsed prior to completion of the operation or the server not responding'
Powered by Wolken Software