Index Migration Process Appears to Hang

book

Article ID: 100050446

calendar_today

Updated On:

Description

Error Message

There is no specific error, but you will see in the \DataInsight\log\migration0.0.log on the Source Indexer that the files have stopped transferring.

The files do resume transferring at 07:00

2021-05-09 08:29:49 INFO:    #{180} [FilerMigrationHelper.compressedSend] Sending file by compressed way...file name = scan_cifs_2328_0_1620525724_snap1.isqlite
2021-05-09 08:29:49 INFO:    #{180} [FilerMigrationHelper.compressedSend] Sending migration copy URL from source : https://destinationidx.vtas.local:8383/commd/config?config_servlet_cmd=cp_idx_files&source_node=36&source_hostname=SourceIDX&product_version=6.1.0.0&product_version_real=6.1.6.13500¤t_time=1620574189707&msu_id=2328&cksum=1607236030&compressed=true&dest_folder=inbox
2021-05-09 08:29:49 INFO:    #{180} [FilerMigrationHelper.compressedSend] File sent completed with code = 200
2021-05-09 08:29:49 INFO:    #{180} [FilerMigrationHelper.compressedSend] Sent F:\DataInsight\data\inbox\scan_cifs_2328_0_1620525724_snap1.isqlite.zip to destinationidx.vtas.local:8383
2021-05-09 08:29:49 INFO:    #{180} [FilerMigrationHelper.populateMsuFilesInfoInDB] Files have been inserted for msu id : 2328
2021-05-09 08:29:49 INFO:    #{180} [MsuMigrationUpdater$MsuMigrationUpdaterThread.migrateIndex] Inbox files are sent to new indexer for msu id : 2328
2021-05-10 07:00:00 INFO:    #{50} [MsuMigrationUpdater.populateJobs] Updating migration thread for all msus

2021-05-10 07:00:00 INFO:    #{50} [MsuMigrationUpdater.addToQueue] Migrating index for 14 msus
2021-05-10 07:00:00 INFO:    #{50} [MsuMigrationUpdater.addToQueue] Adding msu 4979 for migrating updating list at position 0
2021-05-10 07:00:00 INFO:    #{50} [MsuMigrationUpdater.addToQueue] Adding msu 3218 for migrating updating list at position 1
2021-05-10 07:00:00 INFO:    #{50} [MsuMigrationUpdater.addToQueue] Adding msu 3252 for migrating updating list at position 2
2021-05-10 07:00:00 INFO:    #{50} [MsuMigrationUpdater.addToQueue] Adding msu 4644 for migrating updating list at position 3
2021-05-10 07:00:00 INFO:    #{50} [MsuMigrationUpdater.addToQueue] Adding msu 4645 for migrating updating list at position 4

Cause

There are many issues that can cause the migration to stop.

07:00 is the default schedule for the MsuMigrationJob which handles index migrations.

Resolution

When starting an index migration it is recommended to edit the default schedule for the MsuMigrationJob from Daily at 07:00 to run every hour on the Source Indexer as follows:

In the Data Insight Console, navigate to Settings - Data Insight Servers - - Advanced Settings - Set Custom Properties

Property Name = job.MsuMigrationJob.cron

Property Value = 0 0 0/1 1/1 * ? *

Click the Save button to save your changes

You can confirm the setting in the Jobs list of the Source Indexer

The job will now check the migration status each hour and restart the transfer if it stops instead of 07:00 daily

Issue/Introduction

When performing an index migration, the process appears to periodically hang when files are being transferred from the Source Indexer to the Destination Indexer.