Reprotect Fails SRM / SRM service crash

The issue may appear with a similar backtrace.

 

Reprotect fails.

–> Panic: Assert Failed: “ok” @ d:/build/ob/bora-4130000/srm/src/replication/providers/storageProvider/reverseReplication.cpp:1411
–> Backtrace:
–> backtrace[00] rip 00007ffdd64cabea
–> backtrace[01] rip 00007ffdd6390a3f
–> backtrace[02] rip 00007ffdd6391b5e
–> backtrace[03] rip 00007ffdd64e1d4c
–> backtrace[04] rip 00007ffdd64e1eac
–> backtrace[05] rip 00007ffdd6382800
–> backtrace[06] rip 000000000a186037
–> backtrace[07] rip 000000000a17d18d
–> backtrace[08] rip 0000000009f19bd1
–> backtrace[09] rip 00000000085cfd93
–> backtrace[10] rip 00000000085dac7d
–> backtrace[11] rip 00000000085dab69

 

Service fails during Test Recovery

 

–>
–> Panic: VERIFY d:/build/ob/bora-3037005/srm/src/recovery/settings/vmRecoverySettingsRepository.cpp:1291
–>
–> Backtrace:
–>
–> [backtrace begin] product: VMware vCenter Site Recovery Manager, version: 6.1.0, build: build-3037005, tag: –
–> backtrace[00] vmacore.dll[0x001BD30A]
–> backtrace[01] vmacore.dll[0x0005CA8F]
–> backtrace[02] vmacore.dll[0x0005DB9E]
–> backtrace[03] vmacore.dll[0x001D4CE5]
–> backtrace[04] vmacore.dll[0x001D4DDD]
–> backtrace[05] vmacore.dll[0x00026EBA]
–> backtrace[06] common.dll[0x00049163]
–> backtrace[07] dr-recovery.dll[0x005EA028]
–> backtrace[08] dr-recovery.dll[0x00377194]
–> backtrace[09] dr-recovery.dll[0x001635AA]
–> backtrace[10] dr-recovery.dll[0x001ACA68]
–> backtrace[11] dr-jobs.dll[0x00009004]
–> backtrace[12] dr-jobs.dll[0x0000DAF5]
–> backtrace[13] dr-jobs.dll[0x0000E0D1]
–> backtrace[14] dr-jobs.dll[0x00020807]
–> backtrace[15] dr-jobs.dll[0x00022017]
–> backtrace[16] dr-jobs.dll[0x000206E6]
–> backtrace[17] dr-jobs.dll[0x00022239]
–> backtrace[18] functional.dll[0x00028089]
–> backtrace[19] vmacore.dll[0x00151E6E]
–> backtrace[20] vmacore.dll[0x00155A1F]
–> backtrace[21] vmacore.dll[0x00156F71]
–> backtrace[22] vmacore.dll[0x00158CA5]
–> backtrace[23] vmacore.dll[0x0006602B]
–> backtrace[24] vmacore.dll[0x00153FD0]
–> backtrace[25] vmacore.dll[0x001D034B]
–> backtrace[26] MSVCR90.dll[0x00002FDF]
–> backtrace[27] MSVCR90.dll[0x00003080]
–> backtrace[28] KERNEL32.DLL[0x000016AD]
–> backtrace[29] ntdll.dll[0x000154E4]
–> [backtrace end]
–>

Resolution:

 

Stop SRM service both sites.
Backup the SRM db both sites.

psql -d srm_db -U srmadmin –host 127.0.0.1 –port 5678> srmadmindump

 

Open psql from the command prompt. If using embedded srm db.

psql -d srm_db –host 127.0.0.1 –port 5678 –username=srmadmin

From the psql prompt run query below.

DELETE FROM pdr_protectiongroupfailoverin;
DELETE FROM pdp_persistablelocktoken;
DELETE FROM pdj_job;
DELETE FROM pdj_engine;
DELETE FROM pdj_dataitem;

To reset the db password check out this link

 

http://www.virtuallypeculiar.com/2018/01/resetting-site-recovery-managers.html

Scenario 1

Start SRM services both sites.
Login to webclient.
Delete the protection group
Rerun reprotect will fail or if test plan Run cleanup.
Delete recovery plan.

If after the above steps, the DR site SRM server crashed on restart.

 

To get SRM service to start, I also needed to reset the “state” for the VM in pdr_protectedvm on DR site to from inflight value of “22” back to “6” (ready to be recovered state). Matching table on the production site had required VM state value of “0”. (This may not be required in every case, depending on the point at which the plan is interrupted)

Scenario 2

Start SRM services both sites.
Login to webclient.
If root cause of interrupt is known fix this and rerun the test
Rerun reprotect will fail or if test plan Run cleanup.
Delete recovery plan.

 

After the above steps, the DR site SRM server crashed on restart.

 

To get SRM service to start, I also needed to reset the “state” for the VM in pdr_protectedvm on DR site to from inflight value of “22” back to “6” (ready to be recovered state). Matching table on the production site had required VM state value of “0”. (This may not be required in very case, depending on the point at which the plan is interrupted)

Voliá!