RPO, RTO, WRT, MTD…WTH?!

Some time ago, I was engaged in a discussion with one of our customers to investigate the possibility of VMware Site Recovery Manager implementation in their datacenter. The discussion turned technical pretty soon and when I asked what their RPO or RTO requirements were, they could not answer it straight away simply because they didn’t know what it was or what it meant. And when I mentioned WRT and MTD, they were stunned even more. So to clarify it a little bit for them I started drawing and explaining the following along the way.

Consider the following scenario.

Stage 1: Business as usual

At this stage all systems are running production and working correctly.

Stage 2: Disaster occurs

BCDR-02

On a given point in time, disaster occurs and systems needs to be recovered. At this point the Recovery Point Objective (RPO) determines the maximum acceptable amount of data loss measured in time. For example, the maximum tolerable data loss is 15 minutes.

Stage 3: Recovery

BCDR-03

At this stage the system are recovered and back online but not ready for production yet. The Recovery Time Objective (RTO) determines the maximum tolerable amount of time needed to bring all critical systems back online. This covers, for example, restore data from back-up or fix of a failure. In most cases this part is carried out by system administrator, network administrator, storage administrator etc.

Stage 4: Resume Production

BCDR-04

At this stage all systems are recovered, integrity of the system or data is verified and all critical systems can resume normal operations. The Work Recovery Time (WRT) determines the maximum tolerable amount of time that is needed to verify the system and/or data integrity. This could be, for example, checking the databases and logs, making sure the applications or services are running and are available. In most cases those tasks are performed by application administrator, database administrator etc. When all systems affected by the disaster are verified and/or recovered, the environment is ready to resume the production again.

BCDR-05

The sum of RTO and WRT is defined as the Maximum Tolerable Downtime (MTD) which defines the total amount of time that a business process can be disrupted without causing any unacceptable consequences. This value should be defined by the business management team or someone like CTO, CIO or IT manager.

This is of course a simple example of a Business Continuity/Disaster Recovery plan and should be included in your Business Impact Analysis (BIA).

I hope this short explanation gives you some starting points when discussing a Business Continuity/Disaster Recovery implementation with your customer.

 

Thanks for your post Marek.

Ref: https://defaultreasoning.com/2013/12/10/rpo-rto-wrt-mtdwth/

 

 

Upgrade VMware Tools

#Export the CVS from the flash client 6.5.

#Save your ExportList.csv.

Import-Csv ExportList.csv |

foreach {

#Pulls  the var from the CSV

#”guestToolsUnmanaged|10304″ is a test value I want to avoid upgrading.
$strNewVMName = $_. “Name”
$toolsVersion = $_. “VMware Tools Version Status”

if( $toolsVersion -ne “guestToolsUnmanaged|10304” ){echo “requires upgrade” + $strNewVMName

  #Update VMtools without reboot
    Get-VM $strNewVMName | Update-Tools –NoReboot
      write-host “Updated $strNewVMName —— ” }

else

{echo “does not require upgrade” + $strNewVMName}

}

 

 

Common SRM DB tool errors.

Recently I had to upgrade my laptop to windows 10 so I needed to reinstall the SRM-DB tool. Here is one of the challenges I faced. While listing recovery plans was successful and listing protection groups also was successful I got this error when deleting the protection group function with the tool

HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
‘SELECT pdv_storageproviderdatastorel.db_id AS pdv_storageproviderdatastorel_db_id, pdv_storageproviderdatastorel.mo_id AS pdv_storageproviderdatastorel_mo_id, pdv_storageproviderdatastorel.ref_count AS pdv_storageproviderdatastorel_ref_count, pdv_storageproviderdatastorel.typestring AS pdv_storageproviderdatastorel_typestring, pdv_storageproviderdatastorel.protectedurl AS pdv_storageproviderdatastorel_protectedurl, pdv_fileinfo.db_id AS pdv_fileinfo_db_id, pdv_fileinfo.mo_id AS pdv_fileinfo_mo_id, pdv_fileinfo.ref_count AS pdv_fileinfo_ref_count, pdv_fileinfo.datastore AS pdv_fileinfo_datastore, pdv_fileinfo.vmfilepath AS pdv_fileinfo_vmfilepath, pdv_fileinfo.snapshotdirectory AS pdv_fileinfo_snapshotdirectory, pdv_fileinfo.suspenddirectory AS pdv_fileinfo_suspenddirectory, pdv_fileinfo.logdirectory AS pdv_fileinfo_logdirectory, pdv_fileinfo.vmconfigfilename AS pdv_fileinfo_vmconfigfilename, pdv_fileinfo.vmconfigdirectory AS pdv_fileinfo_vmconfigdirectory, pdv_datastorelocator.db_id AS pdv_datastorelocator_db_id, pdv_datastorelocator.mo_id AS pdv_datastorelocator_mo_id, pdv_datastorelocator.ref_count AS pdv_datastorelocator_ref_count, pdv_datastorelocator.typestring AS pdv_datastorelocator_typestring, pdr_protectedvm.db_id AS pdr_protectedvm_db_id, pdr_protectedvm.mo_id AS pdr_protectedvm_mo_id, pdr_protectedvm.ref_count AS pdr_protectedvm_ref_count, pdr_protectedvm.state AS pdr_protectedvm_state, pdr_protectedvm.parentgroupmoid AS pdr_protectedvm_parentgroupmoid, pdr_protectedvm.vminfo AS pdr_protectedvm_vminfo, pdr_protectedvm.vmid AS pdr_protectedvm_vmid, pdr_protectedvm.settingschangeversion AS pdr_protectedvm_settingschangeversion, pdr_protectedvm.protectionsettings AS pdr_protectedvm_protectionsettings, pdr_protectedvm.rlschangeversion AS pdr_protectedvm_rlschangeversion, pdr_protectedvm.excludeddevicekey AS pdr_protectedvm_excludeddevicekey, pdr_protectedvm.userprotecteddevice AS pdr_protectedvm_userprotecteddevice, pdr_protectedvm.providerprotecteddevice AS pdr_protectedvm_providerprotecteddevice, pdr_protectedvm.fileinfo AS pdr_protectedvm_fileinfo, pdr_protectedvm.unresolveddevicekey AS pdr_protectedvm_unresolveddevicekey, pdr_protectedvm.peer AS pdr_protectedvm_peer, pdr_protectedvm.operationlockid AS pdr_protectedvm_operationlockid, pdr_protectedvm.drvmidentity AS pdr_protectedvm_drvmidentity, pdv_diskbackinglocator.db_id AS pdv_diskbackinglocator_db_id, pdv_diskbackinglocator.mo_id AS pdv_diskbackinglocator_mo_id, pdv_diskbackinglocator.ref_count AS pdv_diskbackinglocator_ref_count, pdv_diskbackinglocator.typestring AS pdv_diskbackinglocator_typestring, pdv_diskbackinglocator.diskchain AS pdv_diskbackinglocator_diskchain, pdv_diskbackinglocator.datastore AS pdv_diskbackinglocator_datastore, g_do_array.seq_id AS g_do_array_seq_id, g_do_array.db_id AS g_do_array_db_id, g_do_array.array_index AS g_do_array_array_index, g_do_array.ref_count AS g_do_array_ref_count, pdv_deviceinfo.db_id AS pdv_deviceinfo_db_id, pdv_deviceinfo.mo_id AS pdv_deviceinfo_mo_id, pdv_deviceinfo.ref_count AS pdv_deviceinfo_ref_count, pdv_deviceinfo.device AS pdv_deviceinfo_device, pdv_deviceinfo.backing AS pdv_deviceinfo_backing, pdv_deviceinfo.devicekey AS pdv_deviceinfo_devicekey \nFROM pdv_storageproviderdatastorel, pdv_fileinfo, pdv_datastorelocator, pdr_protectedvm, pdv_diskbackinglocator, g_do_array, pdv_deviceinfo \nWHERE pdr_protectedvm.parentgroupmoid LIKE %(parentgroupmoid_1)s AND pdv_diskbackinglocator.datastore LIKE g_do_array.seq_id AND g_do_array.db_id LIKE pdv_datastorelocator.db_id AND (pdr_protectedvm.userprotecteddevice = g_do_array.seq_id AND pdv_deviceinfo.db_id = g_do_array.db_id OR pdr_protectedvm.providerprotecteddevice = g_do_array.seq_id AND pdv_deviceinfo.db_id = g_do_array.db_id) AND pdv_diskbackinglocator.db_id = pdv_deviceinfo.backing AND pdv_datastorelocator.db_id = pdv_storageproviderdatastorel.db_id’ {‘parentgroupmoid_1′: u’vm-protection-group-302450958:dr.replication.VmProtectionGroup:cd9aadcb-de87-45d2-8866-d89dcbfe2128’}
Resolution:

Uninstall python.

Delete folder containing python references.

Download activepython version 2.7.14.2717 from this link.

https://www.activestate.com/activepython/downloads

Set the python path
setx PATH “%PATH%;C:\Python27\Scripts”

NTP check with Powercli

 

Oneliners to speed up troubleshooting common NTP related issues. 

Connect-ViServer <vCenter address> -user administrator@vsphere.local -password VMware123!

Check NTP Server is configured.

Get-VMHost |Sort Name|Select Name, @{N=“NTPServer“;E={$_ |Get-VMHostNtpServer}}, @{N=“ServiceRunning“;E={(Get-VmHostService -VMHost $_ |Where-Object {$_.key-eq “ntpd“}).Running}}

Check timestamp for ESXi host.

foreach($esxcli in get-vmhost|get-esxcli){“”|select @{n=’Time’;e={$esxcli.system.time.get()}},@{n=’hostname’;e={$esxcli.system.hostname.get().hostname}} }

Set time manually on esxi host. 

Directly on host: esxcli system time set -H 03 -m 29 -s 40

H =Hour

m = minute

s = mecond

$esxlist = get-vmhost

foreach ($_ in $esxlist) {$esxcli = Get-EsxCli -VMHost $_; $esxcli.system.time.get();}

 

Set time via powercli

$DevHost.system.time.set.help()

owerCLI C:\TEMPP> $DevHost.system.time.set.help()

=======================================================================================
im.EsxCLI.system.time.set
—————————————————————————————
et the system clock time. Any missing parameters will default to the current time

aram
—————————————————————————————
day | Day

hour | Hour

min | Minute

month | Month

sec | Second

year | Year

 

Example 

 

For all hosts.

$esxlist = get-vmhost

foreach ($_ in $esxlist) {$esxcli = Get-EsxCli -VMHost $_; $esxcli.system.time.get();}

foreach($_ in $esxlist){$esxcli = Get-EsxCli -VMHost $_; $esxcli.system.time.set(10,09,53,01,20,2019); }

/// Parameters are ///

(day,hour,minute,month,second,year)

 

Unable to pair VRMS after PSC certificate update. Thumbprint mismatch on the VR site pairing.

Dump database of both VRMS servers from either site.

cd /opt/vmware/vpostgres/9.3/bin/

For 6.5 VR

cd /opt/vmware/vpostgres/9.5/bin/

./pg_dump -U vrmsdb vrmsdb > vrmsdb.dump to backup the db .  For larger dbs backup to a newly added disk.

echo "SELECT * FROM vmomiserverentity" |  ./psql -U vrmsdb vrmsdb


Snapshot VR appliances with a nonmemory and non quiesced snapshot.

Example update.

./psql -U vrmsdb vrmsdb

update vmomiserverentity set thumbprint = 'C5:A1:31:FA:1F:A5:90:32:90:DD:3E:5F:49:A3:EA:51:79:4C:FE:A3' where dbid = '330';

Repeat for remote site if required.
Reconfigure site pairing. 

Sites should now be paired.

Export IP Customization and convert

This guide will provide a method of exporting VM’s IP customization settings from the SRM database into an XML format using the dr-ip-reporter utility that’s included with all SRM installs and then taking this XML file and converting it into a more user friendly CSV file which can be viewed in EXCEL.

To start with there are a couple of things to note.
–          This will not work if the SRM service is not running.
–          You may need to change the port for the lookup service depending on what version the customer is running. (7444 / 7080 / 443)
–          I don’t know of any way of re-importing this information into the SRM database.
–          This is useful if a customer needs to reinstall SRM with a blank DB and doesn’t have a record of the VM’s IP customization details.

Step 1: Exporting the IP customization settings into XML format from the SRM database using the dr-ip-reporter.exe tool

Connect to the server running the SRM service.

1. Launch command prompt as administrator.
2. Type in:
cd “C:\Program Files\VMware\VMware vCenter Site Recovery Manager\bin”

3. Use the command below to obtain IP customization settings:

Information required:
–          IP Address or FQDN of the PSC
–          A location to store the xml output file

Capture IP details of all Plans:
dr-ip-reporter.exe –cfg “C:\Program Files\VMware\VMware vCenter Site Recovery Manager\config\vmware-dr.xml” –out “C:\Temp\IP-DR.xml” –uri “https://PSC-address:7444/lookupservice/sdk” -i -v

or

Specific Plan:
dr-ip-reporter.exe –cfg “C:\Program Files\VMware\VMware vCenter Site Recovery Manager\config\vmware-dr.xml” –out “C:\Temp\IP-DR.xml” –plan “RP Name” –uri “https://PSC-address:7444/lookupservice/sdk” -i –v

Note: The output of the xml file location is important as this will be used for converting the XML file to CSV. In this example we are outputting it to C:\Temp\IP-DR.xml

Explanation:
–out is the location of the output file
–plan is the name of the single recovery plan you want to export
–uri is the address of the vcenter lookup service

After running the command the user will be prompted for credential to log into the PSC, its recommended to use administrator@vsphere.local but any VC admin or member of the SSO admin group should work.

The XML output file looks like this:

Step 2: Converting the XML file to a CSV file so that it’s more user friendly.

Install VMware vSphere PowerCLI on the SRM server or alternatively copy the XML file to another system where powerCLI is installed. (This may also work in PowerShell but I have not tried)

Create a .ps1 file with the following contents:

Copy from below this line:

$filename = Read-host “Specify path to XML file” 

[xml]$input_file = Get-Content  $filename 
$output = $input_file.DrMappings.ProtectionGroups.ProtectionGroup | Sort Name | ForEach-Object { 
    $protection_group = $_.Name 
    $_.ProtectedVm | Sort Name | ForEach-Object { 
        $vm = $_.Name 
          
        $_.CustomizationSpec | Sort Site | ForEach-Object { 
            $site = $_.Site 
            $ip_settings = $_.ConfigRoot.e.ipSettings 
              
            if ($ip_settings) { 
                $ip_address = $ip_settings.ip.ipAddress 
                $subnetmask = $ip_settings.subnetMask 
                $gateway = $ip_settings.gateway.e.”#text” 
                  
                if ($ip_settings.dnsServerList.e) { 
                    $dns_server = [string]::Join(“,”, ($ip_settings.dnsServerList.e.”#text”)) 
                } 
            } 
  
            “” | Select @{N=”Protection Group”;E={$protection_group}}, 
                        @{N=”VM”;E={$vm}}, 
                        @{N=”Site”;E={$site}}, 
                        @{N=”IP Address”;E={$ip_address}}, 
                        @{N=”SubnetMask”;E={$subnetmask}}, 
                        @{N=”Gateway”;E={$gateway}}, 
                        @{N=”DNS Servers”;E={$dns_server}}
                         
            $ip_address = “”; 
            $subnetmask = “”; 
            $gateway = “”; 
            $dns_server = “”; 
        } 
    } 

  
$output | Export-Csv -UseCulture -NoTypeInformation srm_final_output.csv
.\srm_final_output.csv

Copy from above this line

Note: I’ve also attached this script to this email

Launch PowerCLI and change directory into the folder where the .ps1 script is located as below:

You will notice that I’ve placed the script in the same folder as I output the XML file from the previous step for ease of use.

Next we need to run the script to convert XML to CSV.

.\xmltocsv.ps1

After running the script a prompt will be returned. Enter the path to the XML file from Step 1. In my case this is C:\Temp\IP.xml and hit enter.

This creates a csv file in the same directory with the name srm_final_output.csv

Open this file in Excel to see the IP customization details note that VMs that do not have IP customization settings set will have blank values:

Information displayed:
Protection group name
VM Name
Site 1 / 2 or Production / DR
IP address
Subnet Mask
Gateway
DNS Servers

Some issues I’ve encountered:
–          Depending on the version of SRM the XML output may be in XML version 1.0 or 2.0. The powerCLI script will not work with XML version 2.0

Open the XML file and change the version from 2.0 to 1.0:

How to troubleshoot unable to pair vSphere Replication Servers 6.X.

By default we do not give debug logging in the webclient for vSphere Replication site pairing.   A really good way to find root causes of site pairing is to follow the below steps.
<logger level=”DEBUG” additivity=”false” name=”com.vmware.vim.vmomi.client.http”>
      <appender-ref ref=”LOG_FILE” />
   </logger>

 

 

Add that snippet to the bottom of the security-config.xml

  • vCenter Server on Windows – Inventory service install location/lib/server/config/
  • vCenter Server Appliance – /usr/lib/vmware-vpx/inventoryservice/lib/server/config/

 

Then reverse lookup the error in the virgo logs.

Remove Stale SRM Entries from PSC

Step 1. Find out from vCenter where is my PSC
/usr/lib/vmware-vmafd/bin/vmafd-cli get-ls-location --server-name localhost

C:\Program Files\VMware\vCenter Server\vmafdd\vmafd-cli get-site-name --server-name localhost

/usr/lib/vmidentity/tools/scripts/lstool.py list --url http://localhost:7080/lookupservice/sdk > /tmp/psc_services.txt

grep -i vrms -A1 -B2 psc_services.txt
grep -i h5-dr -A1 -B2 psc_services.txt
grep -i vrUi -A1 -B2 psc_services.txt
grep -i vcdr -A1 -B2 psc_services.txt
grep -i vrms -A1 -B2 psc_services.txt
Step 2. 
/usr/lib/vmidentity/tools/scripts/lstool.py list --url http://localhost:7080/lookupservice/sdk --type VCDR

 "%VMWARE_PYTHON_BIN%" "%VMWARE_CIS_HOME%\VMware Identity Services\lstool\scripts\lstool.py" list --url http://localhost:7080/lookupservice/sdk --type VCDR

Output to a text log if you prefer

/usr/lib/vmidentity/tools/scripts/lstool.py list --url http://localhost:7080/lookupservice/sdk --type VCDR > VRPSC.txt

"%VMWARE_PYTHON_BIN%" "%VMWARE_CIS_HOME%\VMware Identity Services\lstool\scripts\lstool.py" list --url http://localhost:7080/lookupservice/sdk type VCDR  > c:\psc_services.txt 

Output should look like this.

Name: vCenter DR service (vCenter-dr)        Service Product: com.vmware.dr        Service Type: vcDr        Service ID: 47a38601-09dc-4b24-8c60-c80c93a90e31        Site ID: site-1        Node ID: 80b10878-ffb4-11e7-aa05-000c298c82ea        Owner ID: SRM-47a38601-09dc-4b24-8c60-c80c93a90e31@vsphere.local        Version: 6.5.1        Endpoints:                Type: com.vmware.dr.extApi                Protocol: vmomi                URL: https://192.168.1.152:9086/vcdr/extapi/sdk                SSL trust: MIIDVTCCAj2gAwIBAgIJAOwNcvFh5SwWMA0GCSqGSIb3DQEBCwUAMF4xLTArBgNVBAoMJFZNd2FyZSB2Q2VudGVyIFNpdGUgUmVjb3ZlcnkgTWFuYWdlcjEtMCsGA1UECwwkVk13YXJlIHZDZW50ZXIgU2l0ZSBSZWNvdmVyeSBNYW5hZ2VyMB4XDTE4MDEyNDEyNTYyN1oXDTIzMDEyMzEyNTYyN1owNDEMMAoGA1UECgwDdm13MQwwCgYDVQQLDAN2bXcxFjAUBgNVBAMMDTE5Mi4xNjguMS4xNTIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDH+4hV/uqF892kMaNvBY36Py/VTF7vNOV7QpuQhfOzMjW0JMbSxRKuOLvc8UggEXAmqOi26s0Z3ZlDIXeCtJ9sk4YM3TsNRnkGWcgwBUR9LpSOyKFbiy4/WLnPM+Iw7w2oluPpvfSVvl7lH760ivqOdJ/o3UPI26T3SGDWhT8wtD9jeW1vbMhEke8ERuU8Y9yEICA/JEqhezf9KGKT/e4P6ZEAT7KzUKJvmRu+0L8OsdENyfpwgo/6nxTslcnf1JJG+R7sEOd3iUpppLiq2TX8ofb0Oq/epoJfgw2pFxifNZBgdbYZQjURWLCuO5ryiEirI5fRCnU8y1Da9nmfDzxzAgMBAAGjQDA+MAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMBMGA1UdJQQMMAoGCCsGAQUFBwMBMA8GA1UdEQQIMAaHBMCoAZgwDQYJKoZIhvcNAQELBQADggEBAC5p5zsiVmIiu1bzzmt4M+RiqJm0rS6NEjIIts6u8a/pfsnIE8iHTwmG6UDwIVK6CVI+zPa7Ido0KVUXekDk456PT+3xblj5m++M9WFu4Yc0WRW9e5Y1BubZoyfeLgZhk2egCz+xtAR1UZvqUMNrrGF0mrm5TxXw1vqwlU2kiauBMgtrb7w7qS7VBtFXYFGGbXtoTCA8EuiPqZuAZMXCqsBl0y2niORL+PPa96RlYwF+8WpDtyfKwezcu6YeSGhFeG36VRuoDclga5hCUFAPRY6YAlgK7qmXyYcM7mPuY/Wvkz7G+hWTAbiu3wG1adXeAf0hR7JIYungF/AVIBVwUms=                Type: com.vmware.dr.vcDr                Protocol: vmomi                URL: https://192.168.1.152:9086/vcdr/vmomi/sdk                SSL trust: MIIDVTCCAj2gAwIBAgIJAOwNcvFh5SwWMA0GCSqGSIb3DQEBCwUAMF4xLTArBgNVBAoMJFZNd2FyZSB2Q2VudGVyIFNpdGUgUmVjb3ZlcnkgTWFuYWdlcjEtMCsGA1UECwwkVk13YXJlIHZDZW50ZXIgU2l0ZSBSZWNvdmVyeSBNYW5hZ2VyMB4XDTE4MDEyNDEyNTYyN1oXDTIzMDEyMzEyNTYyN1owNDEMMAoGA1UECgwDdm13MQwwCgYDVQQLDAN2bXcxFjAUBgNVBAMMDTE5Mi4xNjguMS4xNTIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDH+4hV/uqF892kMaNvBY36Py/VTF7vNOV7QpuQhfOzMjW0JMbSxRKuOLvc8UggEXAmqOi26s0Z3ZlDIXeCtJ9sk4YM3TsNRnkGWcgwBUR9LpSOyKFbiy4/WLnPM+Iw7w2oluPpvfSVvl7lH760ivqOdJ/o3UPI26T3SGDWhT8wtD9jeW1vbMhEke8ERuU8Y9yEICA/JEqhezf9KGKT/e4P6ZEAT7KzUKJvmRu+0L8OsdENyfpwgo/6nxTslcnf1JJG+R7sEOd3iUpppLiq2TX8ofb0Oq/epoJfgw2pFxifNZBgdbYZQjURWLCuO5ryiEirI5fRCnU8y1Da9nmfDzxzAgMBAAGjQDA+MAkGA1UdEwQCMAAwCwYDVR0PBAQDAgSwMBMGA1UdJQQMMAoGCCsGAQUFBwMBMA8GA1UdEQQIMAaHBMCoAZgwDQYJKoZIhvcNAQELBQADggEBAC5p5zsiVmIiu1bzzmt4M+RiqJm0rS6NEjIIts6u8a/pfsnIE8iHTwmG6UDwIVK6CVI+zPa7Ido0KVUXekDk456PT+3xblj5m++M9WFu4Yc0WRW9e5Y1BubZoyfeLgZhk2egCz+xtAR1UZvqUMNrrGF0mrm5TxXw1vqwlU2kiauBMgtrb7w7qS7VBtFXYFGGbXtoTCA8EuiPqZuAZMXCqsBl0y2niORL+PPa96RlYwF+8WpDtyfKwezcu6YeSGhFeG36VRuoDclga5hCUFAPRY6YAlgK7qmXyYcM7mPuY/Wvkz7G+hWTAbiu3wG1adXeAf0hR7JIYungF/AVIBVwUms=        Attributes:                supportedVmodlVersions: /vcdr/vmomi/drserviceversions.xml                vcInstanceUuid: 0275943f-0310-4db2-8af8-52a91886c61c                vmodlVersion: /vcdr/vmomi/drversion.xml                extensionKey: com.vmware.vcDr-------------------------------------------------------        Name: vCenter DR service (vCenter-dr)        Service Product: com.vmware.dr        Service Type: vcDr        Service ID: 19e3917f-7dd8-4c48-8fd0-403402a7a97d        Site ID: site-1        Node ID: faa9b94a-0018-11e8-8667-000c29d9d620        Owner ID: SRM-19e3917f-7dd8-4c48-8fd0-403402a7a97d@vsphere.local        Version: 6.5.1        Endpoints:                Type: com.vmware.dr.vcDr                Protocol: vmomi                URL: https://192.168.2.57:9086/vcdr/vmomi/sdk                SSL trust: MIIDVDCCAjygAwIBAgIJAIvOJ13CmKANMA0GCSqGSIb3DQEBCwUAMF4xLTArBgNVBAoMJFZNd2FyZSB2Q2VudGVyIFNpdGUgUmVjb3ZlcnkgTWFuYWdlcjEtMCsGA1UECwwkVk13YXJlIHZDZW50ZXIgU2l0ZSBSZWNvdmVyeSBNYW5hZ2VyMB4XDTE4MDEyNDEzNDkxNFoXDTIzMDEyMzEzNDkxNFowMzEMMAoGA1UECgwDdm13MQwwCgYDVQQLDAN2bXcxFTATBgNVBAMMDDE5Mi4xNjguMi41NzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANdjmw/13HAbeMvBguc9t9P3hmi3ijmw/GMnyz3tj7Jm/QoaBls94Tu0uhSb6bTeS3laPMx7cUAn35WP/KPtVrrv7fIwAtP4wBDXRhQ1YLbs8vLErVXaCgdHX/PLBSq8Qba7zkGFwUNTCncaeC7dSR8zYIBekLsWQ5N4BrVQwrgz+WdOKJfryt1uQw4ZSVt+bsq23FBbV42LtGmOpIe0h7ptfQ+j0YmqXJtap/BqBMHGny/P4Xbufv+yTexERqJXp6u4ub7PJzGgtVCaD07aTltWqsVAFJLOtBlx1jVwPOxzdI9OOtpueimWCphChFUgN3e2ML9PlVtXzHLuFOaB70ECAwEAAaNAMD4wCQYDVR0TBAIwADALBgNVHQ8EBAMCBLAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwDwYDVR0RBAgwBocEwKgCOTANBgkqhkiG9w0BAQsFAAOCAQEAQE8Wlco9B3q5znHxLnH5uvFptSYbsEftNUP6wC0CVQOiLOCVn9phO3EIDV3Su06IDN2UCWbxwYWmLeyKTqCiqbDLl+cwzaoenw5BE9yca0dLiwEhqa9KJ9SnH3LMX8bTNlNaXcYuHvh+WFLYRdud6zLqwrKMDXLI5E1AMMZpG0q0Fx+ALpiZ0xlJVSjRrVnMPdDw5NHwzPRPFybeE3jBhBumnajWRTBDKZBEcPSJ+o4DlMgadltXcEZO6/1yFwyb6p4yg2TKG3rKIP+6fv0etRN30EnVXtzJm3r+8953x5ED9R7H6No+XH1wS3l6hY81mQPE+yDrwxN45F13pXe0EQ==                Type: com.vmware.dr.extApi                Protocol: vmomi                URL: https://192.168.2.57:9086/vcdr/extapi/sdk                SSL trust: MIIDVDCCAjygAwIBAgIJAIvOJ13CmKANMA0GCSqGSIb3DQEBCwUAMF4xLTArBgNVBAoMJFZNd2FyZSB2Q2VudGVyIFNpdGUgUmVjb3ZlcnkgTWFuYWdlcjEtMCsGA1UECwwkVk13YXJlIHZDZW50ZXIgU2l0ZSBSZWNvdmVyeSBNYW5hZ2VyMB4XDTE4MDEyNDEzNDkxNFoXDTIzMDEyMzEzNDkxNFowMzEMMAoGA1UECgwDdm13MQwwCgYDVQQLDAN2bXcxFTATBgNVBAMMDDE5Mi4xNjguMi41NzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANdjmw/13HAbeMvBguc9t9P3hmi3ijmw/GMnyz3tj7Jm/QoaBls94Tu0uhSb6bTeS3laPMx7cUAn35WP/KPtVrrv7fIwAtP4wBDXRhQ1YLbs8vLErVXaCgdHX/PLBSq8Qba7zkGFwUNTCncaeC7dSR8zYIBekLsWQ5N4BrVQwrgz+WdOKJfryt1uQw4ZSVt+bsq23FBbV42LtGmOpIe0h7ptfQ+j0YmqXJtap/BqBMHGny/P4Xbufv+yTexERqJXp6u4ub7PJzGgtVCaD07aTltWqsVAFJLOtBlx1jVwPOxzdI9OOtpueimWCphChFUgN3e2ML9PlVtXzHLuFOaB70ECAwEAAaNAMD4wCQYDVR0TBAIwADALBgNVHQ8EBAMCBLAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwDwYDVR0RBAgwBocEwKgCOTANBgkqhkiG9w0BAQsFAAOCAQEAQE8Wlco9B3q5znHxLnH5uvFptSYbsEftNUP6wC0CVQOiLOCVn9phO3EIDV3Su06IDN2UCWbxwYWmLeyKTqCiqbDLl+cwzaoenw5BE9yca0dLiwEhqa9KJ9SnH3LMX8bTNlNaXcYuHvh+WFLYRdud6zLqwrKMDXLI5E1AMMZpG0q0Fx+ALpiZ0xlJVSjRrVnMPdDw5NHwzPRPFybeE3jBhBumnajWRTBDKZBEcPSJ+o4DlMgadltXcEZO6/1yFwyb6p4yg2TKG3rKIP+6fv0etRN30EnVXtzJm3r+8953x5ED9R7H6No+XH1wS3l6hY81mQPE+yDrwxN45F13pXe0EQ==        Attributes:                supportedVmodlVersions: /vcdr/vmomi/drserviceversions.xml                vcInstanceUuid: 3523f7ee-7eca-4ae4-9b99-c51026056869                vmodlVersion: /vcdr/vmomi/drversion.xml                extensionKey: com.vmware.vcDr
Step 3. 

To remove the stale entries run this command with the Service ID you want remove.

/usr/lib/vmidentity/tools/scripts/lstool.py unregister --url http://localhost:7080/lookupservice/sdk --id 47a38601-09dc-4b24-8c60-c80c93a90e31 --user 'administrator@vsphere.local' --password 'VMware123!' --no-check-cert




"%VMWARE_PYTHON_BIN%" "%VMWARE_CIS_HOME%\VMware Identity Services\lstool\scripts\lstool.py" unregister --url http://localhost:7080/lookupservice/sdk --id Service_ID from Step 4--user "administrator@vsphere.local" --password "administrator_password"--no-check-cert




For example:




"%VMWARE_PYTHON_BIN%" "%VMWARE_CIS_HOME%\VMware Identity Services\lstool\scripts\lstool.py" unregister --url http://localhost:7080/lookupservice/sdk --id TMP:608BF497-A198-40C1-9866-545533A488BE --user "administrator@vsphere.local" --password "VMware123!" --no-check-cert 




Step 4. 
Check that it has removed. 

Go to step 2

Warning : Copy to notepad first and paste to command prompt.  If commands fail type manually or copy from output.  "%VMWARE_PYTHON_BIN%" lstool.py

VR failed upgrade no Snapshot to restore

 

Step 1. Download and install the java jre for vSphere Replication

 

https://buildweb.eng.vmware.com/ob/4540178/#
download vmware-jre-1.7.0_121-1server.x86_64.rpm

 

This rpm can also be pulled from the vSphere Replication installer ISO using 7z.  

 

install it and run following commands
mkdir /usr/java
rm -f /usr/java/latest /usr/java/default
ln -s /usr/java/jre-vmware /usr/java/latest
ln -s /usr/java/latest /usr/java/default
rm -f /usr/bin/java
ln -s /usr/java/default/bin/java /usr/bin/java

rpm -ivh vmware-jre-1.7.0_121-1server.x86_64.rpm

Step 2. Download and install the VAMI.

Check the VAMI version.

rpm -qa | grep studio

vmware-studio-appliance-config-2.6.0.9-160505024738
vmware-studio-vami-service-network-2.6.0.9-0
vmware-studio-init-2.6.0.9-160505024738
vmware-studio-vami-login-2.6.0.9-3609964
vmware-studio-vami-tools-2.6.0.9-3609964
vmware-studio-vami-servicebase-2.6.0.9-3609964
vmware-studio-vami-service-hms-6.1.1.13216-3849281
vmware-studio-vami-service-system-2.6.0.9-0
vmware-studio-vami-service-core-2.6.0.9-0
vmware-studio-vami-lighttpd-2.6.0.9-3609964
vmware-studio-vami-service-update-2.6.0.9-0

Step 3. Download and install the VAMI in this case

rpm -ivh vmware-studio-vami-cimom_2.6.0.9-3609964_x86_64.rpm

 

http://build-squid.eng.vmware.com/build/mts/release/bora-4595498/publish/exports/Update_Repo/package-pool/vmware-studio-vami-cimom_2.6.0.9-3609964_x86_64.rpm

 

This rpm can also be pulled from the vSphere Replication installer ISO.  

Reboot VR

 

Step 4. VAMI now available. https://kb.vmware.com/s/article/2149560

Step 5 VR Upgrade to 6.1.2.0 After this plugin not showing. 

Step 6. Upgrade to 6.1.2.1 which has the plugin issue resolved. 

Step 7. plugin appears. NTP service not running on the appliance. Followed this kb to resolve issue.

To continue configuring the NTP server:
Log in to the vSphere Replication appliance console as the root user.
Run the command:
yast2 ntp-client add server=your_chosen_time_server

This console error can be ignored:

Error: Cannot update the dynamic configuration policy.

Run this command:
yast2 ntp-client enable

Run this command:
sntp –P no -r your_chosen_time_server

The ntp.conf file is updated and the NTP synchronization is configured successfully.
Refresh the vSphere Web client.