Skip to content

Conversation

@brainexe
Copy link
Member

@brainexe brainexe commented Oct 11, 2023

I got some very rare errors during igvm migrate when two migrations of the same source hypervisor finished at the same second with the same CPU value:


INFO: igvm.commands: Picked hv-destination.local as destination Hypervisor
[hv-destination.local] sudo: while [ ! -L '/dev/xen-data/469788_vm-name' ]; do sleep 1; done
INFO: igvm.kvm: Starting online migration of vm vm-name from hv-source.local to hv-destination.local
INFO: igvm.kvm: Waiting for migration stats to show up
INFO: igvm.kvm: Migration progress: disk 0% 0/0MiB, memory 0% 0/0MiB,
INFO: igvm.kvm: Migration progress: disk 0% 0/0MiB, memory 0% 0/0MiB,
INFO: igvm.kvm: Migration progress: disk 2% 305/17408MiB, memory 0% 0/0MiB,
...
INFO: igvm.kvm: Migration progress: disk 100% 17410/17410MiB, memory 93% 2869/3073MiB,
libvirt: Domain Config error : Requested operation is not valid: domain is not running
INFO: igvm.kvm: Awaiting migration to finish
INFO: igvm.kvm: Migration finished
INFO: igvm.transaction: Rolling back transaction
libvirt:  error : internal error: Child process (/sbin/lvchange -aln /dev/xen-data/469788_vm-name) unexpected exit status 5:   Logical volume xen-data/469788_vm-name in use.
WARNING: igvm.transaction: Rollback action "destroy storage" failed: internal error: Child process (/sbin/lvchange -aln /dev/xen-data/469788_vm-name) unexpected exit status 5:   Logical volume xen-data/469788_vm-name in use.

Disconnecting from hv-destination.local... done.
Traceback (most recent call last):
  File "/usr/bin/igvm", line 33, in <module>
    sys.exit(load_entry_point('igvm==2.2.0', 'console_scripts', 'igvm')())
  File "/usr/lib/python3/dist-packages/igvm/cli.py", line 496, in main
    args.pop('func')(**args)
  File "/usr/lib/python3/dist-packages/igvm/host.py", line 25, in decorator
    return fn(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/igvm/commands.py", line 531, in vm_migrate
    previous_hypervisor.log_migration(_vm, '-')
  File "/usr/lib/python3/dist-packages/igvm/hypervisor.py", line 1157, in log_migration
    self.dataset_obj.commit()
  File "/usr/lib/python3/dist-packages/adminapi/dataset.py", line 488, in commit
    _handle_exception(result)
  File "/usr/lib/python3/dist-packages/adminapi/dataset.py", line 574, in _handle_exception
    raise exception_class(result['message'])
adminapi.exceptions.DatasetError: {'__all__': ['Server string attribute with this Server, Attribute and Value already exists.']}

Note:

  • adapt the regexp for the SA attribute to something like \A([0-9]{10}|\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d+)\s[-|+][0-9]{1,3}\Z
  • link needed innogames/serveradmin changes

src_hv = self.vm.hypervisor.dataset_obj['hostname']
cpu_usage_vm_src = self.vm.hypervisor.estimate_vm_cpu_usage(self.vm)
timestamp = 1234567890
timestamp = 1234567890000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this test timestamp be updated to the new format too, in case we ever restrict in Serveradmin?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants