[RFC] Save/restore state design proposal

[lauralt]

No description provided.

[jiangliu]

That means we need to move versionize crate to rust vmm, and get an agreement about data structures versions?

[lauralt]

Moving the versionize crate to rust-vmm shouldn't be required, but yes we will need to get an agreement on the structures versions. Other option would be to not have the explicit versionize support in rust-vmm crates, and let the VMMs define their own Versionize structures.

[jiangliu]

Moving the versionize crate to rust-vmm shouldn't be required, but yes we will need to get an agreement on the structures versions. Other option would be to not have the explicit versionize support in rust-vmm crates, and let the VMMs define their own Versionize structures.

It's not easy to cooperate several vmms about the data structure versioning.

[jiangliu]

As we have offline discussed with @lauralt, the major challenge is to cooperate on the data structure version numbering. One possible solution is:

1. Each library crate defines version numbers for its data structures
2. the app consolidates all data version numbers of library crates into an app version number
3. create companion xxx-state crate library crate xxx. The clients may choose to use the upstream xxx-state crate or implement its own version. This also avoids dependency explosion for clients not using the capability.
4. it would be better to move versionize crate into rust-vmm project.

[lauralt]

@jiangliu thanks! I just opened a PR with the RtcState implementation to better showcase the proposal.

[liuw]

How do you plan to enforce this standard to all the components? Is there a way to mechanically generate the required code snippets?

I was hoping you introduced some traits that all components must implement to conform to the standard, which also enables programmatically dealing with states from VMM's point of view, like looping through a list of trait objects to retrieve or set their states. But looking at the example PR it looks manual and bespoke to each component.

[russell-islam]

What happens if the contents in the states are different across hypervisors for example KVM states are a little bit different than Microsoft Hypervisor?

[lauralt]

How do you plan to enforce this standard to all the components? Is there a way to mechanically generate the required code snippets?

I was hoping you introduced some traits that all components must implement to conform to the standard, which also enables programmatically dealing with states from VMM's point of view, like looping through a list of trait objects to retrieve or set their states. But looking at the example PR it looks manual and bespoke to each component.

I'm not sure it is possible to programmatically enforce this standard, I assume even if we'd add a trait, the component would still not be forced to implement it.
We didn't start with adding a state trait for simplicity reasons. But we can do that later when we find it necessary/useful (maybe for virtio objects). The current proposal is just a starting point.

[liuw]

How do you plan to enforce this standard to all the components? Is there a way to mechanically generate the required code snippets?
I was hoping you introduced some traits that all components must implement to conform to the standard, which also enables programmatically dealing with states from VMM's point of view, like looping through a list of trait objects to retrieve or set their states. But looking at the example PR it looks manual and bespoke to each component.

I'm not sure it is possible to programmatically enforce this standard, I assume even if we'd add a trait, the component would still not be forced to implement it.

Exactly. I think the only way to "force" the adoption is to have a nice framework that provides a lot of goodies when you implement those traits.

We didn't start with adding a state trait for simplicity reasons. But we can do that later when we find it necessary/useful (maybe for virtio objects). The current proposal is just a starting point.

Sure. This sounds fair.

[lauralt]

What happens if the contents in the states are different across hypervisors for example KVM states are a little bit different than Microsoft Hypervisor?

Hmm, I think this one belongs to the broader discussion of how the interfaces will look like in vm-vcpu. We can decide at that point where/how to define the state and the setter/getter for it. I can't think of any blocker with the current proposal for the vcpu use case, but maybe I am missing something.

[polarathene]

As someone familiar with using QEMU and VirtManager (or similar) where a VM guest could be saved and it's state later restored, does this RFC imply rust-vmm presently lacks performing that equivalent functionality?

Beyond that, I'm trying to seek more information on what OSS equivalent components I should be engaging with to help push forward support for a feature VMware has long had, the ability to snapshot a VM guest that has 3D acceleration enabled. I'm not sure how VMware achieved that, just that anyone I've interacted with thus far doesn't seem confident about how to proceed with implementing that feature.

If someone could point me in the right direction to focus on for attempting to build a proof of concept, I'd appreciate that. I am aware of CRIU support for CUDA/ROCm and QT6 in Wayland having similar functionality, I'm specifically interested in whatever approach VMware is doing for the guest, rather than solutions that also work on a host.