Vr stabilization

[baltoche]

Problem your feature solves? Please describe.
The HMD mode feels very laggy

Describe the solution you'd like
New frames should be fixed in space instead of following the headset

Additional context
This could be achieved by adding the transform information as metadata with every frame So that it can be projected on a sphere with the view position at it's center. This way the headset would interpolate naturally between frames.

[lukehb]

We would be happy to review a PR that made these changes and try it out.

In the described rendering setup would the frame be projected onto one of the hemispheres of the sphere or something else?

A diagram may help us to understand the proposed solution?