-
Notifications
You must be signed in to change notification settings - Fork 162
guide: updates to openhcl arch and boot docs #2594
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR refactors and reorganizes the OpenHCL architecture documentation. The documentation has been restructured from a flat organization to a nested directory structure under Guide/src/reference/architecture/openhcl/, improving navigation and separating concerns.
Key changes:
- Split the monolithic boot process document into focused pages covering processes/components, boot flow, sidecar architecture, and IGVM artifact details
- Enhanced the boot flow documentation with a new Mermaid sequence diagram visualizing the entire boot process
- Streamlined the main OpenHCL architecture page to focus on high-level concepts with links to detailed pages
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| Guide/src/reference/architecture/openhcl_boot.md | Deleted the original boot process documentation (content migrated to new structure) |
| Guide/src/reference/architecture/igvm.md | Deleted the original IGVM overview (content moved to openhcl/igvm.md) |
| Guide/src/reference/architecture/openhcl/processes.md | New file documenting OpenHCL processes and components (Boot Shim, kernels, paravisor, workers, diagnostics) |
| Guide/src/reference/architecture/openhcl/boot.md | New file with enhanced boot flow documentation including a sequence diagram and step-by-step explanations |
| Guide/src/reference/architecture/openhcl/sidecar.md | New file dedicated to the x86_64 sidecar kernel architecture and its purpose |
| Guide/src/reference/architecture/openhcl/igvm.md | New file describing the IGVM artifact format and package contents |
| Guide/src/reference/architecture/openhcl.md | Updated main architecture page to focus on overview and scenarios, with links to detailed component pages |
| Guide/src/SUMMARY.md | Updated table of contents to reflect the new nested documentation structure |
Comments suppressed due to low confidence (1)
Guide/src/reference/architecture/openhcl/boot.md:116
- The term is inconsistently capitalized throughout the document. In some places it appears as "Device Tree" (capitalized) and in others as "device tree" (lowercase). For consistency, consider using lowercase "device tree" throughout, as this is the more common convention in Linux kernel documentation unless referring to the Device Tree Specification formally.
* **Host Device Tree:** A device tree provided by the host containing topology and resource information.
* **Command Line:** It parses the kernel command line, which can be supplied via IGVM or the host device tree.
3. **Device Tree:** It constructs a Device Tree that describes the hardware topology (CPUs, memory) to the Linux kernel.
4. **Sidecar Setup (x86_64):** The shim determines which CPUs will run Linux (typically just the BSP) and which will run the Sidecar (APs). It sets up control structures and directs Sidecar CPUs to the Sidecar entry point.
* **Sidecar Entry:** "Sidecar CPUs" jump directly to the Sidecar kernel entry point instead of the Linux kernel.
* **Dispatch Loop:** These CPUs enter a lightweight dispatch loop, waiting for commands.
5. **Kernel Handoff:** Finally, the BSP (and any Linux APs) jumps to the Linux kernel entry point, passing the Device Tree and command line arguments.
## 3. Linux Kernel Boot
The **Linux Kernel** takes over on the BSP and initializes the operating system environment. Sidecar CPUs remain in their dispatch loop until needed (e.g., hot-plugged for Linux tasks).
1. **Kernel Init:** The kernel initializes its subsystems (memory, scheduler, etc.).
2. **Driver Init:** It loads drivers for the paravisor hardware and standard devices.
3. **Root FS:** It mounts the initial ramdisk (initrd) as the root filesystem.
4. **Expose DT:** It exposes the boot-time Device Tree to userspace (e.g., under `/proc/device-tree`) for early consumers.
4. **User Space:** It spawns the first userspace process, `underhill_init` (PID 1).
## 4. Userspace Initialization (`underhill_init`)
`underhill_init` prepares the userspace environment.
1. **Filesystems:** It mounts essential pseudo-filesystems like `/proc`, `/sys`, and `/dev`.
2. **Environment:** It sets up environment variables and system limits.
3. **Exec:** It replaces itself with the main paravisor process, `/bin/openvmm_hcl`.
## 5. Paravisor Startup (`openvmm_hcl`)
The **Paravisor** process (`openvmm_hcl`) starts and initializes the virtualization services.
1. **Config Discovery:** It reads the system topology and configuration from `/proc/device-tree` and other kernel interfaces.
2. **Service Init:** It initializes internal services, such as the VTL0 management logic and host communication channels.
3. **Worker Spawn:** It spawns the **VM Worker** process (`underhill_vm`) to handle the high-performance VM partition loop.
## 6. VM Execution
At this point, the OpenHCL environment is fully established.
The `underhill_vm` process runs the VTL0 guest, handling exits and emulating devices, while `openvmm_hcl` manages the overall policy and communicates with the host.
|
|
||
| VTLs can be backed by: | ||
| - **VTL2:** OpenHCL runs here[^sk]. It has higher privileges and is isolated from VTL0. | ||
| - **VTL0:** The Guest OS (e.g., Windows, Linux) runs here. It cannot access VTL2 memory or resources. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe include VTL1 in this list instead of as a footnote, since from our perspective they're both just 'guest' VTLs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I made a reference to VTL1 here. I think we want to keep this focused on VTL0 and VTL2, which are the most relevant. (right?)
| 1. **Hardware Init:** The shim initializes the CPU state and memory management unit (MMU). | ||
| 2. **Config Parsing:** It parses configuration from multiple sources: | ||
| * **IGVM Parameters:** Fixed parameters provided by the host that were generated at IGVM build time. | ||
| * **IGVM Parameters:** Fixed parameters encoded into the measured section of the IGVM image, loaded by the host. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IGVM Parameters have a specific meaning in the IGVM spec, IE they mean host-provided dynamic values. This should instead say Measured Parameters and perhaps the DT & command line should be under a new Dynamic Parameters section?
This is a follow up to #2577. Here, I clean up the OpenHCL architecture page. I introduce a separate page that describes the processes and components. This allows me to streamline the boot process page. I've also added a sequence diagram.
All written with the help of CoPilot. I've edited this and added some stuff manually, as well.