Skip to content

Conversation

@mattkur
Copy link
Contributor

@mattkur mattkur commented Dec 19, 2025

This is a follow up to #2577. Here, I clean up the OpenHCL architecture page. I introduce a separate page that describes the processes and components. This allows me to streamline the boot process page. I've also added a sequence diagram.

All written with the help of CoPilot. I've edited this and added some stuff manually, as well.

  • guide: refactor architecture / boot / processes docs
  • guide: enhance boot flow documentation and clarify configuration sources

Copilot AI review requested due to automatic review settings December 19, 2025 21:11
@mattkur mattkur requested a review from a team as a code owner December 19, 2025 21:11
@github-actions github-actions bot added the Guide label Dec 19, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors and reorganizes the OpenHCL architecture documentation. The documentation has been restructured from a flat organization to a nested directory structure under Guide/src/reference/architecture/openhcl/, improving navigation and separating concerns.

Key changes:

  • Split the monolithic boot process document into focused pages covering processes/components, boot flow, sidecar architecture, and IGVM artifact details
  • Enhanced the boot flow documentation with a new Mermaid sequence diagram visualizing the entire boot process
  • Streamlined the main OpenHCL architecture page to focus on high-level concepts with links to detailed pages

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Guide/src/reference/architecture/openhcl_boot.md Deleted the original boot process documentation (content migrated to new structure)
Guide/src/reference/architecture/igvm.md Deleted the original IGVM overview (content moved to openhcl/igvm.md)
Guide/src/reference/architecture/openhcl/processes.md New file documenting OpenHCL processes and components (Boot Shim, kernels, paravisor, workers, diagnostics)
Guide/src/reference/architecture/openhcl/boot.md New file with enhanced boot flow documentation including a sequence diagram and step-by-step explanations
Guide/src/reference/architecture/openhcl/sidecar.md New file dedicated to the x86_64 sidecar kernel architecture and its purpose
Guide/src/reference/architecture/openhcl/igvm.md New file describing the IGVM artifact format and package contents
Guide/src/reference/architecture/openhcl.md Updated main architecture page to focus on overview and scenarios, with links to detailed component pages
Guide/src/SUMMARY.md Updated table of contents to reflect the new nested documentation structure
Comments suppressed due to low confidence (1)

Guide/src/reference/architecture/openhcl/boot.md:116

  • The term is inconsistently capitalized throughout the document. In some places it appears as "Device Tree" (capitalized) and in others as "device tree" (lowercase). For consistency, consider using lowercase "device tree" throughout, as this is the more common convention in Linux kernel documentation unless referring to the Device Tree Specification formally.
    * **Host Device Tree:** A device tree provided by the host containing topology and resource information.
    * **Command Line:** It parses the kernel command line, which can be supplied via IGVM or the host device tree.
3. **Device Tree:** It constructs a Device Tree that describes the hardware topology (CPUs, memory) to the Linux kernel.
4. **Sidecar Setup (x86_64):** The shim determines which CPUs will run Linux (typically just the BSP) and which will run the Sidecar (APs). It sets up control structures and directs Sidecar CPUs to the Sidecar entry point.
    * **Sidecar Entry:** "Sidecar CPUs" jump directly to the Sidecar kernel entry point instead of the Linux kernel.
    * **Dispatch Loop:** These CPUs enter a lightweight dispatch loop, waiting for commands.
5. **Kernel Handoff:** Finally, the BSP (and any Linux APs) jumps to the Linux kernel entry point, passing the Device Tree and command line arguments.

## 3. Linux Kernel Boot

The **Linux Kernel** takes over on the BSP and initializes the operating system environment. Sidecar CPUs remain in their dispatch loop until needed (e.g., hot-plugged for Linux tasks).

1. **Kernel Init:** The kernel initializes its subsystems (memory, scheduler, etc.).
2. **Driver Init:** It loads drivers for the paravisor hardware and standard devices.
3. **Root FS:** It mounts the initial ramdisk (initrd) as the root filesystem.
4. **Expose DT:** It exposes the boot-time Device Tree to userspace (e.g., under `/proc/device-tree`) for early consumers.
4. **User Space:** It spawns the first userspace process, `underhill_init` (PID 1).

## 4. Userspace Initialization (`underhill_init`)

`underhill_init` prepares the userspace environment.

1. **Filesystems:** It mounts essential pseudo-filesystems like `/proc`, `/sys`, and `/dev`.
2. **Environment:** It sets up environment variables and system limits.
3. **Exec:** It replaces itself with the main paravisor process, `/bin/openvmm_hcl`.

## 5. Paravisor Startup (`openvmm_hcl`)

The **Paravisor** process (`openvmm_hcl`) starts and initializes the virtualization services.

1. **Config Discovery:** It reads the system topology and configuration from `/proc/device-tree` and other kernel interfaces.
2. **Service Init:** It initializes internal services, such as the VTL0 management logic and host communication channels.
3. **Worker Spawn:** It spawns the **VM Worker** process (`underhill_vm`) to handle the high-performance VM partition loop.

## 6. VM Execution

At this point, the OpenHCL environment is fully established.
The `underhill_vm` process runs the VTL0 guest, handling exits and emulating devices, while `openvmm_hcl` manages the overall policy and communicates with the host.

@github-actions
Copy link

@mattkur mattkur requested a review from chris-oo December 24, 2025 01:34

VTLs can be backed by:
- **VTL2:** OpenHCL runs here[^sk]. It has higher privileges and is isolated from VTL0.
- **VTL0:** The Guest OS (e.g., Windows, Linux) runs here. It cannot access VTL2 memory or resources.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe include VTL1 in this list instead of as a footnote, since from our perspective they're both just 'guest' VTLs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a reference to VTL1 here. I think we want to keep this focused on VTL0 and VTL2, which are the most relevant. (right?)

1. **Hardware Init:** The shim initializes the CPU state and memory management unit (MMU).
2. **Config Parsing:** It parses configuration from multiple sources:
* **IGVM Parameters:** Fixed parameters provided by the host that were generated at IGVM build time.
* **IGVM Parameters:** Fixed parameters encoded into the measured section of the IGVM image, loaded by the host.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IGVM Parameters have a specific meaning in the IGVM spec, IE they mean host-provided dynamic values. This should instead say Measured Parameters and perhaps the DT & command line should be under a new Dynamic Parameters section?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants