Skip to content

Add support for multiple image layers in rootfs using custom annotation #12348

@mmishra100

Description

@mmishra100

Description

Problem Statement
Currently, runsc only supports specifying a single root filesystem path in config.json When users need layered container images, they must merge these layers (pre-create an overlayfs mount on the host combining all layers) and pass that as a single source. This requires host-level setup and prevents gVisor from natively handling multi-layer container images.

Proposed Solution
Add support for a new annotation dev.gvisor.spec.rootfs.layers that accepts colon-separated paths to image layers, allowing gVisor to natively handle multi-layer container images.

"annotations": {
    "dev.gvisor.spec.rootfs.layers": "/base-layer:/app-layer:/config-layer",
    "dev.gvisor.spec.rootfs.type": "bind",
    "dev.gvisor.spec.rootfs.overlay": "memory"
  }

Benefit
No host kernel mount creation needed

Is this feature related to a specific bug?

No

Do you have a specific solution in mind?

The solution leverages gVisor's existing overlayfs support to handle multiple image layers natively:

  • Layer Specification: A new annotation dev.gvisor.spec.rootfs.layers accepts colon-separated paths to image layer directories (e.g., /layer1:/layer2:/layer3)

  • Gofer Process Setup: A single gofer process handles all layers. Before chroot, the gofer bind-mounts each layer directory to numbered subdirectories (/__layer0, /__layer1, etc.). After chroot, the gofer sees all layers as separate directories within its filesystem

  • File Descriptor Creation: For each layer, runsc creates a separate socket pair for LISAFS communication. This results in N file descriptors for N layers. The gofer serves each layer directory over its dedicated socket connection

  • Sentry Overlay Creation: Sentry receives N file descriptors (one per layer) for rootfs. For each file descriptor, sentry creates a disconnected LISAFS mount representing that layer. It combines these disconnected mounts into a single overlayfs with multiple read-only lower layers (the image layers) and one writable upper layer (tmpfs). The overlayfs is mounted at the container's root (/)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions