Back to [linux] virtualization & container fundamentals; Getting serious about virtualization & CI on [Apple Silicon] macOS; Virtualization isn’t just for CPU’s

I am tabling my “massive Bonus post” for the time being since I took on a personal challenge to weave four things together into a cool resource + commentary post, and I’m obsessing a bit too much on the connective tissue between said groups.

Instead, since I owe y’all some Bonus Drops, let me provide a few sections with resources (with posits) around VMs, containers, et al., that have captured my interest but haven’t made their way into the M-Thu dailies.

Resources covered:

• libvirt + lxc

• tart & cilicon;

• alpha & ray

libvirt + lxc

I’ve been growing increasingly weary of both cloud providers and commercial software, of late. Since I’m more responsible for AWS costs than I have been in previous roles, I am just floored at the egregious costs and forced-complexity that — and other cloud environments — cause.

Similarly, I don’t appreciate Docker’s 💰 move and — while I do like using OrbStack (I still need to write that up) — I am loath to depend on one more Silicon Valley-produced “thing” to get stuff done.

I know this statement is relative to any individual’s personal conditions, but: a 128 GB RAM, multicore, >1 TB SSD x86_64 system can be had for $300.00−600.00 USD (depending on the week) from trustworthy sellers on various action sites (FWIW I still use ebay). i.e., That’s pretty cheap for a bonkers amount of power.

My previous gigs were, um, kinda cheapskates (that’s as much as I’ll say), so I’ve had an incarnation of such a personal home “data science” server for the past decade or so. I really needed the contiguous RAM space and storage to get real DS work-work done before tossing things into production. Now, this system fuels personal experimentation and back-end services (over Tailscale) for various web experiments. These systems have also run Linux (ofc).

Containers and virtual machine support have been baked into linux for quite some time, now, so we really don’t need “commercial” suppliers for our tooling.

Libvirt is an open-source API, daemon, and management tool that allows you to interact with the virtualization capabilities of a modern Linux operating system. It is software that provides a way to manage virtual machines, storage, and network interfaces. Ultimately, it just serves as an abstraction layer and supports various types of virtualization, including but not limited to, QEMU, KVM, Xen, and LXC. This means that you can manage these different virtualization technologies using the same libvirt commands, which makes it a versatile tool in a heterogeneous infrastructure.

My experiments/workloads sometimes go beyond what containers can provide, so having a unified way to do both full virtualization and also use containers (via LXC). If you you haven’t played with either the foundation utilities of libvrt (i.e., the vir… utilities) and the various lxc commands, I highly suggest doing so.

One caveat for folks on modern macOS systems: libvirt support used to be pretty “easy”, but the new generation of Apple Silicon-based systems have made navigating this ecosystem a bit more painful. I recommend waiting for the dust to settle a bit more before gnashing any teeth.

tart & cilicon

As the saying goes, “keep your code close & your continuous integration (CI) closer”.

Having just concluded the previous section with a bit of a whine about modern macOS systems, let’s counteract that a bit with two resources that will level-up your use of the virtualization capabilities of any fancy macOS Apple Silicon system you may have.

The new job I jumped into last year provided me with a (for that time) maxxxxxed out Mac Studio. It is glorious. I do my best to utilize all the power in the system. Two components (amongst a plethora of others) that make that possible are:

• cirruslabs/tart: macOS and Linux VMs on Apple Silicon to use in CI and other automations; and,

• traderepublic/Cilicon: 🛠️ Self-Hosted ephemeral macOS CI on Apple Silicon

Cilicon is (as the above bullet might have explicitly said) a self-hosted macOS CI solution specifically designed for Apple Silicon devices. I mention it, now, since the latest version, Cilicon 2.0, introduces several improvements. This includes an integrated SSH client that directly executes commands on the virtual machine (VM) instead of relying on a user-defined Login Item script and partial adoption of the Tart image format, allowing for automatic conversion of v1.0 images. It also comes with an integrated OCI client that can download pre-built CI images created with or for Tart.

Tart is a virtualization toolset designed to build, run, and manage macOS and Linux VMs on Apple Silicon devices. It was born out of the creative desperation of hardened CI engineers, and offers several key features, including:

• near native performance due to the use of Apple’s native virtualization framework

• integration with OCI-compatible container registries

• A Tart (HashiCorp) Packer Plugin for automating VM creation

• Built-in CI integration, with adoption by several automation services

Both resources are documented very well, and if you want to do your best to fully utilize your Apple Silicon systems, I’d recommend poking at both. They’re helping me do some testing of the quality of our tags, and I’ve migrated some other CI/CD tasks over to them from the evil that is GitHub.

alpha & ray

We’re kind of stuck with LLMs — now. Sure, they’re cool, and all, but training models for them can be/is computationally intensive and time-consuming. It also tends to require advanced algorithms and powerful hardware like GPUs. Two open-source frameworks, Alpha.ai and Ray.io, can help with GPU virtualization and speeding up the training of LLMs (provided you have access to such hardware).

Alpha.ai is a framework that focuses on pipeline parallelism, allowing for the efficient parallelization of large models across multiple GPUs. Ray.io is a framework that natively supports GPU resources and allows tasks and actors to specify their GPU resource requirements. When combined, these two frameworks can help you scale the aforementioned model development.

Four clever folks from Nvidia recently did a fine job explaining this, so I’ll let them provide the details. We’re embarking on the journey at work right now, hence my interest and the share. As we experiment more, I’ll share the results in future Drops.

FIN

Thanks for both your patience and support as I figure out a better Bonus Drop topic chooser and cadence for this busy and sometimes debilitating summer. ☮