Enter The Magic🪄-Wormhole🌀

Earlier this week, I noticed that the infamous @b0rk had only recently learned about the Magic-Wormhole. If someone as amazingly thoughtful, talented, observant, and on-top of technology as Julia is had never used it before, I will make the assumption other mere mortals (like myself) might also not have used it before. We use it quite a bit at work to get stuff from one system to another (since we haven’t finished our rollout of Tailscale just yet) and I can assure you it lives up to the “magic” moniker.

So, today, I’ll stick with the “keep it light” summer WPE theme and introduce magic wormhole to you and challenge you to use it at least once over the coming few days. If you’ve never used it before, you won’t believe just how easy it is to use.

Magic-WAT?

Magic-Wormhole is an open-source project for transferring files securely between computers. It was developed by Brian Warner and is available in at least Python, Golang, and Rust. The core idea behind Magic-Wormhole is to make data transfer “as simple as handing someone a piece of paper”. The tool uses the scary sounding “public key cryptography” to establish a direct and secure connection between two systems, allowing for the safe transmission of files or text.

It’s not like we have no way of sending stuff between two systems. However, existing methods like FTP or SCP are not only complex for non-technical users, but also can have inherent security risks. On the other hand, using cloud-based file sharing solutions can raise concerns about data privacy and control. Magic-Wormhole was created to bridge this gap, providing an easy-to-use tool that also prioritizes security and privacy.

You won’t believe this until you do it, but the process of sending a file with MW1 is as easy as running a command in the terminal and sharing a short, human-readable code with the recipient. This simplicity makes it accessible to users of all levels of technical proficiency. It doesn’t hurt that the entire process is fully protected from prying eyes on the network.

OK, Tell Me More!

In its core, MW uses a combination of the aforementioned public key cryptography and the PAKE (Password Authenticated Key Exchange) protocol to establish a secure connection between the sender and the recipient.

The process begins when the sender initiates a file transfer. MW generates a unique, human-readable code that the sender shares with the recipient. This code is used to establish a direct and secure connection between the two systems.

The code also acts as a password for a PAKE session, which allows the recipient to authenticate their identity and establish a secure connection with the sender. Once this connection is established, the file transfer begins. The data is encrypted before it leaves the sender’s system and is only decrypted once it arrives at the recipient’s system, ensuring that the data is secure throughout the transfer process.

Under the hood, MW uses a combination of a Rendezvous Server and a Transit Relay to facilitate this secure transfer of files.

The Rendezvous Server is a simple WebSocket-based relay that delivers messages from one client to another, allowing the wormhole codes to omit IP addresses and port numbers. The underlying rendezvous protocol is responsible for connecting the two parties who want to transfer files. When you initiate a file transfer, MW connects to a rendezvous server and creates a “wormhole”. The wormhole is, as previously noted, identified by a short, human-readable code that you share with the recipient. The recipient then uses this code to connect to the same wormhole on the rendezvous server, establishing a direct connection between the two systems. The rendezvous server only helps in establishing the connection and does not participate in the actual file transfer.

The Transit Relay is another simple server that gathers two incoming TCP connections and transfers the data between the two. The underlying transit protocol takes over for the actual data transfer. The data to be transferred is encrypted and sent directly from the sender to the recipient, without passing through any intermediate servers. This end-to-end encryption ensures that the data remains secure throughout the transfer process.

Well, Perhaps Just Show Me How To Use It? That Was Kind Of Boring.

Fine.

Please pick from the above set of languager URLs and install your fav client. We’ll assume you lazily went with the Python one (i.e., pip3 install magic-wormhole). To prove this is not just some fancy Python thing and an actual protocol with a bonafide API, I’ll show an example using both the python client and golang client.

From my macOS Sonoma box:

$ wormhole send super-seekrit-doc.md
Sending 39 Bytes file named 'super-seekrit-doc.md'
Wormhole code is: 3-upcoming-accrue
On the other computer, please run:

wormhole receive 3-upcoming-accrue

Sending (<-10.1.10.XXX:55932)..
100%|███████████████████████| 39.0/39.0 [00:00<00:00, 26.8kB/s]
File sent.. waiting for confirmation
Confirmation received. Transfer complete.

to a random internet-connect Linux safe house I have out there somewhere:

$ wormhole-william receive 3-upcoming-accrue
Receiving file (39 B) into: super-seekrit-doc.md
ok? (y/N):y
39 B / 39 B [---------------------] 100.00% 8 B p/s 5.2s

Your Mission

Use it! No, really! It’s an oddly satisfying experience. Even at the best of times, lots of our fun pieces of technology fail to “just work”.

MW “just works”.

Your Stretch Goal

Set up and use your own rendezvous server.

FIN

I heartily encourage folks to dig in deep, as this is one super cool piece of user-friendly technology that abstracts a ton of complexity and provides a super useful service. ☮