Architecting Browsers; Forging Protocols; Doing Nothing

I should have known trying to crank out last weekend’s missed Bonus Drop the week everyone at work went to Hacker Summer Camp was not going to end well. I’ll have to get a Knowledge Drop out this week to make up for it.

However, today we have two topics you can take a deep-dive into, and one that may change part of the way you work.

As usual, the contents from Bonus Drops will trickle into M-F editions.

Here’s what we’re covering today:

• Web Browser Engineering

• Smithy

• Do-nothing scripting

Web Browser Engineering

Web Browser Engineering is a newly finished book by Pavel Panchekha and Chris Harrelson that explains how web browsers work by building a basic but complete web browser, from networking to JavaScript, in a couple thousand lines of Python.

It’s divided into four parts:

• Introduction: Provides an overview of the book and its purpose.

• Part 1: Drawing Graphics

  • Downloading Web Pages: Covers URLs and HTTP requests.

  • Drawing to the Screen: Explains creating windows and drawing to a canvas.

  • Formatting Text: Covers word wrapping and line spacing.

• Part 2: Viewing Documents

  • Constructing a Document Tree: Covers parsing and fixing HTML.

  • Laying Out Pages: Explains inline and block layout, plus the box model.

  • Applying Author Styles: Covers parsing and applying CSS.

  • Handling Buttons and Links: Explains hyperlinks and browser chrome.

• Part 3: Running Applications

  • Sending Information to Servers: Covers form submission and web servers.

  • Running Interactive Scripts: Explains changing the DOM and reacting to events.

  • Keeping Data Private: Covers cookies and logins, XSS and CSRF.

• Part 4: Modern Browsers

  • Adding Visual Effects: Explains blending, clipping, and compositing.

  • Scheduling Tasks and Threads: Covers the event loop and the rendering pipeline.

  • Animating and Compositing: Explains smooth animations using the GPU.

  • Making Content Accessible: Covers keyboard input, zooming, and the accessibility tree.

  • Supporting Embedded Content: Explains images, iframes, and scripting.

  • Reusing Previous Computation: Covers editing, invalidation, reflow.

The average chapter takes ~4-6 hours to read, implement, and debug for someone with a few years of programming experience. I highly recommend actually going through those motions as well, though just reading the book has merit, too.

The authors are fairly irreverent toward standards, and they handle only a sliver of the full HTML, CSS, and JavaScript languages. They also paradoxically emphasize that building a browser is both easy and incredibly hard, both intentional and accidental, and everywhere you look, you see the evolution and history of the web wrapped up in one codebase.

Even just skimming through it will make you appreciate the WebKit/Safari, Chrome, and Firefox developers a bit more than you hopefully do now.

Smithy

Smithy is a protocol-agnostic Interface Definition Language (IDL) and a set of tools for generating clients, servers, documentation, and other artifacts. Developed by AWS Labs, Smithy is designed to work with any programming language, describe services running in any environment, and work with any kind of transport or serialization. It provides a powerful and flexible way to define APIs and data structures, making it easier to build and maintain services and SDKs.

The framework consists of a semantic model, file formats used to define a model, and a build process used to validate models and facilitate model transformations. The Smithy model describes services and data structures, and it can be serialized into one or more model file representations. The ultimate purpose is to help evolve, codify, and enforce API governance while being resource-based.

Introducing Smithy into your workflow can provide several benefits:

• it is designed to work with any programming language and environment, making it highly versatile

• the IDL is easy to read and write, allowing developers to quickly define services and data structures

• with it, you can generate clients, servers, and documentation for any [supported] programming language, reducing the need for manual coding and maintenance

• it adds an enforcement layer on API standards and best practices, ensuring consistency across services

• the AWS Labs folks thought ahead, and Smithy can be extended with custom traits to support specific use cases and requirements

You may be thinking, “why do I need this if I already use OpenAPI/Swagger?”. While both Smithy and OpenAPI are used for defining APIs, they have some key differences. Specifically, Smithy:

• is an interface definition language (IDL) created by Amazon for defining APIs, while OpenAPI/Swagger is a specification for describing REST APIs. Smithy is programming language agnostic, while OpenAPI uses JSON or YAML.

• defines an entire API including the data shapes, resources, operations, etc. OpenAPI just focuses on the API interface and structure.

• has built-in support for JSON RPC and XML protocols, whereas OpenAPI only supports REST/HTTP by design.

• also has a code generator that can generate client/server stubs and docs in multiple languages. OpenAPI is just for documentation and you need separate tools to generate code.

• uses model specifications that are encoded in XML or JSON. OpenAPI definitions are in JSON or icky YAML.

It is important to note that Smithy is newer and less widely adopted than OpenAPI. OpenAPI has gained massive popularity due to being simple and focused specifically on REST APIs. However, Smithy models can be converted to OpenAPI for integration with tools like SwaggerUI or access to OpenAPI client and server code generators when Smithy generators are not available.

The core difference is really that Smithy focuses on interfaces and abstractions rather than how data is sent over the wire.

I’m considering using Smithy in one of the Q3 WPEs (def let me know if that’d be useful).

Oh, and of course there’s an awesome list for it.

Do-nothing scripting

In this article, Dan Slimmon discusses the concept of “do-nothing scripting” as a way to gradually automate manual procedures.

Dan explains that many operations/engineering teams have procedures that are ripe for automation but feel daunting to tackle all at once. He uses an example of the need to provision a new user account, with the following example steps:

• Create an SSH key pair for the user.

• Commit the public key to Git and push to master.

• Wait for the build job to finish.

• Find the user’s email address in the employee directory.

• Send the user their private key via 1Password.

Do-nothing scripting involves creating a script that encapsulates each step of a procedure, but instead of automating the steps, it prompts the user to manually complete them one at a time. The value of a do-nothing script lies in its ability to help maintain focus, provide a framework for future automation, and lower the activation energy for automating tasks over time.

It’s an intriguing concept that turns what originally was a checklist into a set of prompts/completion ACKs that are less likely to be skipped over.

I’m not certain this particular example (which you should read, as Dan provides a comprehensive example of the whole thing) is the best one to use to defend the concept Dan’s putting forth (this process should 100% be automated). I also assert that instead of “doing nothing”, there might be room within the process to have the “doer” provide artifacts that prove each step was done along the way (where that makes sense). Unfortunately, that may end up turning what is a simple CLI script into a more complex [web] app.

Either way, there are some “checklist-y” procedures we have on our team at work that may end up benefiting from this concept. If we do go down this path, I’ll let y’all know the results.

FIN

If there’s a specific entity (that I actually have knowledge about 😃 you’d like covered in the aforementioned Knowledge Drop, please do not hesitate to drop said suggestion wherever you’re most comfortable. ☮