The drive back from seeing #2.1 was long, and Portland (ME) airport has one runway down, so @mrshrbrmstr's plane back from Reno to see #1 was also super delayed; thus there was no newsletter yesterday. So…you get two today! (i.e., there will be a 2022-04-22.02 in the afternoon.)

ShellCheck ✅

I have a confession to make: until a decade ago, I was a linter slacker.

What's a “linter”, you ask? Simply put, a linter is a tool that that helps you improve your code.

The concept of a linter was defined and implemented [direct PDF] by Stephen Johnson back in 1978 at Bell Labs. The linked paper describes a command-line utility lint which was written to process C code, but there are many linters out there for virtually every programming language. Here's how Johnson defined a linter:

"Lint is a command which examines C source programs, detecting a number of bugs and obscurities. It enforces the type rules of C more strictly than the C compilers. It may also be used to enforce a number of portability restrictions involved in moving programs between different machines and/or operating systems. Another option detects a number of wasteful, or error prone, constructions which nevertheless are, strictly speaking, legal."

Three things have made me both a fan and user of linters. Two of those things are the RStudio and Xcode integrated development environments (IDEs). These GUI environments (IMO) do a great job providing real-time lint notices and suggestions. Sure, my favorite editor, vi (feel free to take a shot at that in the comments, emacs-folk) has ways of providing the same functionality. Butm I wasn't in the habit of running linters before Xcode/RStudio, so I've never really dug in to linting in vi.

The third is having to create and share code/projects in teams. Linters help enforce coding standards, which is incredibly important when working on any sized project with one or more coding partners. It is fine to have a personal style when coding. However, a team really needs to agree upon a set of rules. That way, each member can expect to see source files in a certain format and using certain idioms, so everyone can Get Stuff Done™ as quickly and error-free as possible.

One area where I am still somewhat neglectful (as far as linters are concerned) is using them on shell scripts, specifically sh, bash, and zsh scripts. Since I made a commitment (“resolutions” are so pre-pandemic) to improving in many areas this year, I figured it was time to level up my shell scripts and start linting ones that go into (personal) production. The shell script linter I've found most useful is ShellCheck (online version).

ShellCheck is a linter for shell scripts (which you likely figured out from the previous paragraph). It is written in Haskell (just like the incredibly awesome pandoc!), in numerous operating system package repositories, works in almost every programming IDE (or plain editor), and is tailor-made for CI/CD pipelines. (We'll also be discussing CI/CD in the next section).

The goals of ShellCheck are (in the team's own words) to point out:

• typical beginner's syntax issues that cause a shell to give cryptic error messages.

• typical intermediate level semantic problems that cause a shell to behave strangely and counter-intuitively.

• subtle caveats, corner cases and pitfalls that may cause an advanced user's otherwise working script to fail under future circumstances.

ShellCheck has absolutely made my bash and zsh scripts cleaner and safer. Truth be told, I may have made much of the mistakes in the ShellCheck gallery of bad code,

and the tool has caught my forgetfulness about which shell I'm coding for (which is “a thing” mostly due to Apple).

To go all “full disclosure”, thanks to ShellCheck I'm also learning quite a bit about those two shell languages that I did not know before. When a tool can do its job and also make you smarter, you know it has to be good.

I heartily encourage even the most casual of shell scripters to give ShellCheck a go.

Dagger 🗡

As Michael Corleone said, “Keep your friends close, and your CI/CD pipelines closer.”

The CI/CD link in the first section was meant to give a high level overview. This CI/CD 'splainer by GitHub does a much better job filling in the details. Here's how they define CI and CD²:

• Continuous integration (CI) automatically builds, tests, and integrates code changes within a shared repository; then

• Continuous delivery (CD) automatically delivers code changes to production-ready environments for approval; or

• Continuous deployment (CD) automatically deploys code changes to customers directly.

And, here's an example workflow from the same article:

While GitHub has a vested interest in CI/CD due to their GitHub Actions service.

It has been a painful couple of years for such services.Both Travis CI and Codecov have suffered breaches (Travis has had more than one), and it is far too easy to misconfigure most of them (just like to glory days of Amazon S3 buckets). These incidents can shake the trust of organizations and individuals who use them with their private (and, for some, public) repos.

Setting safety and resilience aside, I'm not sure it's wise to rely on "the cloud" for everything, especially with the storage capacity and computing power available even on an individual's workstation. With containers and virtual environments, one can have a robust, cloud-like setup that will work even when your internet connection does not, and can help secrets (such as API keys), secret.

Dagger is a portable “devkit” for CI/CD pipelines developed by the creators of Docker. You can use any language you like for the CI/CD actions. The entire workflow is defined in the CUE language, to avoid the problems and shortcomings of using a terrible format such YAML (which seems to be the “it” format for most cloud-based CI/CD services).

While I've just espoused the virtues of local CI/CD, Dagger also plays nicely in all the cloud CI/CD environments. As a result, you can rest assured your codebase (and organization) can still be breached if you choose to stay in the 🌩.

I've only recently started tinkering with Dagger, and will blog about the home server setup in a few weeks after kicking the tyres a bit more.

CI/CD Design Principles

If you are new to CI/CD (or are a veteran who wants to see if your CI/CD mental model needs a tune up), these CI/CD design principles:

• First Principle of CI/CD: Segregation for Stakeholder Responsibility

• Second Principle of CI/CD: Risk Reduction

• Third Principle of CI/CD: Short Feedback Loop

and two CI/CD context references:

• Environment Implementations in CI/CD

• Tools for CI/CD

are succinct, accessible, and should provide both food for thought (for veterans) and foundations beginners can use to get up to speed quickly.

FIN

Remember to be on the lookout for 2022-04-22.02 this afternoon, and be kind in the comments (especially if you emacs folk will be sassing me) if you choose to engage. ☮

#shellcheck #cicd #ci #cd #dagger