Drop #130 (2022-11-01): Tuesday Productivity Killer Edition

Simple data analysis (SDA) in JavaScript; Decker & Lil

Just two sections today as either resource could very well end up reducing your work output in some way; and, when combined, I suspect you might even forget to nom some tacos at lunchtime.

Simple data analysis (SDA) in JavaScript

Data journalists are often unsung heroes in news reporting. They're also usually quite clever humans, who build tools to Get Stuff Done™ that end up helping all of us crunch more effectively.

Since the first alert() was displayed in a browser, JavaScript has always felt like an odd language to perform "real" data work with. The event-driven, functional nature of it can make procedural analyses feel clumsy, and the need for browser-based "safe" execution environments means the sandbox can get in the way of accessing the data one needs.

Having said that, platforms like Observable — especially when combined with Quarto {ojs} blocks, improvements in letting one skirt the sandbox a bit to gain file access, and the NodeJS/Deno/etc ecosystems have truly made JavaScript a first class citizen for data analyses and data visualization (which it was already pretty good at).

Naël Shiab (@NaelShiab) is a Senior Data Producer for CBC/Radio-Canada, a computational journalist, and creator of Simple data analysis (SDA) in JavaScript. SDA is "a collection of easy-to-use methods for the most common data analysis tasks. The library works both in the browser and with NodeJS. Its expressive syntax makes it easy to read and understand. The goal is that data journalists who are not programming experts and web developers who are not data specialists can learn it quickly."

Naël built SDA out of a passion to personally Get Stuff Done™ and enable others to do so. The "Stuff", in Naël's case, is thoughtful data analysis and visual communication/storytelling.

There's a great explanation of the above over at Observable where you can walk through various data analysis examples (with a nod to R's {tidyr} package) then use brief, finely tuned JavaScript function calls to make compelling charts.

You're not stuck in Observable (tbh it's not a bad place to be stuck in). The NPM link above and the accompanying GH repo has everything you need to do SDA work at the CLI or in an IDE. As Naël notes in the Observable document, you should be cautious not to put sensitive data into Observable notebooks unless you really know what you're doing.

If you are still learning to code, you may be interested in a SDA sibling project SDA Flow [GH], a visual editor that lets you create an analysis-to-vis workflow, then save it off for re-use. The section header is an example of a toy project, but the video below shows off more of the features:

I'll definitely be using the SDA library in my Observable notebooks and highly recommend giving it a try.

Decker & Lil

Decker [GH] is "a multimedia platform for creating and sharing interactive documents, with sound, images, hypertext, and scripted behavior." It is heavily influenced by HyperCard, one of the best programs that came with early macOS systems.

Anyone can use Decker to create E-Zines, organize their notes, give presentations, build adventure games, or even just doodle some 1-bit pixel art. The holistic "ditherpunk" aesthetic is cozy, a bit nostalgic, and provides fun and distinctive creative constraints. As a prototyping tool, Decker encourages embracing a sketchy, imperfect approach. Finished decks can be saved as standalone .html documents which self-execute in a web browser and can be shared anywhere you can host or embed a web page. Decker also runs natively on MacOS, Windows, and Linux.

For more complex projects, Decker features a novel scripting language named Lil which is strongly influenced by both Lua, an imperative language popular for embedding in tools and game engines, and Q, a functional language in the APL family used with time-series databases. Lil is easy to learn and conventional enough not to ruffle any feathers for users with prior programming experience, but also includes pleasant surprises like implicit scalar-vector arithmetic and an integrated SQL-like query language. A few lines of Lil can go a long way.

The Lil language is quaint but pretty powerful, and can be used outside the Decker ecosystem via the Lilt CLI tool.

Decker "decks" are, themselves, human-readable, and you can poke around at some examples to get the feel for it.

Finally, you can see Decker in action in the tour, but you should really just take some time to build some decks. It's great fun, and they'll work on the web, or via a locally installable Decker native app. Remember: constraints fuel creativity!

If you build a deck or two, drop a note in the comments!

FIN

One week til the midterm elections in the U.S. Make sure you vote if you're eligible! ☮