Drop #238 (2023-04-11): 🎉 A Year Of Drops (But The Celebrations Will Have To Wait)

The Daily Drop: One Year On; Choose Your Own Mad Libs; Javascript and the next decade of data programming

I had intended to hit the first anniversary of The Daily Drop pretty hard with stats, prospects, and a whole cadre of “Thank You”'s to, well, you!

However, Whoop informed me that I made it into the “3% Club” (i.e., my “recovery” barely showed up in the red ring at a whopping “3%”), but I really didn't need it to tell me that. The long remnants of the spike protein invasion just would not let me sleep.

BUT! There's no way I can let the day pass without publishing something. So…

The Daily Drop: One Year On

I'll have a more expansive, navel-gazing post on just this topic when I get back into the Whoop green. But, I wanted to share some plans I have for the second year of this endeavor that has cranked out just under 250 newsletters since April 11, 2022!

• More Knowledge Drops: Around six weeks ago, I seem to have broken free of the last vestiges of burnout from my old gig. Despite dealing with lingering covid effects, I've had more energy/drive, and have been absorbing information at pre-burnout levels. I truly enjoy making technology bits accessible to the widest possible audience, because we need more folks, and a few orders of magnitude more diversity across all tech spheres. I don't know what shape this will take or what topics we'll explore together, but there will be more.

• ♫“Notes”♫: While I am still fighting (in, now, different ways) to help protect the last scraps of liberal democracy that are left, Twitter is now just the place to cross-post links/info. Mastodon is great, but it's not the place for things that make the cutting room floor of The Drop. So, as I gather up bits to share, but cannot figure out how to craft at least some narrative for them, I'll be using Substack's Notes to disseminate them, and engage with folks as much or as little as y'all want to 🙃.

• Continued focus on the fundamentals for the Weekend Project Editions. This aligns well with the expected increase in Knowledge Drop volume, since I can go deep in those and keep the WPEs focused and achievable for all.

• Data-driven index of resources. I’m slowly going back and “tagging” all the old drops. Once that’s done, I’ll have an off-Substack site where folks can go to do topic searches or hit curated areas of posts. Plus, it’ll bake it easier to search for things I’ve covered (for me as well as you).

Keep the fav's and feedback coming, too! It 100% helps shape future editions.

Choose Your Own Mad Libs

The full title for this article is “Choose Your Own Mad Libs (or, how you can plug data into automated stories and free up lots of reporting time)”. It resonated with me as I've been informed that I need to do more storytelling at work (yay “goals & objectives” time!). It's aimed at “journalists”, but I think many readers of The Drop have content cranking goals of their own, and data that can power many stories, especially ones that may have a regular cadence.

Mike Stucka discusses the use of natural language generation (b/c it seems we cannot escape this topic) to generate news stories using templates. You're familiar with these if you ever read the horrible daily financial news. But, such posts do not have to be as horrible as that!

The approach involves using templates that are similar to Mad Libs, with some Choose Your Own Adventure thrown on top. This generative approach is centered around templates that can create hundreds of stories from data.

Now, these stories are not a replacement for good journalism (or storytelling). However, they are a way to get good stories out while giving you time to do more involved work/storytelling that can't be automated. Mike suggests prioritizing stories around quantitative things such as housing prices, restaurant inspections, weather watches and warnings, employment and wages, gas prices, and more. Remember I said this was aimed at journalists. I think many of us have similar types of data. I, for one, should be able to riff from Mike's suggestions and have regular stories about our internet attack traffic. I’m pretty sure that will be useful to folks who may have a hard time interpreting the massive amounts of data we toss their way.

One more item I appreciated about the post is that it discusses the tradeoffs between timeliness, frequency, accuracy, and [geographic] detail when collecting data. Mike notes that quality considerations are important when generating stories, including adding [historical] context and using solid verbs.

Make sure to heed the note on the importance of having strong guardrails on data and templating if you do choose to go down this road!

Javascript and the next decade of data programming

Photo by Ramón Salinero on Unsplash

In this post, Ben Schmidt — a digital historian and Director of Digital Humanities at NYU — articulated something I'll have a similar post on in the next week or so. He discusses the completely light topic of the future of data programming and how new data formats are making the differences between Python, R, and javascript less important.

If you haven't been paying attention, javascript is increasingly becoming the back-end for data work in Python and R.

Despite some advances, we're still stuck in the “CPU” mode of thinking and pipelining when it comes to “data” things, while the graphical and interface primitives of all programs have started to move to the web.

Like it or not, the browser is a high-performance computing environment embedded in an excellent interactive graphics environment (well, if you're on macOS/iPadOS 🙃). The last two pieces of the puzzle are lurking on the horizon: Web Assembly (WASM) (💙) and the javascript virtual machine (the real, modern JVM).

The vast superiority of javascript over R/Python for both visualization speed — thanks to GPU integration — and interface creation — thanks to the ubiquity of HTML5 — means that people will increasingly bring their own data to websites for initial exploration first, and may never get any farther. (If you’ve been holding off working with Observable notebooks, don’t just take my word for it! There are some great demos in the post.)

Ben did a phenomenal job on the post, so I'll save you my blathering and ask that you give him some 👀.

FIN

Thank you, once more, for reading and supporting this whimsical endeavor of mine! And, I am so looking forward to another year of The Drop! (Perhaps I'll get that subdomain change in before the end of year two?) ☮