Moar CLI Data Wrangling; The Laws of Simplicity; The gif Gallery
jApologies for two missed days. This is my penultimate week at my current employer and Wednesday was the day the establishment announced it, so it was more than a tad stressful. Yesterday was rather selfish as it was the perfect weather here in southern Maine for a massively long road cycle, which ate up the time usually devoted to these missives. Rest assured there will be five newsletters this week come hook or crook!
Moar CLI Data Wrangling
A large percentage of my computing life has been and still is at the command line as I’m nigh obsessed with the power contained in that thin (or blocky) blinking cursor. One challenge that still remains is how to turn the output of various commands into something that resembles data. Sure, many new CLI tools have delimited or JSON output as an option, but most of the old, reliable ones still do not, and many of us have spent years accumulating
*sh functions/scripts to handle the output. While a previous newsletter edition showed one tool that is frighteningly good at turning CLI output into "data", one can never have too many tools. With that in mind, today's CLI installment features jc ("JSON Convert"), a "CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts."
It, too, has an astonishing number of parsers:
airport -Icommand parser
airport -scommand parser
--asciitable: ASCII and Unicode table parser
--asciitable-m: multi-line ASCII and Unicode table parser
crontabcommand and file parser
crontabfile parser with user support
--csv: CSV file parser
--csv-s: CSV file streaming parser
dpkg -lcommand parser
git logcommand parser
--hashsum: hashsum command parser (
--ini: INI file parser
iostatcommand streaming parser
iw dev [device] scancommand parser
--jar-manifest: MANIFEST.MF file parser
--kv: Key/Value file parser
lscommand streaming parser
mpstatcommand streaming parser
ntpq -pcommand parser
pidstatcommand streaming parser
ping6command streaming parser
pip listcommand parser
pip showcommand parser
rpm -qicommand parser
rsynccommand streaming parser
statcommand streaming parser
systemctl list-jobscommand parser
systemctl list-socketscommand parser
systemctl list-unit-filescommand parser
timedatectl statuscommand parser
ufw statuscommand parser
ufw app info [application]command parser
uname -acommand parser
update-alternatives --get-selectionscommand parser
update-alternatives --querycommand parser
vmstatcommand streaming parser
--xml: XML file parser
--yaml: YAML file parser
You can install it with a simple
pip3 install jc (you can
brew install jc on macOS with Homebrew), but if you're keen to "try it before you buy it", there's a web version — which may take a few seconds to warm up, being a Heroku app — where you can paste command output and get your conversion.
One CLI tool I like to use
jc with is
dig DNS lookup tool. Try running this command to see just how many DNS
TXT records Optimizely has:
$ dig optimizely.com TXT | jc --dig | jq -r '..answer.data' # everybody has 'jq' installed, right?
Alternatively, you can just see it in this gist.
Remember that this is a Python module as well, and that means R folks can use it to process CLI output in a more straightforward way, such as:
library(reticulate) jc <- import("jc") system("dig optimizely.com TXT", intern = TRUE) |> paste0(collapse = "\n") |> jc$parse("dig", data = _) -> res sapply(res[]$answer, `[[`, "data") |> writeLines()
which has the same output you'll find in the linked gist.
The Laws of Simplicity
The book was written at the time of the iPod, but the principles within it hold up to today, and the principles aren't just about product time. The book's subtitle is "Design, Technology, Business, Life", and the the essays withing are, indeed, applicable almost everywhere.
I'll let John describe how you can use this book:
The ten Laws outlined in the body of this book are generally independent of each other and can be used together or alone. There are three flavors of simplicity discussed here, where the successive set of three Laws (1 to 3, 4 to 6, and 7 to 9) correspond to increasingly complicated conditions of simplicity: basic, intermediate, and deep. Of the three clusters, basic simplicity (1 to 3) is immediately applicable to thinking about the design of a product or the layout of your living room. On the other hand, intermediate simplicity (4 to 6) is more subtle in meaning, and deep simplicity (7 to 9) ventures into thoughts that are still ripening on the vine. If you wish to save time (in accordance with the third Law of time), I suggest you start with basic simplicity (1 to 3) and then skip to the tenth Law of the one which sums up the entire set.
Each section is a collection of micro-essays that cluster around the main topic presented. Rarely do I have answers, but instead I have a lot of questions just like you. Every Law begins with an icon of my design that represents the basic concepts I present. The images are not a literal explanation of the con- tents, but may help you to better appreciate each of the Laws. There is also associated Web content at
lawsofsimplicity.comwhere you can download the artwork as desktop patterns in case that will help to motivate you.
These are the ten "laws":
reduce: The simplest way to achieve simplicity is through thoughtful reduction. 2 organize Organization makes a system of many appear fewer.
time: Savings in time feel like simplicity.
learn: Knowledge makes everything simpler.
differences: Simplicity and complexity need each other.
context: What lies in the periphery of simplicity is definitely not peripheral.
emotion: More emotions are better than less.
trust: In simplicity we trust.
failure: Some things can never be made simple.
the one: Simplicity is about subtracting the obvious, and adding the meaningful.
I am especially fond of the essays, guidance, and examples in "Law 2" (organize) and check back in with it every so often to remind myself both how to engage in said process and the absolute need to do so if I'm going to remain sane.
It's a quick read, the website is a great companion, and the resources will, hopefully, be as useful to y'all as they have been for me.
The gif Gallery
This is just to give you a Friday distraction that doubles as a reminder of that the bad ol' days of "Geocities" pages were. You start off in one "room" and scan through the page of gifs to find the next one.
Sound up!; Don't be afraid to click on things; and, take a digital walk down animated memory lane.
Click the gif to get started!
Look for another newsletter drop later today! ☮