Drop #233 (2023-04-03): Multi-threaded Edition v0.4.0

trurl; aicmd; Splendid R Games

I usually spend a portion of the weekend curating the Drop topics for the week, but I was way more distracted with WebR stuff than I usually am. I try not to use the Drops to pimp my own kit, so I'm not dropping any links, but if you watch my blathering on 🐘 or the bird site, you know what I've been up to.

So, I'm — once again — leaning into the “all-purpose” Drops with three different topics that should provide something for everyone to use/enjoy.

trurl

Photo by Florian Klauer on Unsplash

A few years ago, the “curl” project exposed their internal C API for parsing URLs. I, quickly, shoved that into a dev-mode R package. Sadly, I (once again) failed to convince the powers that be to care enough about this cool and highly useful functionality to provide LinkingTo support for it in the bonkers popular obviously named CRAN package (a familiar R tune to me).

During curl's 25th birthday celebrations, a discussion was had around adding yet-another curl command line option to let curl do this itself (at the CLI), or break it off to a separate utility.

A separate utility it is! (I, for one, am glad abt this.)

Right now, it's at https://github.com/curl/urler, but the new name is trurl, so you may need to use https://github.com/curl/trurl at some point, hence no direct link.

They provide sufficient examples, which I present here:

  $ trurl --url https://curl.se --set host=example.com
  https://example.com/

  $ trurl --set host=example.com --set scheme=ftp
  ftp://example.com/

  $ trurl --url https://curl.se/we/are.html --redirect here.html
  https://curl.se/we/here.html

  $ trurl --url https://curl.se/we/../are.html --set port=8080
  https://curl.se:8080/are.html

  $ trurl --url https://curl.se/we/are.html --get '{path}'
  /we/are.html

  $ trurl --url https://curl.se/we/are.html --get '{port}'
  443

  $ trurl https://example.com/hello.html --get '{scheme} {port} {path}'
  https 443 /hello.html

  $ trurl --url https://curl.se/hello --append path=you
  https://curl.se/hello/you

  $ trurl --url "https://curl.se?name=hello" --append query=search=string
  https://curl.se/?name=hello&search=string

  $ trurl --url-file url-list.txt --get '{host}'
  [one host name per URL in the input file]

  $ cat url-list.txt | trurl --url-file - --get '{host}'
  [one host name per URL in the input file]

This will be a wonderful utility for scrapers, archivers, cyber folk and more. Hopefully, you'll find some use for it as well.

aicmd

Photo by Rock'n Roll Monkey on Unsplash

I rag on “ChatGPT” as much — or more — than anyone. I don't really call it “AI”. I call it “AJI” — or, “Alex Jones Intelligence” — given how proud it is to confidently lie to you and provide gobs of misinformation.

However, if you have a sufficient corpus of exquisite “set up” prompts (perhaps “configuration” prompts is a better phrase?), and can also feed it a model honed with custom/domain-specific content, it (and other LLMs) can do wonders.

Said wonders manifest themselves in this highly focused aicmd (GH) javascript utility that aims to (for FREE!¹) let you describe what you want in human terms, and let it generate a “working” command.

After doing their free “API key” dance², I asked it for a pretty common idiom:

$ aicmd grab json from an input url pull out a a list of items and feed them to myscript using xargs

And, be darned, it did gone do good:

curl -s <input_url> | jq -r '.items[]' | xargs myscript

It even offered to run that (I deliberately made it generic).

It even did pretty well with a more complex prompt:

create a new vite vanilla js project from a given name, ensure .gitignore has 'node_modules' in it, use hub to create a github repo with the given name, add all the files from the directory and perform an initial commit and push

npx create-vite@latest my-project && 
  cd my-project && 
  echo "node_modules" >> .gitignore && 
  hub create my-project && git add . && 
  git commit -m "Initial commit" && 
  git push -u origin main

And, one more for good measure:

create a bash function for rsyncing a specified local directory to a remote filesystem as efficiently as possible, allowing me to specify the local directory and remote filesytem as parameters, but for the filesystem part, i only want to specify a partial path, that is, assume the prefix is 'user@remote:/path/to/'

function rsync_partial_path() { 
  rsync -avz --delete "$1" user@remote:"/path/to/$2"; 
}

I'm going to use the heck out of it this week and report back.

This is the “AI” future I can 100% get behind. Something that helps with the mundane tasks. And, something that will help lift up others and help democratize tech/coding a bit. Yes, you do need to have enough domain knowledge to double-check if what aicmd spits back at you is correct. However, it (and, tools like it) will most certainly help many folks save time and learn new things.

Splendid R Games

Photo by Nadine Shaabana on Unsplash

As noted in some long ago Drop, I deliberately avoid “R” topics, b/c it's just too easy for me to wax poetic about my second fav³ programming language.

Y'all may think R is “just” a “data science” language, but I'm used to humans making terrible mistakes and misjudgments.

R is now a perfectly reasonable GAMING engine, as demonstrated by this lovely curated repo by Matt Dray.

Now, go tell your IT overlords that you need R for “critical data science work”, and have some fun!

FIN

Next Monday will mark ONE YEAR of Drops! What should I do for The Daily Drop's first anniversary? ☮

1. Given how many times I’ve been asked to provide my “Open API key” in the past few months, this is a fairly refreshing experience.

2. Which I 100% support, since they can then quickly stop abuse of this free service; b/c you just know the grifters are going to try to grift as much as possible.

3. Long time Drop readers know Smalltalk is my one true 💙.