2022-09-28.01
Jsonnet
This section is brought to you by the letter 'G', in that I learned about jsonnet [GH] via a GreyNoise work Slack conversation yesterday.
It is no secret that I believe YAML should die in a 🔥. I'm not the only one with said sentiment. JSON isn't a great alternative, and TOML regularly brings back INI file PTSD. Thankfully, now I do not have to suffer any of those formats directly thanks to Jsonnet, which is a JSON-like templating language which is great on its own, and has some epic tooling enabling one to convert Jsonnet files to JSON, YAML, INI, and other formats.
An example is in order. This one is complex, but shows off many features of the language:
local application = 'my-app';
local module = 'uwsgi_module';
local dir = '/var/www';
local permission = 644;
{
'uwsgi.ini': std.manifestIni({
sections: {
uwsgi: {
module: module,
pythonpath: dir,
socket: dir + '/uwsgi.sock',
'chmod-socket': permission,
callable: application,
logto: '/var/log/uwsgi/uwsgi.log',
},
},
}),
'init.sh': |||
#!/usr/bin/env bash
mkdir -p %(dir)s
touch %(dir)s/initialized
chmod %(perm)d %(dir)s/initialized
||| % {dir: dir, perm: permission},
'cassandra.conf': std.manifestYamlDoc({
cluster_name: application,
seed_provider: [
{
class_name: 'SimpleSeedProvider',
parameters: [{ seeds: '127.0.0.1' }],
},
],
}),
}
The seminal jsonnet and jrsonnet (or the crazy complete Rust alternative) — with no other command line options — will output JSON that has three different "files" in it:
cassandra.conf:
"cluster_name": "my-app"
"seed_provider":
- "class_name": "SimpleSeedProvider"
"parameters":
- "seeds": "127.0.0.1"
init.sh:
#!/usr/bin/env bash
mkdir -p /var/www
touch /var/www/initialized
chmod 644 /var/www/initialized
and, uwsgi.ini:
[uwsgi]
callable = my-app
chmod-socket = 644
logto = /var/log/uwsgi/uwsgi.log
module = uwsgi_module
pythonpath = /var/www
socket = /var/www/uwsgi.sock
It'll also make those files for you if you tell it to:
jrsonnet --multi . ex.jsonnet
./cassandra.conf
./init.sh
./uwsgi.ini
The language is Turing complete and scarily well-designed. Said design was guided by these criteria (this is directly from the Jsonett reference):
Hermeticity: Code may be treated as data. The same JSON should be generated regardless of the environment, i.e. without non-deterministic or system-dependent behaviors.
Templating language: Code should be interleaved with verbatim data so that it is easy to maintain the two in synchrony. The success of templating languages proves their effectiveness.
Variants: It should be simple and intuitive to derive variants of any existing configurations that override attributes for special or ad-hoc purposes.
Modularity: As configurations grow, it must be possible to manage the complexity using standard techniques for programming in the large. Configurations may span many files and many developers. Errors should provide stack traces that describe the nested context.
Familiarity: Raw data should be specified as JSON. Computation constructs should behave in standard ways and compose predictably.
Powerful yet simple: Trivial cases should be trivial. More complex cases should be approachable, with localized additional complexity. Everything must be possible, yet the language must still have a small footprint for learning and future tooling.
Wide scope: The same language should be able to configure anything. Ideally, all components of a system should be managed via a well-maintained centralized configuration, written in a common language.
Formal rigor: There must be an authoritative specification that is complete and simple enough to understand, and a comprehensive set of tests. This allows new implementations to be developed without compatibility hurdles. The language should not be defined by its first implementation.
The official documentation is so good, that I need not trouble you further here. Hit it up (take some time to read the fractal app example), then stop using YAML, and make Jsonnet part of your daily workflows. #FDLFUY
Hero
Hero [GH] is a "free and open source headless browser that's written in NodeJs, built on top of Chrome, and designed for easy and reliable scraping."
Here's their pitch:
Built for scraping - it's the first modern headless browsers designed specifically for scraping instead of just automated testing.
Designed for web developers - the Heroci developers have recreated a fully compliant DOM directly in NodeJS!! This enables you to bypass the headaches of previous scraper tools.
Powered by Chrome - The powerful Chrome engine sits under the hood, allowing for lightning fast rendering.
Emulates any modern browser - Emulators make it easy to disguise your script as practically any browser.
Avoids detection along the entire stack - Don't be blocked because of TLS fingerprints in your networking stack.
If you've ever had to use Puppeteer, you'll appreciate Hero's brevity:
const Hero = require('@ulixee/hero-playground');
(async () => {
const hero = new Hero();
await hero.goto('https://example.org');
const title = await hero.document.title;
const intro = await hero.document.querySelector('p').textContent;
await hero.close();
})();
Hero's "Time Travel" feature lets you replay mouse clicks, key presses, mouse movements and DOM changes in a browsing session in high fidelity, driving everything with something called “Human Emulators”, which are randomized events based on human-like patterns.
It installs without any pain, and the docs are great, so I can leave you in their capable hands to explore more of Hero.
Chaotic Stable [Diffusion]
These days, you can't swing a lost & bored NFT monkey on the socials without encountering some Stable Diffusion (SD) images. Despite the ethics problems with this and other, modern AI text-to-* creations, one has to admit that SD, and other models like it, are incredibly powerful tools. A blank/page canvas can be daunting for even the most experienced of creators, and having some baseline content to riff from is a real game-changer.
Speaking of games…
One of the example, highly focused/curated SD generators is this Dungeons & Dragons Character Generator, This generator is "an app that helps you generate character illustrations for your Dungeons & Dragons games. You can choose between gender, race, class and many more attributes for variants of character generation — Open text descriptions (where you can type anything you want) and pre-selected characteristics. You can change attributes like gender, race, class, color of eyes, length of hair and beard, using helmets, hoods or not, scars on the face, and really any characteristic you want to add to your final char."
The section header is a sample of generated images.
There's a companion generator for dragons — which means you can pair these resources with the dungeon generators mentioned in a July edition of this newsletter to level up your game/campaign nights.
FIN
A parting drop for you: The HTTP Archive's 2022 Web Almanac — State of the Web Report. ☮