Discover more from hrbrmstr's Daily Drop
Hello, Observable!; Scraping TikTok; Warp's Commands.dev
To help even more individuals and teams harness the power of interactive datavis, Mike started Observable, a collaborative notebook platform that comes with batteries included to make it fairly straightforward to dive into ad hoc data analysis or produce complete reports.
Wrapping your head around Observable may take some doing, depending on your mental model of how notebooks and datavis. Luckily, some talented folks, such as Scott Franz are here to help you grok this new, reactive-based environment, with their Hello, Observable! notebook, which is also in a non-Observable platform context.
Scott's introduction walks you through the fundamentals of the Observable platform with many (many) examples, and also shows how to bring Observable with you to your own hosting environment.
One of the best parts of walkthroughs like Scott's is the fact that you can look at the code that was used to make it right in the notebook and riff off of it for your own creations.
If you've been Observable-curious, this resource should help scratch that itch.
While I am not on TikTok, it is similar to D3 (see above) in that it is also nigh impossible to go a day on the internet without running into one of the short, viral posts from that platform.
TikTok content can be both silly and serious, and it is a large, global social network, so there are good reasons to look at it with the eyes of a data analyst, whether it be mining trends or keeping an eye out for more nefarious uses of the platform.
While TikTok has some developer resources, you're somewhat on your own if you want to use TikTok itself as a dataset. Well, you were until Andrew Nord built the TikTok Scraper & Downloader. With it, you can:
Save post metadata to the JSON/CSV files
Download media with and without the watermark and save to the ZIP file
Download single video without the watermark from the CLI
Sign URL to make custom request to the TikTok API
Extract metadata from the User, Hashtag and Single Video pages
Save previous progress and download only new videos that weren't downloaded before. This feature only works from the CLI and only if download flag is on.
View and manage previously downloaded posts history in the CLI
Scrape and download user, hashtag, music feeds and single videos specified in the file in batch mode
From personal, cursory use, it definitely does what it says on the tin.
I found this resource via Johanna Wild's Bellingcat post, where I also learned about their Investigate TikTok Like A Pro! guide (which I likely glossed over back in 2020 b/c "TikTok"), and the TikTok hashtag analysis toolset, based on Andrew's work.
Now, all we need is for someone to make a TikTok showing how to analyze TikTok.
Back in April, I introduced the new Warp terminal "platform". I've been using it daily since then, and even have an interview (not the job kind) with the Warp team coming up in June.
The Warp team recently debuted Commands.dev, a templated and searchable catalog of popular terminal commands. The website serves up the Warp workflows commands catalog available to the Warp community.
Now, most hardcore terminal denizens make use of shell functions, scripts, and aliases to encapsulate useful idioms and command presets. As the Warp team notes, getting aliases and functions to a productive state requires an upfront investment that’s justifiable for devs who spend most of their workday in the terminal, but less so for beginners and casual users.
Anyone can contribute a workflow, and if this does truly become a broader commmunity resource, I can envision wiring it up to VS Code, Sublime Text, iTerm, Alfred, and even RStudio.
I am in disbelief that I have made eighteen "TikTok" references in this post. O_O
If you make said video, created or found some cool Observable notebooks, or have some Warp tips or crazy cool commands, def drop a link to them in the comments. ☮