ss; SniffNet; sshuttle

We dive deep into the network layer, today, with three vastly different networking utilities/apps that are each filled with their own flavor of crunchy packet goodness.

NOTE: I’d encourage folks to not skip the TL;DR section, since it provides a cautionary tale regarding LLM/GPT models you might be deploying or using for more than just personal reference.

TL;DR

This is an AI-generated summary of today’s Drop.

Today, I used my fav AI assistant (Perplexity) to provide the summary using the following prompt: “There are three main sections in the attached file. Please provide a concise three bullet summary (one for each section) and make sure to include a prominent link to the primary resource focused on in each section.”

It took me a bit literally re: “Primary resource links” but it mostly did an OK job.

I say “mostly” since I used Perplexity’s upload-file capability to upload the markdown file for this drop and it asked me a co-pilot question (which I skipped). I had to sub out the link it used for the first primary resource (for ss) since it gave me this:

https://ppl-ai-file-upload.s3.amazonaws.com/web/direct-files/4298/9659912f-e744-410d-8b5b-29817483f9d3/index.md

The folks at Perplexity seem to have proper IAM policies set up (i.e., “access denied” all along those paths). But, I (and you!) now know some of the inner workings of at least where they store files (and the fact that it’s using AWS for some portions of the processing it does).

I’m not certain how I would go about preventing this particular type of information disclosure, but let this be a caution that if you think you have good “guardrails” on your production LLM/GPT models, perhaps think again.

Anyways, here are the summaries:

1. ss (socket statistics): A powerful Linux utility that provides detailed information about network sockets and connections, replacing the older netstat command. It offers advanced features and flexibility, and was developed as part of the iproute2 utility suite. Primary resource.

2. SniffNet: A Rust-based, cross-platform network monitoring tool designed as an easy-to-use alternative to Wireshark. It focuses on user experience and simplicity, offering customizable notifications, real-time charts, and overall statistics about filtered traffic. Primary resource.

3. sshuttle: A transparent proxy server that works as a “poor dude’s VPN” by forwarding traffic over an SSH connection. It doesn’t require administrative access on the remote server and operates at the application layer, redirecting traffic through the SSH tunnel without the need for a new network interface. Primary resource.

ss

Initially, a portion of readers may feel a bit of FOMO since the ss utility is Linux-only. Fear not!, however, since we’ll give macOS/BSD and (ugh) Windows users some equivalent lsof commands to help you keep up with the cool kids.

The ss (socket statistics) command is a powerful Linux utility that provides detailed information about network sockets and connections. It’s a “modern” (I mean, it’s a cira 2001 utility) replacement for the venerable (but even older) netstat command and offers more advanced features and flexibility. It was developed as part of the iproute2 utility suite. You can see the ss source right here.

A “socket” is a “file descriptor” for communicating with the network. A file descriptor is just a unique identifier assigned to a file or a socket when it is opened for communication. It serves as a bridge between the operating system and the application that uses it, allowing them to communicate with each other.

On Linux systems, “legacy” utilities like netstat interact with a special /proc/net filesystem. The /proc part of that is also known as “procfs”, and is a virtual filesystem that presents information about processes and other system information in a hierarchical file-like structure. It does not exist on a disk; instead, the Linux kernel creates it in memory.

You can work with that virtual filesystem yourself, right from the shell! While we can look at all the TCP network-y things happening at /proc/net/tcp, we can also get even more specific. Say I want to see what Nginx (my rud.is server) is doing:

cat /proc/$(pgrep nginx|tail -1)/net/tcp

I’m not showing the entire out put because the lines are potentially very, very, very long, and I have to break them up here even further, so you can see them.

The first line is the header for the output (I’ve annotated the first four fields):

sl (line number)
local_address (+ port)
rem_address (+ port)
st (connection state)
tx_queue
rx_queue
tr
tm->when
retrnsmt
uid
timeout
inode                                       

One entry from that is:

57:
B7315DAC:01BB (local_address:port)
4293BC4B:9E5C (rem_address:port)
06
00000000:00000000
03:00000D8D
00000000
0
0
0
3
0000000000000000                        

That’s not super useful to most humans. 4293BC4B (the remote IP address) is hex-encoded. Every two characters is the one octet of the IP address (reversed). We can use almost any language to turn that into something we can read. I’ll use R to show how to see all the remote IP addresses my Nginx server is talking to right now:

system(
  "cat /proc/$(pgrep nginx|tail -1)/net/tcp", intern=TRUE
) |> # run the cmd
  tail(-1) |> # skip the header
  strsplit("[[:space:]]+") |> # split the columns
  sapply(getElement, 4) |> # 4th col is the remote ip:port
  stringi::stri_match_all_regex(".{2}") |> # split string every 2 chars
  lapply(\(.x) .x[1:4]) |> # only care about the first 4
  lapply(strtoi, base = 16L) |> # turn them to ints
  lapply(rev) |> # reverse them
  sapply(paste0, collapse=".") |> # shove'm together
  unique() |>
  writeLines()

That yields:

0.0.0.0
127.0.0.1
3.125.194.96
47.128.22.89
114.119.137.128
73.16.132.205
114.119.148.102
47.128.25.1
47.128.30.128
47.128.19.99
114.119.133.69
47.128.30.153
84.202.54.4
209.177.145.120
47.128.20.194
47.128.22.114
47.128.25.52
47.128.21.208
47.128.18.0
47.128.27.29
185.48.22.230
47.128.17.109

A bash/zsh equivalent “/proc/net hex IP to human-readable IP” converter would be:

hex_to_ip() {
  # - prepend a "0x" every 2 chars + space after 
  # - let awk do the stroi & rev
  # - remove trailing extra `.`

  echo ${1} |\
    sed -r 's/(..)/0x\1 /g' |\
    awk '{ for(i=NF;i>0;i--) printf "%d.",$i; print "" }' |\
    sed 's/.$//'
}

NOTE: Linux folks may need to add -Wposix to the `awk` invocation.

Why did I do all of that when we haven’t even really talked about ss yet?

Well, because that’s the machinations “legacy” network utilities that rely on /proc/net kind of have to do as well. It’s dreadfully slow (comparatively speaking), which is a big reason ss was born. The command help and manual page for the utility are fairly large, so we won’t be showing them here. They both provide quite a bit of context for what this tool can do.

• ss -t -a dumps all TCP sockets

• ss -u -a dumps all UDP sockets

• ss -w -a dumps all RAW sockets

• ss -x -a dumps all UNIX sockets

To do what I did in R using ss is a nice one-liner:

$ ss -t | awk '{print $5}' | cut -d: -f1 | sort | uniq

The aforelinked page has extensive information on ss‘s output filtering capabilities, which include the ability to filter by state, address, port.

Before the tcp_diag kernel module came about (it communicates with user space through the netlink protocol, using the NETLINK_INET_DIAG socket family), ss used /proc/net as well. Switching it over to using tcp_diag allows it to provide more detailed and efficient access to TCP socket information compared to that parsing “files” in the /proc/net directory.

For those feeling a bit left out, you can use lsof to mimic some of ss‘s functionality:

• ss -a ↣ lsof -i: Display all open network sockets

• ss -l ↣ lsof -i -sTCP:LISTEN: Display listening sockets only

• ss -t ↣ lsof -iTCP: Display all TCP sockets

• ss -u ↣ lsof -iUDP: Display all UDP sockets

• ss -4 ↣ lsof -i4: Display IPv4 sockets only

• ss -6 ↣ lsof -i6: Display IPv6 sockets only

• ss -p ↣ lsof -i -nP: Display process information for sockets

• ss -n ↣ lsof -n -i: Display numerical addresses instead of resolving hostnames

Oh, and a free Bonus Drop subscription awaits the first reader to write an R netstat package to that interfaces with /proc/net in a way netstat does.

SniffNet

If you haven’t heard of Wireshark, you may want to dig into that utility after reading this section. Folk use it for network troubleshooting, analysis, plus software and general communications protocol development. It is feature rich/dense, which can make it somewhat daunting for folks who don’t deal with packets and protocols every day.

SniffNet (GH) is a Rust-based cross-platform (thanks to Iced) network monitoring tool designed to provide an easy-to-use alternative Wireshark (and other, similar complex tools). Its core focus on user experience and simplicity, makes it accessible to a wider audience, though that means it doesn’t have all the features other utilities might have.

The app offers customizable notifications that can inform you when specific network events occur. You can also set notifications for various thresholds, such as when data intensity exceeds a defined packets-per-second rate or bytes per second rate, or when new data is exchanged from one of the favorite connections. This feature enables us to stay informed about network traffic without constantly monitoring the application.

In addition to its ease of use, SniffNet offers several other features that make it a fairly powerful network monitoring tool:

• select a network adapter to inspect (i.e., you can focus only on Wi-Fi, LAN, or even your Tailscale net)

• apply filters based on IP version, transport protocol, and application protocol

• view real-time charts about traffic intensity (bytes and packets per second, incoming, and outgoing)

• view overall statistics about filtered traffic

• view most relevant connections (most recent, most packets, most bytes)

• view textual reports with detailed information on exchanged packets

The section header shows it running on my system for a period of time.

sshuttle

sshuttle is a transparent proxy server that works as a “poor dude’s VPN” by forwarding traffic over an SSH connection. It is an alternative to traditional VPNs and even Tailscale/WireGuard, especially when you have SSH access to a remote server but might not have full admin control over that box. The main advantage of it is that it’s pretty straightforward to use and the fact that it doesn’t require administrative access on the remote server.

Unlike a traditional VPN which ends up creating a new network interface on your system, sshuttle works by creating a transparent proxy server that forwards traffic over an SSH connection. This means that sshuttle operates at the application layer, redirecting traffic through the SSH tunnel without the need for a new network interface.

It:

• (can) automatically populates your /etc/hosts file with hostnames discovered from the server while the tunnel is active. (Requires --auto-hosts).

• automatically imports network routes from the server, removing the need to specify routes manually on the client side.

• works seamlessly even if you are unfamiliar with SOCKS configuration. It also works with programs that lack SOCKS support, without you needing to toggle SOCKS on/off as tunnels connect and disconnect.

• lets you easily run multiple concurrent tunnels to different remote servers, as long as their subnets use distinct IP ranges.

• foils remote admins who cannot disable it via sshd configuration, providing persistence.

• contains a workaround for ssh’s large transmit/receive socket buffers, which typically introduce horrible latency. This allows you to transfer large files while retaining responsive interactive traffic on the same tunnel.

• does not exhibit the random freeze bugs that can occur with native ssh port forwarding, improving reliability

Tis a mere python3 -m pip install sshuttle away and works well if you aren’t able to use something like Tailscale/WireGuard.

FIN

I had other plans for tomorrow’s WPE, but Iced seems cool enough to try to find a small project for folks to build with it. Stay tuned! ☮