Drop #222 (2023-03-17): Weekend Project Edition

Stuck In The Middle With You

Trying to keep it light this weekend, given that it's another communal drinking holiday, at least in the United States; with folks in towns and cities across the land coming together to misappropriate yet-another culture as an excuse to dye things green and over-consume both food and (adult) beverages.

I'm pretty sure a good percentage of readers have likely popped open browser Developer Tools before; and, I'd bet most of that cohort has used the Network tab to inspect resource requests, and — perhaps — even exported a HAR file or three. Working within a desktop browser like that is fine and may work for a good number of use cases. But, despite there being some solutions to do the same thing on mobile devices, you can't really inspect those resource requests in the same fashion. We won't even discuss the difficulties of trying to do that with "smart" TVs and other opaque IoT gadgets that have no "head" to peek into.

So, today, we're going to take a (again, brief) look at "person-in-the-middle"

¹

proxies and check out what they are, how they work, and point to a couple of them you should consider experimenting with after you recover from any viridescent celebrations.

The Deets On "MalcomPerson In The Middle (PITM)" & Proxies

Folks who work in a large-ish enterprise are likely familiar with proxy servers. In organizations, they tend to be poorly configured and ineptly managed devices that are put in place to, ostensibly, protect folks from harm, and provide a record of activity in case someone needs to investigate an incident of some type. They are sometimes put in place by misguided infosec departments to terminate your attempt at a confidential communication channel at their proxy, inspect the contents, then re-encrypt it and send it on to the desired destination.

At home, you probably use a proxy server without even knowing it. ISPs can silently sit in the middle of your connection to an internet resource and try to save some bandwidth by loading cached resources back to you. This practice has been somewhat obviated by the insane number of content delivery networks (CDNs) which spread "edge" devices increasingly closer to where you have an internet connection. Companies such as Apple, Netflix, Amazon Video, and other streaming services make great use of these edge devices, and said content rarely hits ISP proxies anymore (and, all most folks do at home is stare at streaming glowing rectangles).

It is very unlikely you run a proxy server yourself. They're easy enough to set up, but a pain to manage. They can also slow things down (vs speed them up) if the hardware is poorly configured.

There is another "class" of proxy, though. This is one where you — the "person" — make a deliberate decision to stand up a temporary proxy server, with a goal of capturing communication sessions from a client and server for only a brief period of time. You may want to do this to see hidden APIs of your IoT systems, or discover seekrit keys that are sent to authenticated endpoints. You may also be doing it to try to find weaknesses in a client or server. But, if that is the case, you really don't need to read on since you probably already grok the rest of this article.

I use these PITMs to see how horrible iOS apps are configured, extract hidden API calls from both desktop browsers and IoT devices, and extract data from interactive web surfing sessions when I don't feel like tinkering with headless browsers to sites that are superb at detecting non-human interactions.

How To Configure A Proxy Server

There is absolutely no way I can cover the diversity of clients across all the various operating systems you all use. Suffice it to say, that in almost every compute and IoT device that can communicate over a network, there is an area of settings with configuration options like this on macOS Ventura

You specify a proxy "Server" (which is usually an IP address on your network), the port the proxy is listening on, and usually configure both HTTP and HTTPS proxies (which are handled a bit differently from each other).

• Microsoft has a support page

• As does Apple

• Linux folk are 100% on your own, given how mashed up all the possible interfaces are.

• Avast has an OK guide for multiple OSes and configurations

While I don't and will not use Firefox as a general browser, I do find it one of the easiest ones to configure to use a proxy server since it has explicit in-app settings vs use what you define at the system. I'd highly suggest going this route if you do need to capture data from a client/server interaction since it can reduce the amount of captured data you need to sift through, and doesn't break anything else relying on network comms across your device.

Generally, if you Kagi, Neeva, or Google "configure proxy setting on XYZ", you'll get decent (and safe) information, which you'll need for the next section. I'll be lowering my standards and using Firefox for the remainder of this edition. The setting’s dialog is under "Network" and looks like this:

The One-Time Being A Middle Child Isn't Horrible

I'm going to make a pretend scenario where we will intercept traffic from the super seekrit https://httpbin.org/.

You have Firefox, so now you need a PITM proxy server. We will use mitmproxy (GH), but the Community edition of Burp Suite has strong PITM proxy support, and many folks 💙 Hetty (GH).

Mitmproxy has great docs because it has been around a very long time and has a rich and vibrant community. We're going to assume you've set it up before you continue, but if you need some more assistance, this cheat sheet might be useful to have around and there's some full walk through examples for Android and iOS.

MAKE SURE YOU CONFIGURE CERTIFICATES. Our crazy "HTTPS Everywhere" world means you really cannot skip this step anymore. It's not that hard and worth the effort. Just be cautious with the client certificate, and consider removing it after use.

I recommend using mitmweb for first time users. Run that and you'll see the GUI. Go to Firefox, change the setting to what your mitmweb screen says. If it says something like "HTTP(S) proxy listening at *:8080" then use 127.0.0.1 and 8080 as the IP and port for BOTH HTTP and HTTPS proxies. If you did not do the certificate dance already, point Firefox to mitm.it, pick your OS and follow the instructions.

Now, go to https://httpbin.org/ and you should see more items appear in your mitmweb window. That means mitmproxy is intercepting all your traffic! You're also going to see some other "telemetry" connections you did not initiate. Mozilla, just like Apple, Microsoft, Google, and Brave, all snoop on you because you are the product.

To make it interesting, head to https://httpbin.org/forms/post and fill out the form. httpbin tells you what you sent, but I'm going to ask you to do two things. First, look at it in mitmweb! Click on the post request to see what it sent:

You can continue to explore in the app, but you can export entries in different ways that you should totally explore (remember, this is your weekend project).

Try exporting in all the different ways, and make sure to hit the "File" menu to save the "flows" out and poke at that as well.

The mitmproxy repo has plenty of utilities to work with the dumps from the tool, and one I find useful is this one which lets you turn dumps into a HAR file, which is a heavily nested JSON structure. Numerous tools work with HAR files well, but you can experiment with what works for you.

Your Mission

Capture at least one session from your laptop/desktop, and try to capture one from your iOS or Android device. If you're feeling lucky, try capturing from any of your IoT or streaming devices that supports proxy servers. You'll need to use the local LAN IP address of your system vs 127.0.0.1.

Now, some sites are configured to only work with certain SSL/TLS certificates. Those communications will break, but most iOS/Android apps do not care much about your safety, so most will work fine.

Extra credit if you manage to discover the hidden API for your favorite app!

FIN

PITM proxies can be very helpful for debugging, seeing if an app is misbehaving (or how much it phones home), and — as noted — figuring out hidden APIs or stealing API keys.

I was a bit light on the details, today, for two reasons. First: the daft spike proteins are at it again (impacting sleep) and mitmproxy has been around FOREVER. There are so many good resources out there that it was difficult to justify adding too much more to the pyre. I’ve been wanting to Drop this topic on folks for a while, and this seemed like a good opportunity.

Definitely reach out on all the channels (including here!) if you run into issues, and absolutely brag if you do discover something interesting! ☮

1. This isn’t what some might call “political correctness”. “Man-in-the-Middle” suggests only guys are evil/nefarious, and that is 100% not true! 🙃