The preceding of this article started from a simple task (as I earlier thought) I wanted to achieve.
I wanted to control my MI Smart bulb with code. And since there was no exact documentation available, I thought of seeing what data is being sent by the MI home app to communicate with the bulb.
And this ordeal resulted in a frenzy of articles about HTTPS, proxy servers, man-in-middle attacks, and Certificating authorities.
Capturing HTTP and HTTPS traffic on your own machine is not so uncommon. There are plenty of tools like Wireshark and Chrome Developer tools that can help with this.
When you need to capture or alter this data of other devices on your network, we need a middle man - a special kind of proxy server. This proxy server sits in between the client and the server who are in conversation. It listens to all the client requests and transfers it to the server and does the same for information coming from server to client. All the while, snooping and altering the data that is being exchanged.
This is usually reffered as a typical man-in-the-middle attack.
If it was this simple, we should be doomed. And turns out, playing a middle-man role in altering HTTP connections is fairly straight forward. Since the content going out of the client is not encrypted, this sort of packet sniffing is easy to do with proxy servers.
To the client, it looks as if this proxy server was simply relaying its connection (like your router or your ISP’s servers do). And to the server, this proxy server pretends to be the client.
In order to make the web safe from such attacks, HTTP needs to be more secure and hence HTTPS was introduced.
HTTPS
HTTPS is HTTP with S(ecutity). The only difference between the two protocols is that HTTPS uses TLS (SSL formerly) to encrypt normal HTTP requests and responses. As a result, HTTPS is far more secure than HTTP.
Servers and clients still speak exactly with the same protocol (HTTP), but over a secure TLS connection that encrypts and decrypts their requests and responses. The TLS layer has the following main purposes:
- Authentication - verifying that you are talking directly to the server that you think you are talking to
- Encryption - encrypting the exchanged data to keep it secure from eavesdroppers. Ensuring that only the server can read what you send it and only you can read what it sends back
- Data Integrity - data cannot be modified or corrupted during transfer, intentionally or otherwise, without being detected
Every request and response between client and server is now encrypted with a shared secret between the original client and server. Although anyone can still see who the client is talking to, they will not be able to make out what they are talking about i.e to read any of the actual data that is sent.
client sends:
hello world
attacker sees:
t8Fw6T8UV81pQfyhDkhebbz7+oiwldr1j2gHBB3L3RFTRsQCpaSnSBZ78Vme+DpDVJPvZdZUZHpzbbcqmSW1+3xXGsERHg9YDmpYk0VVDiRvw1H5miNieJeJ/FNUjgH0BmVRWII6+T4MnDwmCMZUI/orxP3HGwYCSIvyzS3MpmmSe4iaWKCOHQ==In order for websites to move from HTTP to HTTPS, they need a way to convince the client about their Authenticity and share further details about how to encrypt the data if they need to communicate.
These details about the servers are stored in the form of SSL certificates. These certificates are publicly visible and contain the following things:
- The domain name that the certificate was issued for
- Which person, organization, or device it was issued to
- Which certificate authority issued it
- The certificate authority's digital signature
- Associated subdomains
- Issue date of the certificate
- Expiration date of the certificate
- The public key (the private key is kept secret)
Technically, anyone can create their own SSL certificate by generating a public-private key pairing and including all the information mentioned above. To maintain the integrity of these digital certificates, they are cryptographically signed by a Certificate Authority to verify that they are legit. There are only a handful of such authorities who can sign these certificates. So again if someone tries to self-sign these certificates, the client will simply drop the connection since it was from a non-trusted party.
To check these details on chrome, click on the padlock 🔒 icon in the address bar.
How well is HTTPS adpoted?
As on April 26, 2020, 95% of the Internet traffic is served over HTTPS. Other way to look at is that still over 5% of internet data is relayed over unsecure HTTP connections.
Bypassing HTTPS Security
By now we know what we need to do to bypass the HTTPS security layer.
- Build a man-in-middle proxy server to relay information across the actual client and server
- Become a certificate authority to self-sign the certificates
This is where mitmproxy comes into play. mitmproxy is a free and open source interactive HTTPS proxy. The MITM in its name stands for Man-In-The-Middle 🤷🏽♂️.
The website lists the following features for mitmproxy
- Intercept HTTP & HTTPS requests and responses and modify them on the fly
- Replay the client-side of an HTTP conversations
- Reverse proxy mode to forward traffic to a specified server
- Transparent proxy mode on macOS and Linux
- SSL/TLS certificates for interception are generated on the fly
mitmproxy installation and setup
The recommended way to install mitmproxy on macOS is to use Homebrew. Head over to this guide for advance setup and installation.
brew install mitmproxyIn order for Mitmproxy to decrypt encrypted traffic on the fly, the mitmproxy CA certificates have to be installed on the client device. The official guide explains all the steps to do this in detail
Intercepting calls via mitmproxy
mitmproxy provides a web interface in form of mitmweb. To run simply execute
mitmweb
Web server listening at http://127.0.0.1:8081/
Proxy server listening at http://*:8080
Once the proxy server is up and running on our system, the IP for this server needs to be entered in phone's proxy settings wifi setting.
The Server address will be the system's address (usually obtained by ifconfig),
and the port will 8080 as per mitmweb
Doing so, out man-in-middle proxy server is in place snooping over all the network request going out of the phone.
And there you go, the data being sent over to the mi servers by the app in plain text format!