How to use Socat and Wireshark for Practical SSL Protocol Reverse Engineering?
Secure Socket Layer (SSL) Man-In-the-Middle (MITM) proxies have two very specific purposes. The first is to allow a client with one set of keys to communicate with a service that has a different set of keys without either side knowing about it. This is typically seen as a MITM attack but can be used for productive ends as well. The second is to view the unencrypted data for security, educational, an reverse engineering purposes.
For instance, a system administrator could set up a proxy to allow SSL clients that don’t support more modern SSL methods or even SSL at all to get access to services securely. Typically, this involves having the proxy set up behind your firewall so that unencrypted content stays within the confines of your local area.
Being able to analyze the unencrypted data is very important to security auditors as well. A very large
percentage of developers feel their services are adequately protected since SSL is being used between
the client and the server. This includes the idea that if the SSL client is custom closed source software that the protocol will be unbreakable and therefore immune to tampering. If you’re investing your companies funds using a service that could easily be subject to tampering then you may end up with a nasty surprise. Lost funds perhaps or possibly having your account information publicly available. This article focuses on using an SSL MITM proxy to reverse engineer a simple web service. The purpose of doing so will be to create your own client that can interact with a database behind an unpublished API. The software used will be based on the popular open source software Socat as well as the widely recognized Wireshark.Both are available on most operating systems.
Let’s get started!
We will be reverse engineering a LiveJournal client called LogJam which supports SSL connections to the LiveJournal API servers. Since this article is purely educational we don’t mind getting some experience using the LiveJournal API which already public and LogJam which is a free and open source project.
Prerequisites
• Install Socat – Multipurpose relay for bidirectional data transfer: http://www.dest-unreach.org/socat/
• Install Wireshark – Network traffic analyzer: http://www.wireshark.org/
• Install OpenSSL – Secure Socket Layer (SSL) binary and related cryptographic tools: http://www.openssl.org/
• Install TinyCA – Simple graphical program for certification authority management: http://tinyca.sm-zone.net/
• Install LogJam – Client for LiveJournal-based sites: http://andy-shev.github.com/LogJam/
Generating a false SSL certificate authority (CA) and server certificate
The API domain name for LiveJournal is simply www.livejournal.com and any SSL compliant client
software will require the server certificate to match the domain when it initially connects to the SSL port of the server.
An SSL CA signs SSL certificates and is nothing more than a set of certificates files that can be used by tools like OpenSSL to sign newly generated certificates via a certificate signature request (CSR) key that is generated while creating new server certificates. The client simply needs to trust the certificate authority public key and subsequently the client will trust all server certificates signed by the certificate authority private key.
Generating a certificate authority
Run tinyca2 for the first time and a certificate authority generation screen will appear to get you started (Figure 1).
It doesn’t matter what you put here if you don’t plan on keeping this certificate authority information for very long. The target server at LiveJournal.com will never see the keys you are generating and they will stay completely isolated to your testing environment. Be sure to remember the password since it will be required for signing keys later on.
Select Export CA from the CA tab and save a PEM version of the public CA certificate to a new file of your choosing.
Generating a server certificate
Click on the Requests tab in TinyCA and then the New button that will help us create a new certificate signing request and private server key (Figure 2).
The common name must be www.livejournal.com. The password can be anything and we will be removing it when we export the key for use.
Under the Requests tab there is now a certificate named www.livejournal.com that needs to be signed.
Right click and select Sign Request and then Sign Request Server. Use the default values to sign the request.
Now there will be a new key under the Key tab now. Right click on it and select Export Key and you’ll be presented a new dialog (Figure 3).
As seen in the figure you want to select PEM (Key) as well as Without Passphase (PEM/PKCS#12) and Include Certificate (PEM). Doing so will export a PEM certificate file that contains a section for the certificate key as well as the certificate itself. The PEM standard allows us to store multiple keys in a single file.
Congratulations, you now have a perfectly valid key for https://www.livejournal.com as long as the web server running the site is under your own control and uses the server key you’ve generated. Trusting the key is the tricky part.
Allow logjam to trust the certificate authority
So we have to dig in a bit to understand what SSL Certificate trust database LogJam will be using. Most Linux based GTK and console programs rely on OpenSSL which has its own certificate authority database that is very easy to add a new certificate to.
In Debian/GNU Linux the following will install your new Yoyodyne CA certificate system wide: Listing 1.
Listing 1. Install Yoyodyne CA certificate
spencersr@bigboote:~$ sudo mkdir /usr/share/ca-certificates/custom
spencersr@bigboote:~$ sudo cp Yoyodyne-cacert.pem \ /usr/share/ca-certificates/custom/Yoyodyne-
-cacert.crt
spencersr@bigboote:~$ sudo chmod a+rw \
/usr/share/ca-certificates/custom/Yoyodyne-cacert.crt
spencersr@bigboote:~$ sudo dpkg-reconfigure -plow ca-certificates -f readline \ ca-certificates
configuration
-----------------------------
...
Trust new certificates from certificate authorities? 1
...
This package installs common CA (Certificate Authority) certificates in /usr/share/ca-certificates.
Please select the certificate authorities you trust so that their certificates are installed into
/etc/ssl/certs. They will be compiled into a single /etc/ssl/certs/ca-certificates.crt file.
...
1. cacert.org/cacert.org.crt
2. custom/Yoyodyne-cacert.crt
3. debconf.org/ca.crt
...
150. mozilla/XRamp_Global_CA_Root.crt
151. spi-inc.org/spi-ca-2003.crt
152. spi-inc.org/spi-cacert-2008.crt
...
(Enter the items you want to select, separated by spaces.)
...
Certificates to activate: 2
...
Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....
Adding debian:Yoyodyne-cacert.pem
done.
Now LogJam as well as programs such as wget, w3m, and most scripting languages will trust all keys signed by your new CA.
Using Socat to proxy the stream and hijacking your own DNS
Socat is basically a swiss army knife for communication streams. With it you can proxy between protocols. This includes becoming an SSL aware server and proxying streams as an SSL aware client to another SSL aware server
Set up your system and start up socat
Since we should aim for transparency we will need to intercept DNS requests for www.livejournal.com as well so that our locally operated proxy running on port 443 on IP 127.0.2.1 is in the loop.
First, we will need to know the original IP of www.livejournal.com:
spencersr@bigboote:~$ nslookup www.livejournal.com 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: www.livejournal.com
Address: 208.93.0.128
Bingo! Now add the following line to /etc/hosts near the other IPv4 records:
127.0.2.1 www.livejournal.com
Now let’s do a test run by listening on port 443 (HTTPS) and forwarding to port 443 (HTTPS) of the real www.livejournal.com:
spencersr@bigboote:~$ sudo socat -vvv \ OPENSSLLISTEN:
443,verify=0,fork,key=www.livejournal.comkeyem,
certificate=www.livejournal.com-key.pem,
cafile=Yoyodyne-cacert.pem \
OPENSSL:208.93.0.128:443,verify=0,fork
Simple enough. Browsing to https://www.livejournal.com with w3m and wget should work sucessfully now and a stream of random encrypted information will be printed by socat.
Chaining two socat instances together with an unencrypted session in the middle
So far so good! Now we need to have socat connecting to another socat using standard TCP4 protocol in order to view the unencrypted data. This works by having one socat instance listening on port 443 (HTTPS) and then forwarding to another socat on port 8080 (HTTP) which then forwards on to port 443 (HTTPS) of the real www.livejournal.com.
Socat instance one:
spencersr@bigboote:~$ sudo socat -vvv \
OPENSSL-LISTEN:443,verify=0,fork,
key=www.livejournal.com-key.pem,certificate=
www.livejournal.com-key.pem,cafile=Yoyodyne-cacert.pem \
TCP4:10.1.0.1:8080,fork
Socat instance two:
spencersr@bigboote:~$ sudo socat -vvv \
TCP-LISTEN:8080,fork \
OPENSSL:208.93.0.128:443,verify=0,fork
Load up LogJam and the socat instances will start printing out the stream to the terminal (Listing 2).
Listing 2. Socat terminal
> 2012/08/29 00:10:27.527184 length=209 from=0 to=208
POST /interface/flat HTTP/1.1\r
Host: www.livejournal.com\r
Content-Type: application/x-www-form-urlencoded\r
User-Agent: http://logjam.danga.com; martine@danga.com\r
Connection: Keep-Alive\r
Content-Length: 23\r
\r
> 2012/08/29 00:10:27.566184 length=23 from=209 to=231
ver=1&mode=getchallenge< 2012/08/29 00:10:29.551570 length=437 from=0 to=436
HTTP/1.1 200 OK\r
Server: GoatProxy 1.0\r
Date: Wed, 29 Aug 2012 08:10:56 GMT\r
Content-Type: text/plain; charset=UTF-8\r
Connection: keep-alive\r
X-AWS-Id: ws25\r
Content-Length: 157\r
Accept-Ranges: bytes\r
X-Varnish: 904353035\r
Age: 0\r
X-VWS-Id: bil1-varn21\r
X-Gateway: bil1-swlb10\r
\r
auth_scheme
c0
challenge
c0:1346227200:656:60:xxxxxx:xxxxxxxxxxxxx
expire_time
1346227916
server_time
1346227856
success
OK
Using Wireshark to capture and view the unencrypted stream
Now it’s time for the easy part. I’m going to assume that you are comfortable capturing packets in Wireshark and focus mainly on the filtering of the capture stream.
Since by default Wireshark captures all traffic we should set up a capture filter that only listens for packets on port 8080 of host 127.0.2.1 (Figure 4).
Once LogJam is run packet will start streaming in while Wireshark is recording (Figure 5).
What now?
This articles is about viewing unencrypted data in an SSL session. Whatever your reverse engineering goal is SSL is less of an obstacle now.
How can SSL be secure then if this method is so simple?
SSL and all of the variations of digests and ciphers contained within it are pretty reliably secure. Some of the major areas this article focused on was the ability to fool a client by having the ability to trust a new certificate.
If you are interested in securing your site or client software against this sort of spying I recommend not using an SSL certificate authority keyring or trust database that is easily modified by the user. Including an SSL server certificate in client software, encrypted and protected by a hard coded key somewhere in the binary, and requiring it for use on SSL connections using a hardened socket library, will dramatically cut down on the looky-loo factor.
Secure Socket Layer (SSL) Man-In-the-Middle (MITM) proxies have two very specific purposes. The first is to allow a client with one set of keys to communicate with a service that has a different set of keys without either side knowing about it. This is typically seen as a MITM attack but can be used for productive ends as well. The second is to view the unencrypted data for security, educational, an reverse engineering purposes.
For instance, a system administrator could set up a proxy to allow SSL clients that don’t support more modern SSL methods or even SSL at all to get access to services securely. Typically, this involves having the proxy set up behind your firewall so that unencrypted content stays within the confines of your local area.
Being able to analyze the unencrypted data is very important to security auditors as well. A very large
percentage of developers feel their services are adequately protected since SSL is being used between
the client and the server. This includes the idea that if the SSL client is custom closed source software that the protocol will be unbreakable and therefore immune to tampering. If you’re investing your companies funds using a service that could easily be subject to tampering then you may end up with a nasty surprise. Lost funds perhaps or possibly having your account information publicly available. This article focuses on using an SSL MITM proxy to reverse engineer a simple web service. The purpose of doing so will be to create your own client that can interact with a database behind an unpublished API. The software used will be based on the popular open source software Socat as well as the widely recognized Wireshark.Both are available on most operating systems.
Let’s get started!
We will be reverse engineering a LiveJournal client called LogJam which supports SSL connections to the LiveJournal API servers. Since this article is purely educational we don’t mind getting some experience using the LiveJournal API which already public and LogJam which is a free and open source project.
Prerequisites
• Install Socat – Multipurpose relay for bidirectional data transfer: http://www.dest-unreach.org/socat/
• Install Wireshark – Network traffic analyzer: http://www.wireshark.org/
• Install OpenSSL – Secure Socket Layer (SSL) binary and related cryptographic tools: http://www.openssl.org/
• Install TinyCA – Simple graphical program for certification authority management: http://tinyca.sm-zone.net/
• Install LogJam – Client for LiveJournal-based sites: http://andy-shev.github.com/LogJam/
Generating a false SSL certificate authority (CA) and server certificate
The API domain name for LiveJournal is simply www.livejournal.com and any SSL compliant client
software will require the server certificate to match the domain when it initially connects to the SSL port of the server.
An SSL CA signs SSL certificates and is nothing more than a set of certificates files that can be used by tools like OpenSSL to sign newly generated certificates via a certificate signature request (CSR) key that is generated while creating new server certificates. The client simply needs to trust the certificate authority public key and subsequently the client will trust all server certificates signed by the certificate authority private key.
Generating a certificate authority
Run tinyca2 for the first time and a certificate authority generation screen will appear to get you started (Figure 1).
It doesn’t matter what you put here if you don’t plan on keeping this certificate authority information for very long. The target server at LiveJournal.com will never see the keys you are generating and they will stay completely isolated to your testing environment. Be sure to remember the password since it will be required for signing keys later on.
Select Export CA from the CA tab and save a PEM version of the public CA certificate to a new file of your choosing.
Generating a server certificate
Click on the Requests tab in TinyCA and then the New button that will help us create a new certificate signing request and private server key (Figure 2).
The common name must be www.livejournal.com. The password can be anything and we will be removing it when we export the key for use.
Under the Requests tab there is now a certificate named www.livejournal.com that needs to be signed.
Right click and select Sign Request and then Sign Request Server. Use the default values to sign the request.
Now there will be a new key under the Key tab now. Right click on it and select Export Key and you’ll be presented a new dialog (Figure 3).
As seen in the figure you want to select PEM (Key) as well as Without Passphase (PEM/PKCS#12) and Include Certificate (PEM). Doing so will export a PEM certificate file that contains a section for the certificate key as well as the certificate itself. The PEM standard allows us to store multiple keys in a single file.
Congratulations, you now have a perfectly valid key for https://www.livejournal.com as long as the web server running the site is under your own control and uses the server key you’ve generated. Trusting the key is the tricky part.
Allow logjam to trust the certificate authority
So we have to dig in a bit to understand what SSL Certificate trust database LogJam will be using. Most Linux based GTK and console programs rely on OpenSSL which has its own certificate authority database that is very easy to add a new certificate to.
In Debian/GNU Linux the following will install your new Yoyodyne CA certificate system wide: Listing 1.
Listing 1. Install Yoyodyne CA certificate
spencersr@bigboote:~$ sudo mkdir /usr/share/ca-certificates/custom
spencersr@bigboote:~$ sudo cp Yoyodyne-cacert.pem \ /usr/share/ca-certificates/custom/Yoyodyne-
-cacert.crt
spencersr@bigboote:~$ sudo chmod a+rw \
/usr/share/ca-certificates/custom/Yoyodyne-cacert.crt
spencersr@bigboote:~$ sudo dpkg-reconfigure -plow ca-certificates -f readline \ ca-certificates
configuration
-----------------------------
...
Trust new certificates from certificate authorities? 1
...
This package installs common CA (Certificate Authority) certificates in /usr/share/ca-certificates.
Please select the certificate authorities you trust so that their certificates are installed into
/etc/ssl/certs. They will be compiled into a single /etc/ssl/certs/ca-certificates.crt file.
...
1. cacert.org/cacert.org.crt
2. custom/Yoyodyne-cacert.crt
3. debconf.org/ca.crt
...
150. mozilla/XRamp_Global_CA_Root.crt
151. spi-inc.org/spi-ca-2003.crt
152. spi-inc.org/spi-cacert-2008.crt
...
(Enter the items you want to select, separated by spaces.)
...
Certificates to activate: 2
...
Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....
Adding debian:Yoyodyne-cacert.pem
done.
Now LogJam as well as programs such as wget, w3m, and most scripting languages will trust all keys signed by your new CA.
Using Socat to proxy the stream and hijacking your own DNS
Socat is basically a swiss army knife for communication streams. With it you can proxy between protocols. This includes becoming an SSL aware server and proxying streams as an SSL aware client to another SSL aware server
Set up your system and start up socat
Since we should aim for transparency we will need to intercept DNS requests for www.livejournal.com as well so that our locally operated proxy running on port 443 on IP 127.0.2.1 is in the loop.
First, we will need to know the original IP of www.livejournal.com:
spencersr@bigboote:~$ nslookup www.livejournal.com 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: www.livejournal.com
Address: 208.93.0.128
Bingo! Now add the following line to /etc/hosts near the other IPv4 records:
127.0.2.1 www.livejournal.com
Now let’s do a test run by listening on port 443 (HTTPS) and forwarding to port 443 (HTTPS) of the real www.livejournal.com:
spencersr@bigboote:~$ sudo socat -vvv \ OPENSSLLISTEN:
443,verify=0,fork,key=www.livejournal.comkeyem,
certificate=www.livejournal.com-key.pem,
cafile=Yoyodyne-cacert.pem \
OPENSSL:208.93.0.128:443,verify=0,fork
Simple enough. Browsing to https://www.livejournal.com with w3m and wget should work sucessfully now and a stream of random encrypted information will be printed by socat.
Chaining two socat instances together with an unencrypted session in the middle
So far so good! Now we need to have socat connecting to another socat using standard TCP4 protocol in order to view the unencrypted data. This works by having one socat instance listening on port 443 (HTTPS) and then forwarding to another socat on port 8080 (HTTP) which then forwards on to port 443 (HTTPS) of the real www.livejournal.com.
Socat instance one:
spencersr@bigboote:~$ sudo socat -vvv \
OPENSSL-LISTEN:443,verify=0,fork,
key=www.livejournal.com-key.pem,certificate=
www.livejournal.com-key.pem,cafile=Yoyodyne-cacert.pem \
TCP4:10.1.0.1:8080,fork
Socat instance two:
spencersr@bigboote:~$ sudo socat -vvv \
TCP-LISTEN:8080,fork \
OPENSSL:208.93.0.128:443,verify=0,fork
Load up LogJam and the socat instances will start printing out the stream to the terminal (Listing 2).
Listing 2. Socat terminal
> 2012/08/29 00:10:27.527184 length=209 from=0 to=208
POST /interface/flat HTTP/1.1\r
Host: www.livejournal.com\r
Content-Type: application/x-www-form-urlencoded\r
User-Agent: http://logjam.danga.com; martine@danga.com\r
Connection: Keep-Alive\r
Content-Length: 23\r
\r
> 2012/08/29 00:10:27.566184 length=23 from=209 to=231
ver=1&mode=getchallenge< 2012/08/29 00:10:29.551570 length=437 from=0 to=436
HTTP/1.1 200 OK\r
Server: GoatProxy 1.0\r
Date: Wed, 29 Aug 2012 08:10:56 GMT\r
Content-Type: text/plain; charset=UTF-8\r
Connection: keep-alive\r
X-AWS-Id: ws25\r
Content-Length: 157\r
Accept-Ranges: bytes\r
X-Varnish: 904353035\r
Age: 0\r
X-VWS-Id: bil1-varn21\r
X-Gateway: bil1-swlb10\r
\r
auth_scheme
c0
challenge
c0:1346227200:656:60:xxxxxx:xxxxxxxxxxxxx
expire_time
1346227916
server_time
1346227856
success
OK
Using Wireshark to capture and view the unencrypted stream
Now it’s time for the easy part. I’m going to assume that you are comfortable capturing packets in Wireshark and focus mainly on the filtering of the capture stream.
Since by default Wireshark captures all traffic we should set up a capture filter that only listens for packets on port 8080 of host 127.0.2.1 (Figure 4).
Once LogJam is run packet will start streaming in while Wireshark is recording (Figure 5).
What now?
This articles is about viewing unencrypted data in an SSL session. Whatever your reverse engineering goal is SSL is less of an obstacle now.
How can SSL be secure then if this method is so simple?
SSL and all of the variations of digests and ciphers contained within it are pretty reliably secure. Some of the major areas this article focused on was the ability to fool a client by having the ability to trust a new certificate.
If you are interested in securing your site or client software against this sort of spying I recommend not using an SSL certificate authority keyring or trust database that is easily modified by the user. Including an SSL server certificate in client software, encrypted and protected by a hard coded key somewhere in the binary, and requiring it for use on SSL connections using a hardened socket library, will dramatically cut down on the looky-loo factor.