Transport Layer Security

Ifeanyi Ukadike · October 4, 2023

crypto   tls   wireshark

Transport Layer Security (TLS) is a protocol that enables secure communication between two devices over a network by encrypting the data exchanged between these devices. When a device wants to establish a secure connection with another device, it starts a TLS handshake. Once the TLS handshake is completed and the secure connection is established, the devices can safely exchange data without worrying that someone else might be able to read or modify it.

The TLS session typically comprises of the following:

• The client initiates the TLS handshake by sending a ClientHello message.
• The server responds with a ServerHello message, selecting the preferred cipher suite and sharing its certificate (if required).
• The client validates the server’s certificate and generates a pre-master secret, which is encrypted using the server’s public key and sent to the server.
• The server decrypts the pre-master secret, and both the client and server derive the master secret using the pre-master secret and other random values.
• The client and server exchange messages to verify their ability to encrypt and decrypt using the agreed-upon cryptographic parameters.
• If successful, both parties send a Finished message containing a hash and MAC of the entire handshake, confirming that the handshake has been completed securely.

SeedLabs: TLS Lab

• TLS Lab

---

Lab Environment

client: 10.9.0.5
server: 10.9.0.43
proxy:  10.9.0.143

TLS Client

In this section of the lab, I incrementally build a simple TLS client program using Python. The aim of this lab is to understand the essential elements and security considerations in TLS programming.

TLS handshake

Before a client and a server can communicate securely, both have to agree on several cryptographic parameters. The TLS Handshake Protocol is responsible for this.

The following Python code initiates a TLS handshake with a TLS server:

#!/usr/bin/env python3

import socket, sys, pprint, ssl

hostname = sys.argv[1]
port = 443
cadir = '/etc/ssl/certs'

# Set up the TLS context
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
context.load_verify_locations(capath=cadir)
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True

# Create the TCP connection
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((hostname, port))
input("After making TCP connection. Press any key to continue ...")

# Add the TLS
ssock = context.wrap_socket(sock, server_hostname=hostname, do_handshake_on_connect=False)
ssock.do_handshake()
print(f"=== Cipher used: { ssock.cipher() }")
print(f"=== Server hostname: { ssock.server_hostname }")
print("=== Server certificate:")
pprint.pprint(ssock.getpeercert())
pprint.pprint(context.get_ca_certs())
input("After TLS handshake. Press any key to continue ...")

# Close the TLS Connection
ssock.shutdown(socket.SHUT_RDWR)
ssock.close()

Code Explanation

---

context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)    #1
context.load_verify_locations(capath=cadir)          #2
context.verify_mode = ssl.CERT_REQUIRED              #3
context.check_hostname = True                        #4

• while creating the TLS context, I specify ssl.PROTOCOL_TLS_CLIENT. This specifies that a valid cert chain and a hostname are required for the TLS connection to be successful. #1
• line #2 specifies a location that contains the CA certificates to trust for certificate verification.
• line #3 specifies that certificates are required from the other side of the socket connection; in this case, from the server.
• line #4 specifies that the certificate hostname must match the host we are visiting.

---

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)              `#1
sock.connect((hostname, port))                                        `#2
input("After making TCP connection. Press any key to continue ...")    #3

• line #1 creates an IPv4 socket object
• line #2 creates a socket connection
• line #3 pauses the program flow, waiting for input from the user

---

ssock = context.wrap_socket(sock, server_hostname=hostname, do_handshake_on_connect=False)    #1
ssock.do_handshake()                                                                          #2
print(f"=== Cipher used: { ssock.cipher() }")                                                 #3
print(f"=== Server hostname: { ssock.server_hostname }")                                      #4
print("=== Server certificate:")                                                              #5
pprint.pprint(ssock.getpeercert())                                                            #6
pprint.pprint(context.get_ca_certs())                                                         #7
input("After TLS handshake. Press any key to continue ...")                                   #8

• line #1 uses the TLS context I created earlier to wrap the socket object. The default behavior is to automatically initiate a TLS handshake, but I have chosen not to do so automatically.
• line #2 initiates the TLS handshake
• lines #3 to line #7 print out information about the TLS session
• line #8 pauses the program flow, waiting for input from the user

---

ssock.shutdown(socket.SHUT_RDWR)    #1
ssock.close()                       #2

• line #1 shuts down both sides of the TLS connection. SHUT_RDWR indicates that further sends and receives are disallowed.
• line #2 closes TLS object

Using Wireshark to capture the network traffic during the execution of the program, the following are observed:

• When the program is run, it starts with the 3-way TCP handshake. The code snippet that is responsible for this is sock.connect((hostname, port)).

  

• After establishing the TCP connection, it goes on to do a TLS handshake. The code snippet that is responsible for this is ssock.do_handshake().

  

• Finally, the program closes the connections by setting the FIN flag on the packets exchanged. The code snippets responsible for this are ssock.shutdown(socket.SHUT_RDWR) and ssock.close().

  

In summary, the TCP handshake is the initial step that establishes a network-level connection between the client and server. Once the TCP handshake completes successfully, we can be sure of a reliable, bidirectional communication channel between the client and server. However, the TLS handshake occurs within the established TCP connection. Its purpose is to establish a secure communication channel. It is initiated by the client, which sends a ClientHello message. Once the TLS handshake is completed successfully, every communication between the server and client is encrypted.

CA’s Certificate

Instead of making use of the certificates in the /etc/ssl/certs folder to verify the server’s certificates, this task makes use of a custom folder.

I proceeded to create a custom folder called “client-certs” and changed the following code snippet:

#cadir = '/etc/ssl/certs'
cadir = './client-certs'

On Running the client program, I observed that the execution failed because the program could not find the CA certificate for the host I wanted to connect to.

To remedy this, I need to get the CA certificate that is needed to verify the www.google.com server’s certificate and place it in the custom folder.

Note

When TLS tries to verify a server certificate, it will generate a hash value from the issuer’s identity information, use this hash value as part of the file name, and then use this name to find the issuer’s certificate in the specified folder.

We can create a symbolic link named the hash value that points to the actual CA certificate. First we generate the subject hash of the certificate.

openssl x509 -in CA.crt -noout -subject_hash

Assuming this gives a value of “4a6481c9”, we go on to create a symbolic link

ln -s CA.crt 4a6481c9.0

After placing the CA’s certificate in the custom location, the program executes successfully.

Experiment with the hostname check

This task helps students understand the importance of hostname checks on the client side.

Step 1: Get the IP address of www.google.com using the dig command

dig www.example.com

Step 2: Modify the /etc/hosts file of the client machine and add the IP address obtained from the dig command.

Step 3: Switch the following line in the client program between True and False, and then run the client program to www.google2023.com.

context.check_hostname = False

The following is observed after running the client program:

• When context.check_hostname = False, the program executes successfully. This means that a TLS handshake was complete even though the certificate was never issued to www.google2023.com. This means an MITM attack with a valid certificate will not be detected by my program.

  

• When context.check_hostname = True, the program doesn’t execute successfully. This means that the TLS handshake was interrupted. The program tells us that there was a hostname mismatch. Thus the program was able to detect that though the certificate is valid, it was not issued to www.google2023.com. This helps protect against an MITM attack with a valid certificate.

  

Sending and getting Data

After establishing a TLS connection with the server, I will send data to it and get its response. Since the server I am communicating with is a HTTPS server, I need to send HTTP requests to it.

The following code sends an HTTP request to www.google.com to request its index page, and displays the response received from the server.

#!/usr/bin/env python3

import socket, sys, pprint, ssl

hostname = sys.argv[1]
port = 443
cadir = '/etc/ssl/certs'

# Set up the TLS context
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
context.load_verify_locations(capath=cadir)
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True

# Create TCP connection
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((hostname, port))

# Add the TLS
ssock = context.wrap_socket(sock, server_hostname=hostname, do_handshake_on_connect=False)
ssock.do_handshake()

# Send HTTP Request to Server
request = b"GET / HTTP/1.0\r\nHost: " + \
    hostname.encode('utf-8') + \
    b"\r\n\r\n"
ssock.sendall(request)
    
# Read HTTP Response from Server
response = ssock.recv(2048)
while response:
    pprint.pprint(response.split(b"\r\n"))
    response = ssock.recv(2048)

# Close the TLS Connection
ssock.shutdown(socket.SHUT_RDWR)
ssock.close()

As a result, the server responds with the homepage of www.google.com as seen below

The following code fetches an image file from i.imgur.com

#!/usr/bin/env python3

import socket, sys, pprint, ssl

hostname = sys.argv[1]
port = 443
cadir = '/etc/ssl/certs'

# Set up the TLS context
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
context.load_verify_locations(capath=cadir)
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True

# Create TCP connection
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((hostname, port))

# Add the TLS
ssock = context.wrap_socket(sock, server_hostname=hostname, do_handshake_on_connect=False)
ssock.do_handshake()

# Send HTTP Request to Server
request = b"GET /C8VxEzh.jpeg HTTP/1.0\r\nHost: " + \
    hostname.encode('utf-8') + \
    b"\r\n\r\n"
ssock.sendall(request)
    
# Read HTTP Response from Server
response = ssock.recv(2048)
while response:
    pprint.pprint(response.split(b"\r\n"))
    response = ssock.recv(2048)

# Close the TLS Connection
ssock.shutdown(socket.SHUT_RDWR)
ssock.close()

As a result, the server responds with the requested image as seen below

TLS Server

prerequisites:

• create a CA public certificate (ca.crt) and private key (ca.key)
• use the CA’s private key to create a server certificate.

Create a CA

openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -keyout ca.key -out ca.crt

Generate Certificate Signing Request for the Server

openssl req -newkey rsa:2048 -sha256 -keyout server.key -out server.csr

Use CA’s credentials to Create a Certificate for the Server

mkdir -p demoCA/newcerts
touch demoCA/indext.txt
echo -n 10 > demoCA/serial
openssl ca -md sha256 -days 3650 -in server.csr -out server.crt -batch -cert ca.crt -keyfile ca.key -policy policy_anything

Implementing a Simple TLS Server

The following Python code creates a TLS server:

#!/usr/bin/env python3

import socket, pprint, ssl

html = \
"""
HTTP/1.1 200 OK\r\nContent-Type: text/html\r\n\r\n
<!DOCTYPE html>
<html>
<body>
<h1>Welcome to Ukadike2023.seedlabs!</h1>
</body>
</html>
"""

SERVER_CERT    = './server-certs/server.crt'
SERVER_PRVTKEY = './server-certs/server.key'

context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain(SERVER_CERT, SERVER_PRVTKEY)

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
sock.bind(('0.0.0.0', 4433))
sock.listen(5)

while True:
    newsock, fromaddr = sock.accept()
    try:
        ssock = context.wrap_socket(newsock, server_side=True)  # Establish TLS connection
        print("TLS connection established")
        data = ssock.recv(1024)              # Read data over TLS
        pprint.pprint("Request: {data}")
        ssock.sendall(html.encode('utf-8'))  # Send data over TLS

        ssock.shutdown(socket.SHUT_RDWR)     # Close the TLS connection
        ssock.close()

    except Exception:
        print("TLS connection fails")
        continue

Code Explanation

---

html = \
"""
HTTP/1.1 200 OK\r\nContent-Type: text/html\r\n\r\n
<!DOCTYPE html>
<html>
<body>
<h1>Welcome to Ukadike2023.seedlabs!</h1>
</body>
</html>
"""

This section of the code contains the HTLM that the server serves to clients.

---

SERVER_CERT    = './server-certs/server.crt'  #1
SERVER_PRVTKEY = './server-certs/server.key'  #2

• line #1 specifies the server’s public certificate
• line #2 specifies the server’s private keys

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)  #1
sock.bind(('0.0.0.0', 4433))                                 #2
sock.listen(5)                                               #3

• line #1 creates an IPv4 TCP socket object
• line #2 binds the socket object to the IP address “any” and TCP port 4433
• line #3 instructs the socket object to listen for incoming connections (with up to 5 connections waiting in the queue)

To test the server program, I will use the client program I previously developed. Since the CA for this task is created by me, I will have to add it to a custom folder and direct the client program to search the custom directory.

openssl x509 -in ca.crt -noout -subject_hash
ln -s ca.crt d7aed4be.0

Next, I will put ukadike2023.seedlabs in the client’s host file and map it to the server’s IP address. After making a connection to the server from the client program, the program connects successfully.

Testing the Server Program Using Web Browsers

When visiting ukadike2023.seedlabs from the Firefox browser I get a warning that there is a potential risk ahead as the CA who issued the certificate that ukadike2023.seedlabs is using is unknown to Firefox. This is rightfully so because the CA was created in the lab and the browser does not have it on its trusted certificate list.

To remedy this, I need to manually add the created CA’s certificate to Firefox. Once I do that and revisit ukadike2023.seedlabs, I do not get any warning from the browser as it now recognizes the CA and the webpage opens.

Certificate with multiple names

Many websites have different URLs that all point to the same web server. Due to the hostname matching policy enforced by most TLS client programs, the common name in a certificate must match the server’s hostname or TLS clients will refuse to communicate with the server. To allow a certificate to have multiple names, the X.509 specification defines extensions to be attached to a certificate called Subject Alternative Name (SAN).

To generate a certificate signing request with such a field, we can use a configuration file and put all the necessary information in this file.

Note:

The field must also include the one from the common name field; otherwise, the common name will not be accepted as a valid name.

Subject Alternative Name (server_openssl.cnf)

[ req ]
prompt = no
distinguished_name = req_distinguished_name
req_extensions = req_ext

[ req_distinguished_name ]
C = NG
ST = Lagos
CN = ukadike2023.seedlabs

[ req_ext ]
subjectAltName = @alt_names

[alt_names]
DNS.1 = www.bank32.com
DNS.2 = ukadike2023.seedlabs
DNS.3 = *.ukadike2023.seedlabs

Generate Certificate Signing Request for the Server

openssl req -newkey rsa:2048  -config ./server_openssl.cnf -batch -sha256 -keyout server.key -out server.csr

Use CA’s credentials to Create a Certificate for the Server

When a CA signs a certificate, by default, it does not copy the extension field from the certificate signing request into the final certificate. In order to copy the fields, I make a copy of /usr/lib/ssl/openssl.cnf into my working directory. Inside this file, I uncomment the “copy_extensions” option. Now I can create a certificate for the server.

openssl ca -md sha256 -days 3650  -config ./myopenssl.cnf -batch -in server.csr -out server.crt -cert ca.crt -keyfile ca.key -policy policy_anything

I finally copied the created server.crt and server.key to the server-certs folder.

Before going on to test the alternate names, I have to add them to the host file. When I visit them in Firefox browser, they open without any issue.

A Simple HTTPS Proxy

TLS can protect against the Man-In-The-Middle attack, but only if the underlying public-key infrastructure is secured. This task demonstrates an MITM attack against TLS servers if the PKI infrastructure is compromised (some trusted CA is compromised or the server’s private key is stolen).

The proxy program integrates the previously programmed client and server programs.

Notes

---

The purpose of this task is to use a simple proxy to understand how an MITM attack works when a PKI infrastructure is compromised. As such, it is not a quality HTTPS such as the open-source mitmproxy.

Handling multiple HTTP requests

To handle multiple simultaneous requests, it is better to spawn a thread to process such requests. As such, I combine the code from my client program and server program and tweak it to create a simple proxy.

#!/usr/bin/env python3

import sys, ssl, threading, socket, pprint

PROXY_CERT    = './proxy-certs/proxy_crt.pem'
PROXY_PRVTKEY = './proxy-certs/proxy_key.pem'
CLIENT_CA_DIR = '/etc/ssl/certs'

    
def process_request(ssock_browser, hostname):    
    # Make a TCP connection to the real server
    sock_svr  = socket.create_connection((hostname, 443))
    context_client = client_ssl_context()
    ssock_svr = context_client.wrap_socket(sock_svr, server_hostname=hostname, do_handshake_on_connect=True)
    
    # Receive request from the browser and forward it to the server
    request = ssock_browser.recv(512)
    if request:
        print("request made")
        #pprint.pprint(f"Request: {request}")
        ssock_svr.sendall(request)
    
    # Get response from server, and forward it to browser
    response = ssock_svr.recv(512)
    while response:
        print("response received")
        #pprint.pprint(f"Request: {response}")
        ssock_browser.sendall(response)
        response = ssock_svr.recv(512)
    
    ssock_browser.shutdown(socket.SHUT_RDWR)
    ssock_browser.close()
    ssock_svr.shutdown(socket.SHUT_RDWR)
    ssock_svr.close()


def client_ssl_context():
    context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    context.load_verify_locations(capath=CLIENT_CA_DIR)
    #context.verify_mode = ssl.CERT_REQUIRED
    #context.check_hostname = False
    return context


# Start the proxy and listen for connections on port 443
hostname = sys.argv[1]
context_svr = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context_svr.load_cert_chain(PROXY_CERT, PROXY_PRVTKEY)
sock_listen = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
sock_listen.bind(('0.0.0.0', 443))
sock_listen.listen(5)
    
while True:
    sock_browser, fromaddr = sock_listen.accept()
    try:
        ssock_browser = context_svr.wrap_socket(sock_browser,server_side=True)
    except Exception as error:
        print("TLS connection fails")
        print(error)
        continue
        
    print("TLS connection established")
    x = threading.Thread(target=process_request, args=(ssock_browser, hostname))
    x.start()

proxy_openssl.cnf

[ req ]
prompt = no
distinguished_name = req_distinguished_name

[ req_distinguished_name ]
C = NG
ST = Lagos
CN = MITM Proxy

Generate Certificate Signing Request for the Proxy

openssl req -newkey rsa:2048  -config ./proxy_openssl.cnf -batch -sha256 -keyout proxy_key.pem -out proxy_csr.pem

Use Stolen CA’s credentials to Create a Certificate for the Proxy

openssl ca -md sha256 -days 3650 -batch -in proxy_csr.pem -out proxy_crt.pem -cert ca_crt.pem -keyfile ca_key.pem -policy policy_anything

Finally copy the created proxy_crt.pem and proxy_key.pem to the proxy-certs folder.

Before going on to test the proxy, I have to add the host I want to intercept to the host file of the machine to simulate one of the following attacks: DNS attacks, BGP attacks, or other redirection attacks.

The host I am intercepting is eocdemo.solarwinds.com

From the screenshot below, the MITM attack is successful

Thanks for reading…

Share: Twitter, Facebook