“SSL/TLS handshake” is the name of the stage of HTTPS connection installation. Most of the work related to SSL/TLS protocol is done at this stage. Last year IETF finalized TLS 1.3, completely updating the handshake process.
The article will cover two types of handshakes – for TLS 1.2 and TLS 1.3 protocols, which we will consider starting from the abstract level and gradually deepening in particular:
- cryptographic protocol negotiation;
- authentication with SSL certificate;
- generation of session key.
How the TLS handshake happens
An HTTPS connection involves two parties: the client (initiator of the connection, usually a web browser) and the server. The purpose of the SSL/TLS handshake is to do all the cryptographic work to establish a secure connection, including authenticating the SSL certificate used and generating the encryption key.
Each software is unique. Therefore, even the most popular web browsers have different functionality. Similarly on the server side – Windows Server, Apache and NGINX are also different. It gets even more complicated when you add custom configurations.
That is why the first step of the TLS handshake is to exchange information about your capabilities between client and server to further select supported cryptographic functions.
As soon as the client and server agree on the encryption used, the server sends its SSL certificate to the client.
Having received the certificate, the client checks it for authenticity. This is an extremely important step. For the connection to be secure, not only do you need to encrypt the data, you also need to make sure that it is sent to the correct website. SSL/TLS certificates provide this authentication and how they do it depends on the encryption device used.
All trusted SSL certificates are issued by a Certificate Authority (CA). The CA must follow strict rules for issuing and checking certificates to be trusted. You can consider the CA to be something like a notary public – its signature means that the data in the certificate is real.
During the authentication part of the TLS handshake the client performs several cryptographically secure checks to make sure that the certificate issued by the server is authentic. The process involves checking the digital signature and whether the certificate was issued by a trusted CA.
At this stage, the client indirectly verifies if the server owns the private key associated with the certificate.
In RSA, the most common public key cryptosystem, the client uses the public key to encrypt random data that will be used to generate the session key. The server will only be able to decrypt and use this data if it has a private key that ensures the authenticity of the party.
If another cryptosystem is used, the algorithm may change, but the other side will still be authenticated.
The last part of the TLS handshake includes creation of a “session key”, which will actually be used for secure communication.
Session keys are “symmetric”, i.e. the same key is used for encryption and decryption.
Symmetric encryption is more productive than asymmetric, making it more suitable for sending data over an HTTPS connection. The exact method of key generation depends on the cipher set selected, the two most common being RSA and Diffy-Hellman.
To complete the handshake, each side reports to the other that they have done all the necessary work, and then checks the checksums to ensure that the handshake happened without any interference or damage.
The entire SSL handshake takes several hundred milliseconds. This is the first thing that happens with an HTTPS connection, even before a web page is loaded. After the SSL handshake, an encrypted and authenticated HTTPS connection begins, and all data sent and received by the client and the server is protected.
Up to TLS 1.3, every time you visit a website, the handshake is repeated. The TLS 1.3 handshake supports 0-RTT or zero resume time, which greatly increases the speed for the returning visitor.
TLS 1.2 step-by-step handshake process
Let’s take a closer look at TLS handshake using RSA. Using Diffy-Hellman algorithm will be described below.
- The first message is called “Client Hello”. This message lists the client’s capabilities so that the server can choose the encryption that will be used for communication. The message also includes a large randomly selected prime number, called a “random client number”.
- The server politely replies with a “Server Hello” message. There it tells the client what connection options have been selected and returns its randomly selected prime number, called a “random server number”. If the client and server do not share the same cipher sets, the connection fails.
- In “Certificate” message, the server sends to the client its chain of SSL certificates, which includes sheet and intermediate certificates. Once the client receives them, it performs several checks to verify the certificate. The client also has to make sure that the server has a private certificate key, which happens during the key exchange/generation process.
- This is an optional message needed only for certain key exchange methods (e.g. Diffy-Hellman algorithm) that require additional data from the server.
- The “Server Hello Done” message notifies the client that the server has finished transmitting data.
- The client then participates in creating the session key. The peculiarities of this step depend on the key exchange method that was chosen in “Hello” source messages. Since we are considering RSA, the client will generate a random byte string called a pre-master secret, encrypt it with the server’s public key and pass it back.
- The Change Cipher Spec communication lets the other party know that the session key is generated and can be switched to an encrypted connection.
- Then the message “Finished” is sent, meaning that the handshake is completed on the client side. From now on, the connection is secured with a session key. The message contains data (MAC) to verify that the handshake has not been tampered with.
- The server decrypts the pre-master secret and calculates the session key. It then sends a “Change Cipher Spec” message to notify that it switches to an encrypted connection.
- The server also sends a “Finished” message using the newly generated symmetrical session key, and checks the checksum to verify the integrity of the entire handshake.
After these steps, the SSL handshake is complete. Both sides now have a session key and they can communicate through an encrypted and authenticated connection.
The TLS 1.3 handshake is much shorter than its predecessor.
- As with TLS 1.2, the message “Client Hello” starts a handshake, but this time it contains much more information. TLS 1.3 has reduced the number of ciphers supported from 37 to 5. This means that the client can guess which key agreement or exchange protocol will be used, so in addition to the message, it sends its part of the shared key from the expected protocol.
- The server will reply with the message “Server Hello”. Like in handshake 1.2, at this stage the certificate is sent. If the client has correctly guessed the encryption protocol with the attached data and the server has agreed to it, the latter sends its part of the shared key, calculates the session key and completes the transfer with the message “Server Finished”.
- Then the client has all the necessary information, he verifies the SSL certificate and uses two shared keys to calculate his copy of the session key. When this is done, it sends the message “Client Finished”.
TLS Handshake Costs
Historically, one of the complaints about SSL/TLS was that it overloaded the servers with additional costs. This affected the now non-existent notion that HTTPS is slower than HTTP.
Handshakes before TLS 1.2 required a lot of resources and on a large scale could seriously load the server. Even the TLS 1.2 handshakes can slow down if there are many at the same time. Authentication, encryption and decryption are expensive processes.
On small websites this is likely not to lead to a noticeable slowdown, but for corporate systems, where hundreds of thousands of visitors come every day, it can be a big problem. Each new version of handshake significantly simplifies the process: TLS 1.2 performs two phases, while TLS 1.3 fits in just one and supports 0-RTT.
Handshake Improvements TLS 1.3 vs. TLS 1.2
In the above explanation, the handshake is divided into ten separate stages. In reality, many of these things happen simultaneously, so they are often grouped and called phases.
In the TLS 1.2 handshake, two phases can be distinguished. Sometimes additional phases may be needed, but when it comes to numbers, by default the optimal scenario is implied.
Unlike 1.2, the TLS 1.3 handshake fits into one phase, although it would be more accurate to say one and a half, but it is still much faster than TLS 1.2.
No one ever intended to use 37 sets for data encryption, so the protocol evolved. Every time a new algorithm was added, new combinations were added and soon IANA administered 37 different encryption sets.
This is bad for two reasons:
- Such variability leads to erroneous configurations that make Internet users vulnerable to known exploits.
- This made the SSL configuration more confusing.
IETF has eliminated support for all algorithms in TLS 1.3 except for the safest ones, eliminating confusion by limiting selection. In particular, the choice of key exchange method was removed. Diffy-Hellman’s ephemeral scheme became the only way to allow the client to send information about his key together with “Client Hello” in the first part of the handshake. RSA encryption was completely removed along with all other static key exchange schemes.
There is one potential Achilles heel in TLS 1.3.
Zero Receive-Time-Restart Time is 0-RTT
0-RTT is what the whole technological world was aspiring to, and here it is with TLS 1.3. As already mentioned, the TLS handshake has historically been not fast, so it was important to speed it up. The 0-RTT does this by keeping some secret information about the client, usually a session ID or session tickets, so that it can be used for the next connection.
Despite all the advantages of 0-RTT, it contains a couple of potential pitfalls. The mode makes clients susceptible to replay attacks when an attacker who somehow manages to access an encrypted session can retrieve the 0-RTT data, including the client’s first request, and send it back to the server.
However, it is not easy to use an exploit. Probably, this risk is a small price to pay for an extremely useful feature.
From the very beginning, the amount of information sent as open text during a handshake was a cause for concern. Obviously, this is not safe, so the more handshake steps are sent in an encrypted form, the better.
In the TLS 1.2 handshake, the negotiation steps were not protected, instead a simple MAC function was used so that no one would interfere with the transmission. The negotiation stage includes messages “Client Hello” and “Server Hello”.
The MAC function acts as an indicator, but does not give any security guarantee.
You may have heard of an attack that forces the parties to use less secure protocols and functions (downgrade attack). If both the server and the client support outdated encryption – easy to get information about by listening to the connection – the attacker can change the encryption selected by the server to a weaker one. Such attacks are not dangerous in themselves, but they open the door to other known exploits of those cipher sets that were changed to the originally selected one.
The TLS 1.3 handshake uses a digital signature in the early stages of the connection, which makes it more secure and protects against attacks that change the cipherboard. The signature also enables faster and more efficient server authentication.
Now let’s see how these TLS 1.3 handshake updates will be implemented in all three main functions of the SSL/TLS handshake itself.
Crypto TLS handshakes
A cipher is a set of algorithms that determine the parameters of a secure connection.
At the beginning of any connection the very first interaction, “Client Hello”, is a list of supported ciphers. The server chooses the best, most secure option that it supports and meets its requirements. You can look at the cipher and find out all the handshake and connection parameters.
TLS Cryphiffers 1.2
- TLS – protocol.
- ECDHE – key exchange algorithm.
- ECDSA – authentication algorithm.
- AES 128 GCM – symmetric encryption algorithm.
- SHA256 – hashing algorithm.
The above example uses the ephemeral Diffy-Hellman (DH) system with an elliptical curve for key exchange and the elliptical curve digital signature algorithm for authentication. DH can also be connected to an RSA (functioning as a digital signature algorithm) to perform authentication.
Here is a list of the most widely supported TLS 1.2 ciphers:
- Syphon sets
TLS 1.3 Cipher Sets
- TLS – protocol.
- AES 256 GCM is an Authenticated Data Attached Encryption (AEAD) algorithm.
- SHA384 – algorithm of hash key formation function (HKFD).
We already know that we will use some version of Diffy-Hellman exchange of ephemeral keys, but we don’t know the parameters, so the first two algorithms in the TLS 1.2 cipher set are no longer needed. These functions are still running, they just don’t need to be coordinated anymore during a handshake.
From the above example you can see that AES (Advanced Encryption Standard) is used to encrypt large amounts of data. It works in Galois counter mode with 256 bit keys.
Here are five cipher sets that are supported in TLS 1.3:
It is important to remember that when creating version 1.3, the main thing was to improve security and performance. For this purpose in TLS 1.3 key generation algorithm was redesigned and known vulnerabilities were fixed.
Some processes such as message authentication and digital signatures also became better in TLS 1.3 handshake.
Finally, in addition to phasing out old key generation or key exchange algorithms, TLS 1.3 fixes old symmetric ciphers. In TLS 1.3 block ciphers are completely eliminated. The only type of symmetric ciphers allowed in TLS 1.3 is called AEAD (additional data authentication) encryption. It combines message encryption and authentication (MAC) into one function.
Authentication in TLS handshake
Historically, the two main key exchange options are RSA and Diffy-Hellman (DH), nowadays DH is often associated with elliptical curves (ECDH). Despite some basic similarities, there are fundamental differences between these two approaches to key exchange.
In other words, RSA’s TLS handshake is different from ECDH’s TLS handshake.
Authentication in a handshake TLS 1.2
As has just been said, the additional functionality of RSA for authentication with digital signatures requires large, brute force attack resistant keys. The size of these keys greatly increases the cost of calculating, encrypting and decrypting them during a handshake.
On the other hand, if Diffy-Hellman does not perform authentication, what does he do? As mentioned above, DH is often used in conjunction with elliptic curve based cryptography to provide authentication and key exchange.
Elliptical Cryptography (ECC) has much smaller key sizes, which correspond to the elliptical curve on which they are based. There are five suitable curves for this context:
- 192 bit;
- 224 bits;
- 256 bit;
- 384 bit;
- 521 bits.
But this is not the only difference between open/closed ECC keys and RSA keys. They are used for two completely different purposes during the TLS handshake.
In RSA, a public/private key pair is used both for server authentication and for exchanging a symmetrical session key. In fact, it is the successful use of the pre-master secret key that authenticates the server.
With Diffy-Hellman, a public/private key pair is NOT used for exchanging a symmetrical session key. When Dffy-Hellman is involved, the private key is actually associated with the included signature algorithm (ECDSA or RSA).
RSA-authentication process is related to key exchange process. More precisely, key exchange is a part of authentication process.
When the client is provided with SSL certificate of the server, it checks several indicators:
- digital signature using public key;
- chain certificates to ensure that the certificate originates from one of the root certificates in the trusted certificate store;
- expire to make sure it hasn’t expired;
- certificate revocation status.
If all these tests are passed, the last test is carried out – the client encrypts the pre-master secret with the server public key and sends it. Any server may try to issue any SSL/TLS certificate as its own. After all, these are public certificates. This way, the client can authenticate the server by seeing the private key “in action”.
Thus, if the server can decrypt the pre-master secret and use it to calculate the session key, it gets access. This confirms that the server is the owner of the used public and private key pair.
When Duffy Hellman and ECDSA/RSA are used, authentication and key exchange are deployed side by side. And this brings us back to the keys and the way they are used. An RSA public/private key is used for both key exchange and authentication. In DH + ECDSA/RSA an asymmetric key pair is only used for the digital signature or authentication phase.
When a client receives a certificate, it still performs standard checks:
- verifies the signature on the certificate,
- chain of certificates,
- review status.
But the possession of a private key is confirmed in a different way. During the TLS handshake key exchange (step 4), the server uses its private key to encrypt a random number of client and server as well as its DH parameter. It acts as a digital signature of the server and the client can use the associated public key to verify that the server is the rightful owner of the key pair.
Authenticate-in-handshake TLS 1.3
In TLS 1.3, authentication and digital signatures still play an important role, but they have been removed from the cipher sets for ease of reconciliation. They are implemented on the server side and use several algorithms supported by the server due to their security and widespread use. Three basic signature algorithms are allowed in TLS 1.3:
- RSA (signature only),
- Elliptical curve digital signature algorithm (ECDSA),
- Edwards Digital Signature Algorithm (EdDSA).
Unlike the TLS 1.2 handshake, the authentication part of the TLS 1.3 handshake is not related to the key exchange itself. Rather, it is processed in parallel with key exchange and message authentication.
Instead of running a symmetric MAC scheme to check the integrity of the handshake, the server signs the entire decryption hash when it returns “Server Hello” with its shared key part.
The client receives all information transmitted from “Server Hello” and performs a standard series of SSL/TLS certificate authentication checks. This includes checking the signature on the certificate, and then checking if the signature has been added to the decryption hash.
A match confirms that the server holds the secret key.
Key exchange in TLS handshake
If you highlight the main idea of this section, it will sound like this:
RSA facilitates key exchange by allowing the client to encrypt the shared secret and send it to the server where it is used to calculate the corresponding session key. Sharing DH keys does not really require sharing a public key at all, rather both sides create a key together.
If it sounds a bit abstract now, everything should be clear by the end of this section.
RSA Key Exchange
It’s really not right to call it RSA key exchange. It is actually RSA encryption. RSA uses asymmetric encryption to create the session key. Unlike DH, a public/private key pair plays a big role.
This is how it happens:
- Customer and server exchange two prime numbers (x and y), which are called random numbers.
- The Client generates a pre-master secret(a) and then uses the public key of the server to encrypt it and send it to the server.
- The server decrypts the pre-master secret with the corresponding private key. Both sides now have all three input variables and mix them with some pseudo-random functions (PRF) to create the master key.
- Both sides mix the master key with even more PRFs and get matching session keys.
DH key exchange
This is how ECDH works:
- Customer and server exchange two prime numbers (x and y), which are called random numbers.
- The other side chooses a secret number called pre-master secret(a), and calculates it: xa mod y. Then, it sends the result (A) to the other party.
- The other side does the same by choosing its own pre-master secret (b) and calculates the xb mod y and then sends back its value (B).
- Both sides finish this part by taking the given values and repeating the operation. One computes the ba mod y, the other computes the ab mod y.
There is a module parameter property that says that each side will get the same value, which will be the key used for symmetric encryption during the connection.
TLS 1.2 for DH
Now that we have learned how DH is different from RSA, let’s see what a TLS 1.2 DH handshake looks like.
Again, there are many similarities between these two approaches. We will use ECDHE for key exchange and ECDSA for authentication.
- And in the case of RSA, the client starts with a “Client Hello” message that includes a list of cipher sets as well as a random number of clients.
- The server replies with a “Server Hello” message, which includes the selected ciphertext and a random number of servers.
- The server sends its SSL certificate. As with the RSA TLS handshake, the client will perform a series of certificate authentication, but since DH itself cannot authenticate the server, an additional mechanism is required.
- In order to perform authentication, the server takes random numbers from the client and server and the DH parameter that will be used to calculate the session key and encrypts them with its private key. The result will act as a digital signature: the client uses the public key to verify the signature and that the server is the rightful owner of the key pair, and will respond with its own DH parameter.
- The server completes this phase with a “Server Hello Done” message.
- In contrast to RSA, the client does not need to send the pre-master secret to the server using asymmetric encryption, instead the client and the server use the DH settings they exchanged earlier to get the pre-master secret. Each then uses the pre-master secret that they have just calculated to obtain the same session key.
- The Client sends a “Change Cipher Spec” message to inform the other party about their transition to encryption.
- The Client sends the final “Finished” message to let them know that they have completed their handshake portion.
- Analytically, the server sends a “Change Cipher Spec” message.
- The handshake ends with a “Finished” message from the server.
Advantages DHE vs. RSA
There are two main reasons why the cryptographic community prefers to use DHE rather than RSA: perfect direct secrecy and known vulnerabilities.
Earlier you may have wondered what the word “ephemeral” means at the end of DHE and ECDHE. Ephemeric literally means “short-lived”. And it can help to understand Perfect Forward Secrecy (PFS), which is a feature of some key exchange protocols. PFS ensures that the session keys exchanged by the parties cannot be compromised, even if the certificate private key is compromised. In other words, it protects previous sessions from retrieval and decryption. PFS got the highest priority after detecting the Heartbleed error. It is the main component of TLS 1.3.
RSA key exchange vulnerability
There are vulnerabilities that can use a filling (padding) used during key exchange in old versions of RSA (PKCS #1 1.5). This is one aspect of encryption. With RSA, the pre-master secret must be encrypted with the public key and decrypted with the private key. But when this smaller pre-master secret is placed in a bigger public key, it has to be supplemented. In most cases, if you try to guess the filling and send a fake request to the server, you will make a mistake and it will recognize the mismatch and filter it out. But there is a good chance that you will be able to send enough requests to the server to guess the correct completion. The server will then send an error message, which in turn will narrow down the possible value of the pre-master secret. This will allow the cracker to calculate and compromise the key.
This is why RSA was removed in favor of DHE in TLS 1.3.
Key exchange in handshake TLS 1.3
In the TLS 1.3 handshake, due to the limited choice of key exchange schemes, a client can successfully guess the scheme and send his part of the shared key during the initial stage (Client Hello) of the handshake.
RSA was not the only key exchange circuit that was removed in TLS 1.3. Diffy-Hellman’s nonfamily schemas were also eliminated, as was the list of insufficiently secure Diffy-Hellman parameters.
What is meant by unsafe enough parameters? Without going deep into mathematics, the complexity of Diffi-Hellman and most public key cryptosystems is the complexity of solving discrete logarithm problems. The cryptosystem must be complex enough to calculate if the input parameters (random client and server numbers) are unknown, otherwise the whole scheme will be useless. Diffy-Hellman diagrams which could not provide sufficiently large parameters were excluded in TLS 1.3.
- In the beginning of the TLS 1.3 handshake, knowing that the DHE schema of the key agreement will be used, the client includes its part of the shared key based on the expected key exchange scheme in its “Client Hello” message.
- The server receives this information and, if the client has guessed, returns its part of the shared key to “Server Hello”.
- Customer and server calculate session key.
This is very similar to what happens with DH in the TLS 1.2 handshake, except that in TLS 1.3 the key exchange occurs earlier.
Includes the conclusion
SSL/TLS handshake is a fascinating process that is key to a secure Internet, yet it happens so quickly and discreetly that most people never even think about it.
At least until something goes wrong.