Bouncycastle does not retransmit the last server handshake flight on receipt of the retransmitted last client flight

[JonathanLennox]

When performing a DTLS handshake as the server, BouncyCastle does not respond to a retransmission of the client's second DTLS flight with a retransmission of the server flight.

See the attached PCAP (gzip'd because GitHub doesn't allow .pcap files).
Stuck-DTLS-handshake.pcap.gz

This is a handshake between Chrome 117.0.5938.62 and BouncyCastle 1.75, with artificial packet reordering introduced with the Linux tc tool (netem delay 100.0ms 75.0ms). A bug in Chrome's stack causes it not to handle the case where the packet containing the server's Encrypted "Finish" message precedes the packet containing the server's ChangeCipherSpec message, so Chrome retransmits its "Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message" flight (which happens to be aggregated into a single packet).

This problem appears to have been introduced in BouncyCastle 1.74; BouncyCastle 1.73 properly retransmits the server flight. 1.76 shows the same behavior as 1.75.

Tested with Chrome (as mentioned) using WebRTC, talking to the Jitsi Videobridge which is using (the various versions of ) BouncyCastle-Java.

[JonathanLennox]

It looks like the problem was introduced in commit 2307836 by @peterdettman , and the issue is that DTLSRecordLayer.receivePendingRecord doesn't recognize the client's change_cipher_spec from the previous epoch as being part of the retransmitted handshake.

[JonathanLennox]

More specifically, when receivePendingRecord fails to parse the change_cipher_spec, it clears out the entirety of the rest of recordQueue, which includes the encrypted "Finish" message. Thus the "Finish" message is never processed, and so the server flight is not retransmitted.

[JonathanLennox]

A fix is now in #1491.

[JonathanLennox]

Note this is not just an artificial problem, though my reproducer is artificial - with Jitsi-Videobridge using BC 1.75 we had lots of reports in production of the DTLS-SRTP handshake not completing successfully and people not being able to join calls, which we concluded was because there was some random packet loss and/or reordering in the DTLS handshake which BC couldn't correctly recover from.

[JonathanLennox]

The issue on the Chrome side is reported at https://bugs.chromium.org/p/webrtc/issues/detail?id=15495, but as their behavior is reasonable (they discard encrypted packets that outrace their corresponding key change) as they mention the BC issues are more serious.

[peterdettman]

Thanks very much for the report and the diagnosis. I will review the proposed fix.

[JonathanLennox]

Fixed in #1491