SSL handshake attack mitigation

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

SSL handshake attack mitigation

j94305
Greetings!

I run a bunch of sites on nginx-plus-r19 (OpenSSL 1.0.2k-fips) and was
recently hit by a nasty DDoS SSL handshake attack.

I noticed nginx worker processes suddenly eating all available CPU and the
"Handshakes failed" counter in the nginx plus dashboard suddenly climbing
out of proportion to the successful handshakes.

If I understand correctly, the limit_req directive would not be effective in
mitigating this type of attack since the SSL handshake occurs earlier in the
request chain.

I ended up setting the error_log level to "info" and feeding the failed
handshake client IPs to fail2ban.

My first question is regarding the particular error log messages produced
during the attack - see example below:

[info] 8050#8050: *146 SSL_do_handshake() failed (SSL: error:14094416:SSL
routines:ssl3_read_bytes:sslv3 alert certificate unknown:SSL alert number
46) while SSL handshaking, client: XXX.XXX.XXX.XXX, server: 0.0.0.0:443

The "certificate unknown" seems to suggest that nginx is trying to verify
the certificate of the client, yet "ssl_verify_client" should be off by
default, so why does nginx care about that certificate?

My second question - is there a better way of mitigating this type of
attack? (Preferably without putting an expensive firewall in front of
nginx)

I would also like to put in a feature request to have a limit_req equivalent
for SSL handshakes.

Thanks!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,286113,286113#msg-286113

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: SSL handshake attack mitigation

lists@lazygranch.com
IMHO you did the right thing with fail2ban. I don't see how a firewall is "expensive" other than they they are a little RAM heavy. Half the internet traffic is bots. That doesn't even count the hot linkers. So the reality is you will need a firewall to block what doesn't have eyeballs, namely datacenters. At a bare minimum you should be blocking all of AWS from everything except port 25.

Firewalls have a low CPU load. I think when the dust settles, killing the trouble makers at the source is the way to go.





  Original Message  



From: [hidden email]
Sent: November 6, 2019 11:41 AM
To: [hidden email]
Reply-to: [hidden email]
Subject: SSL handshake attack mitigation


Greetings!

I run a bunch of sites on nginx-plus-r19 (OpenSSL 1.0.2k-fips) and was
recently hit by a nasty DDoS SSL handshake attack.

I noticed nginx worker processes suddenly eating all available CPU and the
"Handshakes failed" counter in the nginx plus dashboard suddenly climbing
out of proportion to the successful handshakes.

If I understand correctly, the limit_req directive would not be effective in
mitigating this type of attack since the SSL handshake occurs earlier in the
request chain.

I ended up setting the error_log level to "info" and feeding the failed
handshake client IPs to fail2ban.

My first question is regarding the particular error log messages produced
during the attack - see example below:

[info] 8050#8050: *146 SSL_do_handshake() failed (SSL: error:14094416:SSL
routines:ssl3_read_bytes:sslv3 alert certificate unknown:SSL alert number
46) while SSL handshaking, client: XXX.XXX.XXX.XXX, server: 0.0.0.0:443

The "certificate unknown" seems to suggest that nginx is trying to verify
the certificate of the client, yet "ssl_verify_client" should be off by
default, so why does nginx care about that certificate?

My second question - is there a better way of mitigating this type of
attack? (Preferably without putting an expensive firewall in front of
nginx)

I would also like to put in a feature request to have a limit_req equivalent
for SSL handshakes.

Thanks!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,286113,286113#msg-286113

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: SSL handshake attack mitigation

Sergey A. Osokin-2
In reply to this post by j94305
Hi,

On Wed, Nov 06, 2019 at 02:41:15PM -0500, mogwai wrote:
> Greetings!
>
> I run a bunch of sites on nginx-plus-r19 (OpenSSL 1.0.2k-fips) and was
> recently hit by a nasty DDoS SSL handshake attack.

there are several techics are avaialble to mitigate DDoS attacks with NGINX and
NGINX Plus, please see the following link for details:
https://www.nginx.com/blog/mitigating-ddos-attacks-with-nginx-and-nginx-plus/

Since you have NGINX Plus version I'd recommend to contact the NGINX Plus
Support team through the Customer Portal, https://cs.nginx.com/ or via email
[hidden email].

--
Sergey Osokin
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: SSL handshake attack mitigation

Sergey Kandaurov
In reply to this post by j94305

> On 6 Nov 2019, at 22:41, mogwai <[hidden email]> wrote:
>
> My first question is regarding the particular error log messages produced
> during the attack - see example below:
>
> [info] 8050#8050: *146 SSL_do_handshake() failed (SSL: error:14094416:SSL
> routines:ssl3_read_bytes:sslv3 alert certificate unknown:SSL alert number
> 46) while SSL handshaking, client: XXX.XXX.XXX.XXX, server: 0.0.0.0:443
>
> The "certificate unknown" seems to suggest that nginx is trying to verify
> the certificate of the client, yet "ssl_verify_client" should be off by
> default, so why does nginx care about that certificate?

That's opposite: nginx received a certificate_unknown alert message
from a client for some reason while in handshake.

--
Sergey Kandaurov

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx