Behaviour of map changes when using it's result in limit_req/limit_req_zone

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Behaviour of map changes when using it's result in limit_req/limit_req_zone

Felix Gläske
I try to achieve the following:
I want to throttle requests based on upstream response codes.
If a user (repeatedly) fails to login, and therefor the upstream server responds with 403, the requests should be throttled. Not for 2xx or other response codes. Only 403.

Online I found a solution using a `map` and `limit_req_zone`/`limit_req`.

So far so good, I tried out the things separately and the mapping as well as the throttling works.
But when putting them together things get weird, at least as far as I understand so far.

My config:
  error_log  /dev/stdout info;

  daemon off;

  events {
    worker_connections  4096;
  }

  http {
    map $status $bad_guy {
      ~^[2] 1;
      ~^[4] $remote_addr;
      default "default";
    }

    map $status $wup {
      ~^[2] 1;
      ~^[4] $remote_addr;
      default "default";
    }

    log_format   main '$remote_addr - $status ->$wup<- ->$bad_guy<- ';
    access_log   /dev/stdout  main;

    limit_req_zone $bad_guy zone=by_status:10m rate=60r/m;

    server {
      listen       8000;

      location / {
        proxy_pass <a href="http://127.0.0.1:8888$request_uri;">http://127.0.0.1:8888$request_uri;
        limit_req zone=by_status burst=2;
      }
    }
  }

I have two identical maps. The only difference is that $bad_guy is used with "limit_req_zone".
I would assume that in the log statement the values for $wup and $bad_guy are the same since they are based on maps with the same rules.
However, this is _not_ the case:

Scenario 1: Upstream reports 403
Log:
  127.0.0.1 - 403 ->127.0.0.1<- ->default<-

As we can see, only $wup has the expected value and $bad_guy is default for some reason.

Scenario 2: Upstream reports 226
Log:
  127.0.0.1 - 226 ->1<- ->default<-

The same behaviour as before.


Let's change the configuration a little bit and comment out the line "limit_req zone=by_status burst=2;"

Scenario 3: Upstream reports 403
Log:
  127.0.0.1 - 403 ->127.0.0.1<- ->127.0.0.1<-

Ok, now the values seem all right.

Scenario 4: Upstream reports 226
Log:
  127.0.0.1 - 226 ->1<- ->1<-
Ok, good values again.


Final question: Why does using "limit_req" change the behaviour/outcome of the "map"? Maybe I'm not aware enough of the internals but this seems weird to me and also blocks me from reaching my goal to throttle failed login attempts.

I hope someone can shine some light on that.
Thanks!







_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Behaviour of map changes when using it's result in limit_req/limit_req_zone

Francis Daly
On Fri, Oct 11, 2019 at 10:30:14AM +0200, Felix Gläske wrote:

Hi there,

> I want to throttle requests based on upstream response codes.

It is unlikely to be useful to try to limit incoming requests based on
something that is not in the incoming request.

(More specifically: based on something that is not available at the time
that the limiting should happen.)

> If a user (repeatedly) fails to login, and therefor the upstream server responds with 403, the requests should be throttled. Not for 2xx or other response codes. Only 403.
>
> Online I found a solution using a `map` and `limit_req_zone`/`limit_req`.

Could you link to this solution? Perhaps it is valid in some circumstances.

>   http {
>     map $status $bad_guy {
>       ~^[2] 1;
>       ~^[4] $remote_addr;
>       default "default";
>     }
>
>     map $status $wup {
>       ~^[2] 1;
>       ~^[4] $remote_addr;
>       default "default";
>     }

http://nginx.org/r/$status -- that is the response status.

>     log_format   main '$remote_addr - $status ->$wup<- ->$bad_guy<- ';
>     access_log   /dev/stdout  main;
>
>     limit_req_zone $bad_guy zone=by_status:10m rate=60r/m;

>       location / {
>         proxy_pass <a href="http://127.0.0.1:8888$request_uri;">http://127.0.0.1:8888$request_uri;
>         limit_req zone=by_status burst=2;

A request comes in. "limit_req" says "check if this should be limited".

limit_req_zone says "what value has $bad_guy got right now?".

map says "$status is empty, therefore $bad_guy is default".

limiting happens based on "default".

After the request completes, "log_format" wants the values of four
variables. The nginx-built-in ones have their appropriate values. The
extra ones have whatever value they have -- $bad_guy is "default" because
that is what "map" set it to previously; "$wup" was not set previously,
so now "map" sets it based on $status -- which now is not empty.

You can mark $bad_guy "volatile" (http://nginx.org/r/map) if you want
"map" to set it to a new value at log time; but that will not affect
the fact that limit_req was based on the value "default".


> I would assume that in the log statement the values for $wup and $bad_guy are the same since they are based on maps with the same rules.

No.

http://nginx.org/r/map:

"""
Since variables are evaluated only when they are used, the mere
declaration even of a large number of “map” variables does not add
any extra costs to request processing.
"""

> Let's change the configuration a little bit and comment out the line "limit_req zone=by_status burst=2;"
>
> Scenario 3: Upstream reports 403
> Log:
>   127.0.0.1 - 403 ->127.0.0.1<- ->127.0.0.1<-
>
> Ok, now the values seem all right.

Same analysis as above, except in this config $bad_guy is not used before
"log_format", so it has no value before logging, so "map" sets it based
on the value of $status at logging time.

> Final question: Why does using "limit_req" change the behaviour/outcome of the "map"?

Timing.

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx