weight and balancing in upstream proxy

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

weight and balancing in upstream proxy

Frank Liu
Hi,

How does nginx balances traffic to upstream with different weight? If I have 3 servers in upstream, with weight 1, 2, 4, assuming all are healthy, will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2, 3, 3, 3, 3? If I have two servers with both weight 50, will nginx will 50 requests to server 1, and then 50 to server 2, or will it calculate the ration to be 1:1 and send one after another?

Thanks!
Frank


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weight and balancing in upstream proxy

nginx mailing list
That is an interesting questions as intuitively, people could think the former behavior applies.

If I got the source code right, and as the docs state, nginx is following a weighted round-robin algorithm.
It thus means it will go over the same list of servers everytime a peer needs to be chosen (ie for every request), and pick the first not having depleted its weight allocation.

To me, it would use the latter of your proposals.
​Please correct me if I am wrong, so incorrect information does not propagate too much. :o)​
---
B. R.

On Wed, Apr 12, 2017 at 11:50 PM, Frank Liu <[hidden email]> wrote:
Hi,

How does nginx balances traffic to upstream with different weight? If I have 3 servers in upstream, with weight 1, 2, 4, assuming all are healthy, will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2, 3, 3, 3, 3? If I have two servers with both weight 50, will nginx will 50 requests to server 1, and then 50 to server 2, or will it calculate the ration to be 1:1 and send one after another?

Thanks!
Frank


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weight and balancing in upstream proxy

Maxim Dounin
Hello!

On Thu, Apr 13, 2017 at 10:09:16AM +0200, B.R. via nginx wrote:

> That is an interesting questions as intuitively, people could think the
> former behavior applies.
>
> If I got the source code
> <https://trac.nginx.org/nginx/browser/nginx/src/http/ngx_http_upstream_round_robin.c#L507>
> right, and as the docs
> <https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream>
> state, nginx is following a weighted round-robin
> <https://en.wikipedia.org/wiki/Weighted_round_robin> algorithm.
> It thus means it will go over the same list of servers everytime a peer
> needs to be chosen (ie for every request), and pick the first not having
> depleted its weight allocation.
>
> To me, it would use the latter of your proposals.
> ​Please correct me if I am wrong, so incorrect information does not
> propagate too much. :o)​

The Wikipedia link in question doesn't seem to be related to what
nginx does.

--
Maxim Dounin
http://nginx.org/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weight and balancing in upstream proxy

Frank Liu
Hi Maxim,
Thanks for pointing out the link is not related. Do you have the answer to the original question or a related link?
Thanks
Frank

> On Apr 13, 2017, at 7:34 AM, Maxim Dounin <[hidden email]> wrote:
>
> Hello!
>
>> On Thu, Apr 13, 2017 at 10:09:16AM +0200, B.R. via nginx wrote:
>>
>> That is an interesting questions as intuitively, people could think the
>> former behavior applies.
>>
>> If I got the source code
>> <https://trac.nginx.org/nginx/browser/nginx/src/http/ngx_http_upstream_round_robin.c#L507>
>> right, and as the docs
>> <https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream>
>> state, nginx is following a weighted round-robin
>> <https://en.wikipedia.org/wiki/Weighted_round_robin> algorithm.
>> It thus means it will go over the same list of servers everytime a peer
>> needs to be chosen (ie for every request), and pick the first not having
>> depleted its weight allocation.
>>
>> To me, it would use the latter of your proposals.
>> Please correct me if I am wrong, so incorrect information does not
>> propagate too much. :o)
>
> The Wikipedia link in question doesn't seem to be related to what
> nginx does.
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weight and balancing in upstream proxy

Francis Daly
In reply to this post by Frank Liu
On Wed, Apr 12, 2017 at 02:50:08PM -0700, Frank Liu wrote:

Hi there,

> How does nginx balances traffic to upstream with different weight? If I
> have 3 servers in upstream, with weight 1, 2, 4, assuming all are healthy,
> will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2, 3, 3, 3,
> 3?

If you want to know what your current nginx version does, it should not
be too difficult to test:

One nginx.conf. One http section. One upstream{} listing multiple
ip:ports. One server{} which proxy_pass:es to that upstream. Multiple
server{}s, each of which listens on one ip:port, writes to a different
access_log, and does something like "return 200 ok;".

Then for increasing numbers X, "GET /X" on the main server. Look at
the individual access log files to see which server handled /1, which
handled /2, etc. By the time you get to 700, you'll either see that
there is a reliably repeating pattern, or you'll see that it is probably
randomish.


If you actually care about what pattern is used; or if you want to
guarantee that the same pattern will be used in future nginx versions;
then get your preferred code written and use that instead of whatever
nginx uses.


If you want to know what guarantee there is that the behaviour will not
change in the future: I'd say "none", except that there is a good chance
that what is written in the documentation will be honoured. Paraphrasing
that, for the above case: for 7 requests, 1 will go to the first server;
2 to the second; and 4 to the third.

> If I have two servers with both weight 50, will nginx will 50 requests
> to server 1, and then 50 to server 2, or will it calculate the ration to be
> 1:1 and send one after another?

Same answer: it does not seem to difficult to test, if you don't want
to read the available source.

Good luck with it!

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weight and balancing in upstream proxy

Frank Liu
Hi Francis,

Thanks for confirming that there is no document, and any results observed through testing or reviewing code will not be guaranteed. I guess it is purposely undocumented so that people won't rely on one behavior and we are free to change.

Regards,
Frank


On Thu, Apr 13, 2017 at 4:49 PM, Francis Daly <[hidden email]> wrote:
On Wed, Apr 12, 2017 at 02:50:08PM -0700, Frank Liu wrote:

Hi there,

> How does nginx balances traffic to upstream with different weight? If I
> have 3 servers in upstream, with weight 1, 2, 4, assuming all are healthy,
> will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2, 3, 3, 3,
> 3?

If you want to know what your current nginx version does, it should not
be too difficult to test:

One nginx.conf. One http section. One upstream{} listing multiple
ip:ports. One server{} which proxy_pass:es to that upstream. Multiple
server{}s, each of which listens on one ip:port, writes to a different
access_log, and does something like "return 200 ok;".

Then for increasing numbers X, "GET /X" on the main server. Look at
the individual access log files to see which server handled /1, which
handled /2, etc. By the time you get to 700, you'll either see that
there is a reliably repeating pattern, or you'll see that it is probably
randomish.


If you actually care about what pattern is used; or if you want to
guarantee that the same pattern will be used in future nginx versions;
then get your preferred code written and use that instead of whatever
nginx uses.


If you want to know what guarantee there is that the behaviour will not
change in the future: I'd say "none", except that there is a good chance
that what is written in the documentation will be honoured. Paraphrasing
that, for the above case: for 7 requests, 1 will go to the first server;
2 to the second; and 4 to the third.

> If I have two servers with both weight 50, will nginx will 50 requests
> to server 1, and then 50 to server 2, or will it calculate the ration to be
> 1:1 and send one after another?

Same answer: it does not seem to difficult to test, if you don't want
to read the available source.

Good luck with it!

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weight and balancing in upstream proxy

nginx mailing list
In reply to this post by Maxim Dounin
Please, enlighten us then.
---
B. R.

On Thu, Apr 13, 2017 at 4:34 PM, Maxim Dounin <[hidden email]> wrote:
Hello!

On Thu, Apr 13, 2017 at 10:09:16AM +0200, B.R. via nginx wrote:

> That is an interesting questions as intuitively, people could think the
> former behavior applies.
>
> If I got the source code
> <https://trac.nginx.org/nginx/browser/nginx/src/http/ngx_http_upstream_round_robin.c#L507>
> right, and as the docs
> <https://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream>
> state, nginx is following a weighted round-robin
> <https://en.wikipedia.org/wiki/Weighted_round_robin> algorithm.
> It thus means it will go over the same list of servers everytime a peer
> needs to be chosen (ie for every request), and pick the first not having
> depleted its weight allocation.
>
> To me, it would use the latter of your proposals.
> ​Please correct me if I am wrong, so incorrect information does not
> propagate too much. :o)​

The Wikipedia link in question doesn't seem to be related to what
nginx does.

--
Maxim Dounin
http://nginx.org/


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weight and balancing in upstream proxy

Francis Daly
In reply to this post by Frank Liu
On Thu, Apr 13, 2017 at 10:08:21PM -0700, Frank Liu wrote:

Hi there,

> Thanks for confirming that there is no document, and any results observed
> through testing or reviewing code will not be guaranteed. I guess it is
> purposely undocumented so that people won't rely on one behavior and we are
> free to change.

The guarantee is written in the licence, available at
http://nginx.org/LICENSE.

Something similar is true for every aspect of every piece of behaviour
of every piece of software.

If there is specific behaviour you want, you always have the option
of finding someone who will commit to providing and preserving that
behaviour in future versions of the nginx that you use.

Or, if you know through testing and code review that the behaviour of
1.10.0 is exactly what you want, you always have the option of continuing
to use that version. (Compiled with your version of a compiler and
support libraries, and running with your version of an operating system
and runtime libraries.)


I suspect that the upstream round-robin implementation will not change in
the future without good reason, just like I expect that HTTP/1.0 support
will not change without good reason. But if things do change, and if
my comfort depends on the old behaviour persisting, I have the freedom
to ensure that the old behaviour does persist on my systems.

So I'd say that if it matters, you can see what the current code does,
and base your system on that.

Note that if your system relies on all upstreams remaining always
available, your system is probably fragile -- if you need a specific
pattern of access, and one of your upstreams is unavailable when nginx
wants it, the current nginx "recovery" behaviour may or may not be what
you want. And *that* behaviour (I think) depends on timings of incoming
requests, so may not be fully under your control anyway.

Good luck with it,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weight and balancing in upstream proxy

Aleksandar Lazic
In reply to this post by Frank Liu
Hi.

Am 12-04-2017 23:50, schrieb Frank Liu:

> Hi,
>
> How does nginx balances traffic to upstream with different weight?
> If I have 3 servers in upstream, with weight 1, 2, 4, assuming all are
> healthy, will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2,
> 2, 3, 3, 3, 3?
> If I have two servers with both weight 50, will nginx will 50 requests
> to server 1, and then 50 to server 2, or will it calculate the ration
> to be 1:1 and send one after another?

You can find a explanation of the algorithm in the commit messages.

http://hg.nginx.org/nginx/rev/c90801720a0c
http://hg.nginx.org/nginx/rev/d05ab8793a69
http://hg.nginx.org/nginx/rev/0811376954e4

I have found this with this query.

http://hg.nginx.org/nginx/log?rev=weight

Have you seen this doc?
http://nginx.org/en/docs/http/load_balancing.html

Best Regards
Aleks
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: weight and balancing in upstream proxy

Frank Liu
Hi Aleks,

Those information are extremely helpful. Much appreciated!

Regards,
Frank


On Fri, Apr 14, 2017 at 1:47 AM, Aleksandar Lazic <[hidden email]> wrote:
Hi.

Am 12-04-2017 23:50, schrieb Frank Liu:

Hi,

How does nginx balances traffic to upstream with different weight?
If I have 3 servers in upstream, with weight 1, 2, 4, assuming all are healthy, will nginx send traffic to server 1, 2, 3, 2, 3, 3, 3 or 1, 2, 2, 3, 3, 3, 3?
If I have two servers with both weight 50, will nginx will 50 requests to server 1, and then 50 to server 2, or will it calculate the ration to be 1:1 and send one after another?

You can find a explanation of the algorithm in the commit messages.

http://hg.nginx.org/nginx/rev/c90801720a0c
http://hg.nginx.org/nginx/rev/d05ab8793a69
http://hg.nginx.org/nginx/rev/0811376954e4

I have found this with this query.

http://hg.nginx.org/nginx/log?rev=weight

Have you seen this doc?
http://nginx.org/en/docs/http/load_balancing.html

Best Regards
Aleks


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Loading...