worker_process & blockend IO

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

worker_process & blockend IO

Michael Baudino
Hello,

I'm using nginx 0.7.59 on a Debian unstable machine to serve huge files
(300+ MB) from 2 different filesystems : one  is local, the other is
nfs-mounted over an ADSL link.

The nfs filesystem is sooooo slow !
To reduce IO latency, I'm using worker_process 16 (single CPU, though).

How does nginx handles workers ?
Does it choose a worker randomly for each connection ?
Or does it connect to the same worker until it has reached its
worker_connections limit then connect to the next worker ?
In that case, should I set a very low worker_connection to force it to
change worker often ? (I tried it but I got a lot of "worker_connections
are not enough while connecting to upstream")


Thanks in advance.


Reply | Threaded
Open this post in threaded view
|

Re: worker_process & blockend IO

Gabriel Ramuglia
I think it goes without saying that this is hardly a good idea,
serving files that are accessed remotely via ADSL.... My understanding
however is that nginx is designed to be non-blocking everywhere
possible, and I would assume the worker connections would get
distributed more or less evenly, but I'd be happy to hear an official
answer on this as well.

On Wed, Jul 22, 2009 at 10:38 AM, Michael
Baudino<[hidden email]> wrote:

> Hello,
>
> I'm using nginx 0.7.59 on a Debian unstable machine to serve huge files
> (300+ MB) from 2 different filesystems : one  is local, the other is
> nfs-mounted over an ADSL link.
>
> The nfs filesystem is sooooo slow !
> To reduce IO latency, I'm using worker_process 16 (single CPU, though).
>
> How does nginx handles workers ?
> Does it choose a worker randomly for each connection ?
> Or does it connect to the same worker until it has reached its
> worker_connections limit then connect to the next worker ?
> In that case, should I set a very low worker_connection to force it to
> change worker often ? (I tried it but I got a lot of "worker_connections
> are not enough while connecting to upstream")
>
>
> Thanks in advance.
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: worker_process & blockend IO

Igor Sysoev
In reply to this post by Michael Baudino
On Wed, Jul 22, 2009 at 07:38:16PM +0200, Michael Baudino wrote:

> Hello,
>
> I'm using nginx 0.7.59 on a Debian unstable machine to serve huge files
> (300+ MB) from 2 different filesystems : one  is local, the other is
> nfs-mounted over an ADSL link.
>
> The nfs filesystem is sooooo slow !
> To reduce IO latency, I'm using worker_process 16 (single CPU, though).
>
> How does nginx handles workers ?
> Does it choose a worker randomly for each connection ?
> Or does it connect to the same worker until it has reached its
> worker_connections limit then connect to the next worker ?
> In that case, should I set a very low worker_connection to force it to
> change worker often ? (I tried it but I got a lot of "worker_connections
> are not enough while connecting to upstream")

workers get connection randomly.
NFS I/O is blocking and it's better to set up second nginx on remote host
and to use proxy:

   client > nginx/local fs
                 /proxy    > ADSL > nginx/local fs


--
Igor Sysoev
http://sysoev.ru/en/

Reply | Threaded
Open this post in threaded view
|

Re: worker_process & blockend IO

Michael Baudino
Igor Sysoev wrote:

> On Wed, Jul 22, 2009 at 07:38:16PM +0200, Michael Baudino wrote:
>
>> Hello,
>>
>> I'm using nginx 0.7.59 on a Debian unstable machine to serve huge files
>> (300+ MB) from 2 different filesystems : one  is local, the other is
>> nfs-mounted over an ADSL link.
>>
>> The nfs filesystem is sooooo slow !
>> To reduce IO latency, I'm using worker_process 16 (single CPU, though).
>>
>> How does nginx handles workers ?
>> Does it choose a worker randomly for each connection ?
>> Or does it connect to the same worker until it has reached its
>> worker_connections limit then connect to the next worker ?
>> In that case, should I set a very low worker_connection to force it to
>> change worker often ? (I tried it but I got a lot of "worker_connections
>> are not enough while connecting to upstream")
>
> workers get connection randomly.
> NFS I/O is blocking and it's better to set up second nginx on remote host
> and to use proxy:
>
>    client > nginx/local fs
>                  /proxy    > ADSL > nginx/local fs
>


Hi Igor,

Actually I though that proxying to the remote host would be blocking too...
I will try this solution and let you know asap.

Thanks for your fast answer, as usual.

--
Mike
KameHouse Prod.


Reply | Threaded
Open this post in threaded view
|

Re: worker_process & blockend IO

Igor Sysoev
On Thu, Jul 23, 2009 at 09:58:43AM +0200, Michael Baudino wrote:

> Igor Sysoev wrote:
> > On Wed, Jul 22, 2009 at 07:38:16PM +0200, Michael Baudino wrote:
> >
> >> Hello,
> >>
> >> I'm using nginx 0.7.59 on a Debian unstable machine to serve huge files
> >> (300+ MB) from 2 different filesystems : one  is local, the other is
> >> nfs-mounted over an ADSL link.
> >>
> >> The nfs filesystem is sooooo slow !
> >> To reduce IO latency, I'm using worker_process 16 (single CPU, though).
> >>
> >> How does nginx handles workers ?
> >> Does it choose a worker randomly for each connection ?
> >> Or does it connect to the same worker until it has reached its
> >> worker_connections limit then connect to the next worker ?
> >> In that case, should I set a very low worker_connection to force it to
> >> change worker often ? (I tried it but I got a lot of "worker_connections
> >> are not enough while connecting to upstream")
> >
> > workers get connection randomly.
> > NFS I/O is blocking and it's better to set up second nginx on remote host
> > and to use proxy:
> >
> >    client > nginx/local fs
> >                  /proxy    > ADSL > nginx/local fs
> >
>
>
> Hi Igor,
>
> Actually I though that proxying to the remote host would be blocking too...

No, that proxying allows to handle single worker thousands connections.
They may be slow, but they will run simultaneously, while using NFS worker
will blocking on single I/O.

> I will try this solution and let you know asap.


--
Igor Sysoev
http://sysoev.ru/en/