Nginx as Reverse Proxy for multiple servers binded to proxy using UNIX sockets - how to reached in LAN

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Nginx as Reverse Proxy for multiple servers binded to proxy using UNIX sockets - how to reached in LAN

Francis Daly
On Tue, Oct 16, 2018 at 09:23:33PM +0200, Stefan Müller wrote:

Hi there,

> 1. documentation
>    Is there any additional document for the -c command. I find only:
>     1. http://nginx.org/en/docs/switches.html

That page indicates that "-c" means that this nginx instance uses this
named config file instead of the compiled-in default one. I'm not sure
what other documentation is possible.

>    but none of them says that it will start an independent instances of
>    nginx.

Every time you run the nginx binary, you run a new instance of nginx,
independent of any other.

Not all command-line argument combinations to nginx mean that it should
run a web server that never exits; many combinations mean "do this one
thing and exit".

The one command-line argument that is used to interact with an
already-running nginx -- -s -- does that interaction by sending a signal
to the appropriate process-id, in exactly the same way that the "kill"
or "pkill" binaries would. The new short-term nginx instance is still
independent of the already-running one, in the same way that "kill"
would be independent of it.

> 2. command line
>    I assume, that the command line parameters refer to a single
>    instance environment. How do I use the command line parameters for a
>    specific instance? Is it like this nginx -V "pid
>    /var/run/nginx-user1.pid"?

Command-line parameters are used when you start a new instance of
nginx. They do not refer to any other instance (with the one "-s"
exception mentioned above).

> 3. root and non-root
>    only the master / proxy server instance need root access in order to
>    bind to ports <1024 and change its user-id to the one defined in
>    the|user <https://nginx.org/en/docs/ngx_core_module.html#user>|
>    directive in the main context of its .conf file.
>    The other / backend instances don't have to be started as root as
>    they don't need to bind to ports, they communicate via UNIX sockets
>    so all permission are managed by the user account management.
>    That is the same, what you said, isn't it?

Yes.

An nginx that wants to "listen" on a place that only root can "listen"
on, or write to places that only root can write to, needs to start as
root. Any other nginx does not need to. (Again, excepting "-s".)

> 4. all in all there two layers of isolation
>     1. dynamic content provide such as PHP

nginx does not "do" PHP. Or much in the way of "dynamic" content (in
the sense that it seems to be meant here).

>        each "virtual host" / server{} blocks has its own PHP pool. So

That's not an nginx thing, but it is a thing that you can configure if
you want to.

>        the user for pool server{}/user1/ cannot see  the pool
>        server{}/user2/. If /user1/ gets hacked, the hacker won't get
>        immidate acceass to /user2/ or the nginx  master process, correct?

"PHP" is (probably) run under a fastcgi server, as whatever system user
you want that service to run as. You can run multiple fastcgi servers
if you want to, each under their own user account.

nginx can be a client of that server if you use "fastcgi_pass" in your
nginx config that points to the server.


You have nginx-main, starts as root then switches to userM. "The browser"
is a client of this server.

You have (multiple) nginx-userN, runs as userN. nginx-main is a client
of this server.

You have (multiple) php-userP, runs as userP. nginx-userN is a client
of this server.

What specific threat model are you concerned with here?

When a thing gets hacked, is that thing running under the control of root,
or userM, or one of the multiple userNs, or one of the multiple userPs?

The access the outsider gains will (presumably) not exceed the access
that that user has. "root" will be able to access lots of things. "userM"
must be able to access the "userN" listening socket. "userN" must be
able to access the matching "userP" listening socket. "userP" must be
able to access the php files.

>     2. independent instances of nginx.
>        In case the master process is breach for what ever reason, the
>        hacker cannot see the other serves as long as he won't get root
>        privileges of the machine and there is the same exploit in the
>        other servers, correct?

Again, I'm unsure what threat model you are concerned about here.

If someone breaks the main nginx to get "root" access, they have root
access. If they break the main nginx to get "userM" access, they will
be able to access nginx-userN (because userM can do that). Is that
enough access to also break nginx-userN so that they can get "userN"
access? (nginx-main and nginx-userN run the same binary. Is the breakage
due to configuration, which might be different between the two? Or due
to something inherent, which will be the same between the two?)

Without trying to be dismissive: if someone can break nginx to gain user
access, it is unlikely to be you or me that they will be attacking first.

I think that based on history, "badly written CGI scripts" (which in
this case corresponds to "badly written PHP") is the most likely way
that web things will be broken. In this design, that PHP runs under
the control of the fastcgi server, as a user userP. If that happens,
the outsider will have access as userP to do whatever the PHP script
and fastcgi server allow them to do. nginx is not involved except as
(probably) an initial pass-through tunnel.

If userP has access to turn off your fridge or reconfigure your nginx-main
or send a million emails or read secret files on your filesystem, then
the outsider will probably have access to do those things too.

Only you can decide what level of risk you're happy with.

Good luck with it,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Nginx as Reverse Proxy for multiple servers binded to proxy using UNIX sockets - how to reached in LAN

Stefan Mueller
Hi Francis,
thank you for the update
  1. documentation
    I've read up and down the documentation and other sources. I could not find anything saying that if you run the nginx binary, you run a new instance of nginx, independent of any other. Where is it mentioned that nginx is multi instance capable  application only indirectly in Upgrading To a New Binary On The Fly in Starting, Stopping, and Restarting NGINX and How To Upgrade Nginx In-Place Without Dropping Client Connections. I found rather sources telling me to install it twice as here:  How to install Multiple Nginx instances in same Server

  2. command line
    Slowly I being understanding reading Starting, Stopping, and Restarting NGINX and Controlling NGINX Processes at Runtime. If you run multiple instances you use the signal -s with a reference to the desired PID otherwise it will pick the default on what is /var/run/nginx.pid , isn't?
    e.g. I have a master and two user nginx instances relaoding each of them would be:
    master: nginx -s reload
    user1:  nginx -s reload "pid /var/run/nginx_user1.pid"
    user2:  nginx -s reload "pid /var/run/nginx_user2.pid"

    or by its PIDs listed by ps aux -P | grep nginx
    master: nginx -s reload
    user1:  nginx -s reload PID_NoOfUser1Instance
    user2:  nginx -s reload
    PID_NoOfUser1Instance

    Making sure that I understood correctly  nginx -g "pid /var/run/nginx.pid; worker_processes `sysctl -n hw.ncpu`;" means to replace the number of workers by the count of CPUs in nginx master process with the process id given in /var/run/nginx.pid, doesn't it?

  3. root and non-root
    done :-)

  4.  all in all there two layers of isolation
    I didn't mean the nginx does not do the e.g. php interpretation, you just tell nginx where to find the php interpreter, what will do the job and feeds the content through a CGI.
    That what you described is more or less what I meat but I'm bit confused as you say, that if the main nginx is compromised the hacker could gain root privileges? Is this also possible after the main process changed its user after the binding of the ports >1024?
    After all, I just want to have a clear picture of what is at risk when some part of the entire server environment is compromised and how can I minimize the risk by isolation of all involved parts.
    I agree that not many of us will be a target to hack into nginx and if someone tries to hack our servers there are properly weaker parts as the so often mentioned  "badly written PHP" scripts. All that I want is to have a good idea about the risks to put me in a position to do a proper effort vs. risk assessment.
    I reckon the information gathered so far put me in a quite good position, doesn't it?

thx

Stefan

On 17.10.2018 22:59, Francis Daly wrote:
On Tue, Oct 16, 2018 at 09:23:33PM +0200, Stefan Müller wrote:

Hi there,

1. documentation
   Is there any additional document for the -c command. I find only:
    1. http://nginx.org/en/docs/switches.html
That page indicates that "-c" means that this nginx instance uses this
named config file instead of the compiled-in default one. I'm not sure
what other documentation is possible.

   but none of them says that it will start an independent instances of
   nginx.
Every time you run the nginx binary, you run a new instance of nginx,
independent of any other.

Not all command-line argument combinations to nginx mean that it should
run a web server that never exits; many combinations mean "do this one
thing and exit".

The one command-line argument that is used to interact with an
already-running nginx -- -s -- does that interaction by sending a signal
to the appropriate process-id, in exactly the same way that the "kill"
or "pkill" binaries would. The new short-term nginx instance is still
independent of the already-running one, in the same way that "kill"
would be independent of it.

2. command line
   I assume, that the command line parameters refer to a single
   instance environment. How do I use the command line parameters for a
   specific instance? Is it like this nginx -V "pid
   /var/run/nginx-user1.pid"?
Command-line parameters are used when you start a new instance of
nginx. They do not refer to any other instance (with the one "-s"
exception mentioned above).

3. root and non-root
   only the master / proxy server instance need root access in order to
   bind to ports <1024 and change its user-id to the one defined in
   the|user <https://nginx.org/en/docs/ngx_core_module.html#user>|
   directive in the main context of its .conf file.
   The other / backend instances don't have to be started as root as
   they don't need to bind to ports, they communicate via UNIX sockets
   so all permission are managed by the user account management.
   That is the same, what you said, isn't it?
Yes.

An nginx that wants to "listen" on a place that only root can "listen"
on, or write to places that only root can write to, needs to start as
root. Any other nginx does not need to. (Again, excepting "-s".)

4. all in all there two layers of isolation
    1. dynamic content provide such as PHP
nginx does not "do" PHP. Or much in the way of "dynamic" content (in
the sense that it seems to be meant here).

       each "virtual host" / server{} blocks has its own PHP pool. So
That's not an nginx thing, but it is a thing that you can configure if
you want to.

       the user for pool server{}/user1/ cannot see  the pool
       server{}/user2/. If /user1/ gets hacked, the hacker won't get
       immidate acceass to /user2/ or the nginx  master process, correct?
"PHP" is (probably) run under a fastcgi server, as whatever system user
you want that service to run as. You can run multiple fastcgi servers
if you want to, each under their own user account.

nginx can be a client of that server if you use "fastcgi_pass" in your
nginx config that points to the server.


You have nginx-main, starts as root then switches to userM. "The browser"
is a client of this server.

You have (multiple) nginx-userN, runs as userN. nginx-main is a client
of this server.

You have (multiple) php-userP, runs as userP. nginx-userN is a client
of this server.

What specific threat model are you concerned with here?

When a thing gets hacked, is that thing running under the control of root,
or userM, or one of the multiple userNs, or one of the multiple userPs?

The access the outsider gains will (presumably) not exceed the access
that that user has. "root" will be able to access lots of things. "userM"
must be able to access the "userN" listening socket. "userN" must be
able to access the matching "userP" listening socket. "userP" must be
able to access the php files.

    2. independent instances of nginx.
       In case the master process is breach for what ever reason, the
       hacker cannot see the other serves as long as he won't get root
       privileges of the machine and there is the same exploit in the
       other servers, correct?
Again, I'm unsure what threat model you are concerned about here.

If someone breaks the main nginx to get "root" access, they have root
access. If they break the main nginx to get "userM" access, they will
be able to access nginx-userN (because userM can do that). Is that
enough access to also break nginx-userN so that they can get "userN"
access? (nginx-main and nginx-userN run the same binary. Is the breakage
due to configuration, which might be different between the two? Or due
to something inherent, which will be the same between the two?)

Without trying to be dismissive: if someone can break nginx to gain user
access, it is unlikely to be you or me that they will be attacking first.

I think that based on history, "badly written CGI scripts" (which in
this case corresponds to "badly written PHP") is the most likely way
that web things will be broken. In this design, that PHP runs under
the control of the fastcgi server, as a user userP. If that happens,
the outsider will have access as userP to do whatever the PHP script
and fastcgi server allow them to do. nginx is not involved except as
(probably) an initial pass-through tunnel.

If userP has access to turn off your fridge or reconfigure your nginx-main
or send a million emails or read secret files on your filesystem, then
the outsider will probably have access to do those things too.

Only you can decide what level of risk you're happy with.

Good luck with it,

	f

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

RE: Nginx as Reverse Proxy for multiple servers binded to proxy using UNIX sockets - how to reached in LAN

Reinis Rozitis
> 2. command line
> Slowly I being understanding reading Starting, Stopping, and Restarting NGINX and Controlling NGINX Processes at Runtime. If you run
> multiple instances you use the signal -s with a reference to the desired PID otherwise it will pick the default on what is /var/run/nginx.pid ,
> isn't?
> e.g. I have a master and two user nginx instances relaoding each of them would be:
> master: nginx -s reload
> user1:  nginx -s reload "pid /var/run/nginx_user1.pid"
> user2:  nginx -s reload "pid /var/run/nginx_user2.pid"
>
> Making sure that I understood correctly  nginx -g "pid /var/run/nginx.pid; worker_processes `sysctl -n hw.ncpu`;" means to replace the
> number of workers by the count of CPUs in nginx master process with the process id given in /var/run/nginx.pid, doesn't it?

While you can specify the pid (process id) from the command line imo it's more simple to put  the pid file directive in each nginx configuration ( http://nginx.org/en/docs/ngx_core_module.html#pid ). For example nginx_user1.conf could have pid logs/nginx_user1.pid; , nginx_user2.conf  - pid logs/nginx_user2.pid; etc..

Then for the cli commands you can just specify particular configuration file and nginx will pick up the pid automatically. For example to reload user2 nginx instance the cli command would be:
nginx -s reload -c /path/to/user2_nginx.conf

rr

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Nginx as Reverse Proxy for multiple servers binded to proxy using UNIX sockets - how to reached in LAN

Francis Daly
In reply to this post by Stefan Mueller
On Fri, Oct 19, 2018 at 11:27:09PM +0200, Stefan Müller wrote:

Hi there,

> thank you for the update

You're welcome.

> 1. documentation
>    I've read up and down the documentation and other sources. I could
>    not find anything saying that if you run the nginx binary, you run a
>    new instance of nginx, independent of any other.

In my experience, pretty much every time I run a command, it starts a
new instance of the command that I just ran. That's what I expect
to happen.

The few exceptions are the ones that should be documented as exceptional.

So the documentation as-is reads fine to me, and I am not in a position
to suggest alternate words that might be clearer for others.

> 2. command line
>    Slowly I being understanding reading Starting, Stopping, and
>    Restarting NGINX

>    e.g. I have a master and two user nginx instances relaoding each of
>    them would be:
>    master: nginx -s reload
>    user1:  nginx -s reload "pid /var/run/nginx_user1.pid"
>    user2:  nginx -s reload "pid /var/run/nginx_user2.pid"
>    or by its PIDs listed by ps aux -P | grep nginx
>    master: nginx -s reload
>    user1:  nginx -s reload PID_NoOfUser1Instance
>    user2:  nginx -s reload PID_NoOfUser1Instance

My reading of that documentation does not lead me to the same conclusion
that you apparently came to. That's ok; it should be straightforward
enough for you to test.

Run "nginx -c conf1", "nginx -c conf2", and "nginx". Check "ps" or read
log files, so that you can see which instances are running.

Then run "nginx -s stop" with whatever other arguments look right, and
see what the response is, and what nginx instances are running afterwards.

I suspect that the system is much simpler than you might think it is.

If you want to stop the nginx that you started with "nginx -c conf1",
you can run "nginx -c conf1 -s stop".

>    Making sure that I understood correctly nginx -g "pid
>    /var/run/nginx.pid; worker_processes `sysctl -n hw.ncpu`;" means to
>    replace the number of workers by the count of CPUs in nginx master
>    process with the process id given in /var/run/nginx.pid, doesn't it?

I'm afraid I really don't know how to answer that, without just saying
"no" and repeating my previous mail.

What happened when you tried to run that command?

> 4.   all in all there two layers of isolation

>    After all, I just want to have a clear picture of what is at risk
>    when some part of the entire server environment is compromised and
>    how can I minimize the risk by isolation of all involved parts.

That is a good thing to aim for.

Only you can do your risk assessment. You have access to exactly what
nginx does -- it's in the directory "src". If that's not something you
want to read, then you will have to accept someone else's summary of it,
and put your trust in them.

If I were to write "the entire system is now, and will forever remain,
totally unbreakable, no matter what configuration you use", would that
change your opinion of it? Why should you accept random words in a
stranger's email? (The quoted statement is almost certainly incorrect,
by the way.)

Some people and some companies seem willing to run the software. Maybe
they did their own risk assessments and decided it was ok, or it was ok
so long as they avoid some specific parts; or maybe they just assumed
that it was good enough because other people were using it.

I'm happy to use nginx for my purposes.

One nginx which can read all relevant static files, plus one fastcgi
server for each php application that I don't want to fully assess,
is enough for me.

>    I reckon the information gathered so far put me in a quite good
>    position, doesn't it?

I hope so.

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
12