Allow internal redirect to URI x, but deny external request for x?

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Allow internal redirect to URI x, but deny external request for x?

J. Lewis Muir
Hello!

I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that
uses a deployment structure which uses an atomic symlink change for an
atomic app deploy, and I'm wishing to be able to do an internal redirect
in nginx to URL x, but deny an external request to the same URL x so
that I don't serve the same content at more than one URL.  Is there a
way to do that?

For example, given the external request URI

  /my-app/index.html

I want to do an internal redirect to

  /my-app/current/index.html

but deny an external request for that same URI

  /my-app/current/index.html

because I don't want to serve the app from two different URLs (e.g.,
/my-app/ and /my-app/current/).

The app structure on disk is like the Capistrano structure

  https://capistranorb.com/documentation/getting-started/structure/

That is, it's like

  /srv/www/my-app
    current -> releases/1.0.2
    releases
      1.0.0
      1.0.1
      1.0.2

"current" is a symlink.

In my nginx config, I've changed $document_root to $realpath_root in the
appropriate FastCGI parameters, and have the following locations:

  location /my-app/ {
    rewrite ^/my-app/(?!current/)(.*)$ /my-app/current/$1 last;
    index index.php;
  }

  location /my-app/current/ {
    return 404;
  }

  location /my-app/releases/ {
    return 404;
  }

  location ~ ^/my-app/.*?[^/]\.php(?:/|$) {
    include php-fpm-realpath.conf;
  }

Given an external request for a URI that starts with

  /my-app/

this returns 404 after the internal redirect.  If I remove the two
locations that return 404, then it serves the app, but it also allows
external requests such as

  /my-app/current/

which I don't want to allow since that's a duplicate of

  /my-app/

I initially tried using the alias directive which I thought was a better
fit for what I wanted to do

  location /my-app/ {
    alias /srv/www/my-app/current/;
    index index.php;
  }

  location /my-app/current/ {
    return 404;
  }

  location /my-app/releases/ {
    return 404;
  }

  location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) {
    alias /srv/www/my-app/current/$1;
    include php-fpm-realpath.conf;
  }

But that didn't seem to work with the nginx FastCGI implementation.

Thank you!

Lewis
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

J. Lewis Muir
On 08/30, J. Lewis Muir wrote:

> I initially tried using the alias directive which I thought was a better
> fit for what I wanted to do
>
>   location /my-app/ {
>     alias /srv/www/my-app/current/;
>     index index.php;
>   }
>
>   location /my-app/current/ {
>     return 404;
>   }
>
>   location /my-app/releases/ {
>     return 404;
>   }
>
>   location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) {
>     alias /srv/www/my-app/current/$1;
>     include php-fpm-realpath.conf;
>   }
>
> But that didn't seem to work with the nginx FastCGI implementation.

What exactly didn't work when I tried the alias directive, based on the
error log, seems to be that somewhere there's a file op on

  /srv/www/my-app/releases/1.0.2/index.php/my-app/index.php

which is a wrong path; it should be

  /srv/www/my-app/releases/1.0.2/index.php

In my php-fpm-realpath.conf, I have

  fastcgi_split_path_info ^(.+?\.php)(/.*)$;
  if (!-f $realpath_root$fastcgi_script_name) {
    return 404;
  }
  ...
  fastcgi_param DOCUMENT_ROOT $realpath_root;
  fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
  ...

I'm wondering if the fastcgi_split_path_info function or the if
directive is what's emitting the file op message in the error log which
would mean that one or both of $realpath_root or $fastcgi_script_name
are not set to what I expect. (?)

Here are the relevant lines from the error log with the debug option:

  test location: "/my-app/"
  test location: ~ "^/my-app/(.*?[^/]\.php(?:/.*|$))"
  using configuration "/my-app/"
  open index "/srv/www/my-app/current/index.php"
  internal redirect: "/my-app/index.php?"
  rewrite phase: 1
  test location: "/my-app/"
  test location: ~ "^/my-app/(.*?[^/]\.php(?:/.*|$))"
  using configuration "^/my-app/(.*?[^/]\.php(?:/.*|$))"
  rewrite phase: 3
  http script complex value
  http script copy: "/srv/www/my-app/current/"
  http script capture: "index.php"
  http script copy: "/srv/www/my-app/current/"
  http script capture: "index.php"
  http script var: "/srv/www/my-app/releases/1.0.2/index.php"
  http script var: "/my-app/index.php"
  http script copy: ""
  http script var: "/my-app/index.php"
  http script copy: ""
  http script file op 0000000000000001 "/srv/www/my-app/releases/1.0.2/index.php/my-app/index.php"
  http script if
  http finalize request: 404, "/my-app/index.php?" a:1, c:2
  http special response: 404, "/my-app/index.php?"
  http set discard body
  xslt filter header
  HTTP/1.1 404 Not Found

Thank you!

Lewis
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

J. Lewis Muir
On 08/30, J. Lewis Muir wrote:
> I'm wondering if the fastcgi_split_path_info function or the if
> directive is what's emitting the file op message in the error log which
> would mean that one or both of $realpath_root or $fastcgi_script_name
> are not set to what I expect. (?)

Adding

  return 200 "document_root: $document_root\nfastcgi_script_name: $fastcgi_script_name\n";

to the location like this

  location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) {
    alias /srv/www/my-app/current/$1;
    return 200 "realpath_root: $realpath_root\nfastcgi_script_name: $fastcgi_script_name\n";
  }

yields the following:

  $ curl http://localhost/my-app/
  realpath_root: /srv/www/my-app/releases/1.0.2/index.php
  fastcgi_script_name: /my-app/index.php

So, that doesn't seem right!  I was expecting them to be something like:

  realpath_root: /srv/www/my-app/releases/1.0.2
  fastcgi_script_name: /index.php

Puzzled,

Lewis
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

J. Lewis Muir
On 08/30, J. Lewis Muir wrote:

> On 08/30, J. Lewis Muir wrote:
> > I'm wondering if the fastcgi_split_path_info function or the if
> > directive is what's emitting the file op message in the error log which
> > would mean that one or both of $realpath_root or $fastcgi_script_name
> > are not set to what I expect. (?)
>
> Adding
>
>   return 200 "document_root: $document_root\nfastcgi_script_name: $fastcgi_script_name\n";
>
> to the location like this
>
>   location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) {
>     alias /srv/www/my-app/current/$1;
>     return 200 "realpath_root: $realpath_root\nfastcgi_script_name: $fastcgi_script_name\n";
>   }

Hmm, I think I need to call fastcgi_split_path_info first, so now I did

  location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) {
    alias /srv/www/my-app/current/$1;
    fastcgi_split_path_info ^(.+?\.php)(/.*)$;
    return 200 "realpath_root: $realpath_root\nfastcgi_script_name: $fastcgi_script_name\nfastcgi_path_info: $fastcgi_path_info\n";
  }

which yields the following:

  $ curl http://localhost/my-app/
  realpath_root: /srv/www/my-app/releases/1.0.2/index.php
  fastcgi_script_name: /my-app/index.php
  fastcgi_path_info:

That doesn't seem right.

Still puzzled,

Lewis
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

Ian Hobson-3
In reply to this post by J. Lewis Muir
Hi Lewis,

On 30/08/19 18:33, J. Lewis Muir wrote:
> Hello!
>
> I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that
> uses a deployment structure which uses an atomic symlink change for an
> atomic app deploy, and I'm wishing to be able to do an internal redirect
> in nginx to URL x, but deny an external request to the same URL x so
> that I don't serve the same content at more than one URL.  Is there a
> way to do that?
>
You could place the different versions away from the root so they cannot
be obtained from the web. Then they can be served by setting up a
symlink to the desired version.

This can be changed using "ln -sfn version/dir serving/root" and then
restarting nginx to pick up the new version.

By not using redirects, this method should be more efficient.

Regards

Ian

--
Ian Hobson

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

Francis Daly
In reply to this post by J. Lewis Muir
On Fri, Aug 30, 2019 at 12:33:17PM -0500, J. Lewis Muir wrote:

Hi there,

> I'm wishing to be able to do an internal redirect
> in nginx to URL x, but deny an external request to the same URL x so
> that I don't serve the same content at more than one URL.  Is there a
> way to do that?
>
> For example, given the external request URI
>
>   /my-app/index.html
>
> I want to do an internal redirect to
>
>   /my-app/current/index.html
>
> but deny an external request for that same URI
>
>   /my-app/current/index.html

It sounds like you want "internal": http://nginx.org/r/internal

> In my nginx config, I've changed $document_root to $realpath_root in the
> appropriate FastCGI parameters, and have the following locations:
>
>   location /my-app/ {
>     rewrite ^/my-app/(?!current/)(.*)$ /my-app/current/$1 last;
>     index index.php;
>   }
>
>   location /my-app/current/ {
>     return 404;
>   }
>
>   location /my-app/releases/ {
>     return 404;
>   }
>
>   location ~ ^/my-app/.*?[^/]\.php(?:/|$) {
>     include php-fpm-realpath.conf;
>   }

Note that you might want things like "location ^~ /my-app/current/",
if you want that location to handle (and reject) an external request
for /my-app/current/app.php.

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

Francis Daly
In reply to this post by J. Lewis Muir
On Fri, Aug 30, 2019 at 01:58:23PM -0500, J. Lewis Muir wrote:

Hi there,

>   location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) {
>     alias /srv/www/my-app/current/$1;
>     fastcgi_split_path_info ^(.+?\.php)(/.*)$;
>     return 200 "realpath_root: $realpath_root\nfastcgi_script_name: $fastcgi_script_name\nfastcgi_path_info: $fastcgi_path_info\n";
>   }
>
> which yields the following:
>
>   $ curl http://localhost/my-app/
>   realpath_root: /srv/www/my-app/releases/1.0.2/index.php
>   fastcgi_script_name: /my-app/index.php
>   fastcgi_path_info:
>
> That doesn't seem right.

Why not?

http://nginx.org/r/$realpath_root says is it the current root or alias
value, resolving symlinks.

The request was /my-app/, the current request is /my-app/index.php,
and you have alias'ed that to /srv/www/my-app/current/index.php

http://nginx.org/r/$fastcgi_script_name (and what follows) describes
the other variables.

The request is /my-app/index.php and your fastcgi_split_path_info sets
$fastcgi_script_name to "everything up to .php" and $fastcgi_path_info to
"everything after .php", so long as .php is followed by / -- which it
isn't, so both are unchanged from their defaults of "the uri" and "empty".

(I'm somewhat guessing about the last part there; a test can probably
demonstrate whether it is incorrect.)

Cheers,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

lists@lazygranch.com
In reply to this post by Ian Hobson-3
I've been following this thread not really out of need but rather that it is really interesting. That said, I don't think for security you want to "escape" the web root. The risk is that might aid a traversal attack.







          Original Message  



From: [hidden email]
Sent: August 30, 2019 12:01 PM
To: [hidden email]
Reply-to: [hidden email]
Subject: Re: Allow internal redirect to URI x, but deny external request for x?


Hi Lewis,

On 30/08/19 18:33, J. Lewis Muir wrote:
> Hello!
>
> I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that
> uses a deployment structure which uses an atomic symlink change for an
> atomic app deploy, and I'm wishing to be able to do an internal redirect
> in nginx to URL x, but deny an external request to the same URL x so
> that I don't serve the same content at more than one URL.  Is there a
> way to do that?
>
You could place the different versions away from the root so they cannot
be obtained from the web. Then they can be served by setting up a
symlink to the desired version.

This can be changed using "ln -sfn version/dir serving/root" and then
restarting nginx to pick up the new version.

By not using redirects, this method should be more efficient.

Regards

Ian

--
Ian Hobson

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

J. Lewis Muir
In reply to this post by Francis Daly
On 08/30, Francis Daly wrote:

> On Fri, Aug 30, 2019 at 01:58:23PM -0500, J. Lewis Muir wrote:
>
> Hi there,
>
> >   location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) {
> >     alias /srv/www/my-app/current/$1;
> >     fastcgi_split_path_info ^(.+?\.php)(/.*)$;
> >     return 200 "realpath_root: $realpath_root\nfastcgi_script_name: $fastcgi_script_name\nfastcgi_path_info: $fastcgi_path_info\n";
> >   }
> >
> > which yields the following:
> >
> >   $ curl http://localhost/my-app/
> >   realpath_root: /srv/www/my-app/releases/1.0.2/index.php
> >   fastcgi_script_name: /my-app/index.php
> >   fastcgi_path_info:
> >
> > That doesn't seem right.
>
> Why not?
>
> http://nginx.org/r/$realpath_root says is it the current root or alias
> value, resolving symlinks.
>
> The request was /my-app/, the current request is /my-app/index.php,
> and you have alias'ed that to /srv/www/my-app/current/index.php
 
Yes, you're absolutely right.  This is where I went wrong.  I was
initially wishing to use the root directive

  location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) {
    root /srv/www/my-app/current;
    include php-fpm-realpath.conf;
  }

but then the URI would be appended to that, so for a request of

  /my-app/index.php

it would result in

  /srv/www/my-app/current/my-app/index.php

which wasn't what I wanted; instead I wanted

  /srv/www/my-app/current/index.php

I was wishing for a way to specify a new root but with a modified
request URI.  So, I tried the alias directive, and I assumed that
$document_root and $realpath_root would refer to the aliased document
root, but obviously that can't be since nginx has no way of knowing what
the aliased document root should be when all it has is a location alias
which is the full path to the resource.  Sorry for the trouble.

> http://nginx.org/r/$fastcgi_script_name (and what follows) describes
> the other variables.
>
> The request is /my-app/index.php and your fastcgi_split_path_info sets
> $fastcgi_script_name to "everything up to .php" and $fastcgi_path_info to
> "everything after .php", so long as .php is followed by / -- which it
> isn't, so both are unchanged from their defaults of "the uri" and "empty".
>
> (I'm somewhat guessing about the last part there; a test can probably
> demonstrate whether it is incorrect.)

Yep, I'm sure you're right here as well.  Sorry for the trouble; I just
totally missed how this worked.

Thank you for your help!

Lewis
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

jriker1
In reply to this post by J. Lewis Muir
I've been following this, and I would take a slightly different approach.

1. Serve all apps under /{app}/releases/{version}/{path} as you have them
organized in the deployment structure in the file system.

2. Forget about symbolic links and other makeshift versioning/defaulting in
the file system.

3. Use a keyval mapping to handle redirections (307) of
/{app}/current/{stuff} to /{app}/releases/{currentVersion}/{stuff}, where
the keyval mapping provides {app} => {currentVersion}. You can update an
manage this during deployment.

We usually include this in a CI/CD pipeline after deployment to dynamically
switch to the last version (using a curl request to the NGINX API). If you
can't use keyvals, use a static map and dynamically generate that "map"
directive's mapping. Restart NGINX to reflect changes. Keyvals let you do
this on the fly.

The major advantage of this approach is with updates. You are most likely
going to run into issues with browser or proxy caching if you provide
different versions of files/apps under the same path. By having a canonical
form that respects the version structure, you are avoiding this altogether.
Yet, you have the flexibility to run hotfixes (replace existing files in an
existing version without creating a new one), or experimental versions
(which won't update the "current" pointer).

I would try to keep the complexity low.

Cheers,
--j.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,285463,285480#msg-285480

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

Francis Daly
In reply to this post by J. Lewis Muir
On Fri, Aug 30, 2019 at 04:59:36PM -0500, J. Lewis Muir wrote:

Hi there,

> I was wishing for a way to specify a new root but with a modified
> request URI.  So, I tried the alias directive, and I assumed that
> $document_root and $realpath_root would refer to the aliased document
> root, but obviously that can't be since nginx has no way of knowing what
> the aliased document root should be when all it has is a location alias
> which is the full path to the resource.  Sorry for the trouble.

It sounds like your desires are for requests:

 * starts with /my-app/current/ -> reject
 * starts with /my-app/releases/ -> reject
 * matches /my-app/something.php, or /myapp/something.php/anything ->
fastcgi-process the file /srv/www/my-app/current/something.php
 * matches /my-app/something -> just send the file
/srv/www/my-app/current/something

Is that correct? If so -- do exactly that.

For example (but mostly untested):

==
  location ^~ /my-app/current/ { return 200 "nothing to see at /current/\n"; }
  location ^~ /my-app/releases/ { return 200 "nothing to see at /releases/\n"; }
  location ^~ /my-app/ {
    location ~ \.php($|/) {
      fastcgi_split_path_info ^/my-app(/.*php)(.*);
      root /srv/www/my-app/current/;
      include fastcgi.conf;
      fastcgi_pass unix:php.sock;
    }
    alias /srv/www/my-app/current/;
  }
==

Change the "return"s to 404s or whatever; change the "fastcgi_pass"
destination; and don't worry about internal rewrites unless you need them.

fastcgi.conf presumably sets SCRIPT_FILENAME and PATH_INFO and whatever
else is interesting to sensible values; if not, add suitable fastcgi_param
values explicitly here.

You might want an "index index.php" somewhere to handle the request
for /my-app/.

But hopefully, any parts that don't Just Work as-is will leave enough
clues to allow you to find or ask for the solution.

Good luck with it,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

Francis Daly
On Sat, Aug 31, 2019 at 12:21:40AM +0100, Francis Daly wrote:

Hi there,

A few further thoughts here...

> It sounds like your desires are for requests:
>
>  * starts with /my-app/current/ -> reject
>  * starts with /my-app/releases/ -> reject
>  * matches /my-app/something.php, or /myapp/something.php/anything ->

Typo there -- should be "/my-app/".

But note that the "/my-app/" in the request and the "/my-app/" on the
filesystem do not need to the same. (And also: the filesystem /my-app/
for the php files and the filesystem /my-app/ for other files do not
need to be the same; if you want to keep your "static" and "processed"
content separate.)

> fastcgi-process the file /srv/www/my-app/current/something.php
>  * matches /my-app/something -> just send the file
> /srv/www/my-app/current/something
>
> Is that correct? If so -- do exactly that.
>
> For example (but mostly untested):
>
> ==
>   location ^~ /my-app/current/ { return 200 "nothing to see at /current/\n"; }
>   location ^~ /my-app/releases/ { return 200 "nothing to see at /releases/\n"; }
>   location ^~ /my-app/ {
>     location ~ \.php($|/) {
>       fastcgi_split_path_info ^/my-app(/.*php)(.*);

If there might be more than one "php" in the request, that will split on
"the last one". Perhaps you want to split on "the first one followed by
slash". In that case, adjust the regex:

      fastcgi_split_path_info ^/my-app(/.*?php)($|/.*);

>       root /srv/www/my-app/current/;

You did also show a "if (!-f" config, which is "404 if the matching php
file is not present". That can be:

      try_files $fastcgi_script_name =404;

because we have root and the variable set correctly here.

>       include fastcgi.conf;

Possibly the only bits of that file that you care about are:

      fastcgi_param PATH_INFO $fastcgi_path_info;
      fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

so you could just use those lines directly.

>       fastcgi_pass unix:php.sock;
>     }
>     alias /srv/www/my-app/current/;
>   }
> ==

Cheers,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

J. Lewis Muir
In reply to this post by Francis Daly
On 08/31, Francis Daly wrote:

> On Fri, Aug 30, 2019 at 04:59:36PM -0500, J. Lewis Muir wrote:
>
> Hi there,
>
> > I was wishing for a way to specify a new root but with a modified
> > request URI.  So, I tried the alias directive, and I assumed that
> > $document_root and $realpath_root would refer to the aliased document
> > root, but obviously that can't be since nginx has no way of knowing what
> > the aliased document root should be when all it has is a location alias
> > which is the full path to the resource.  Sorry for the trouble.
>
> It sounds like your desires are for requests:
>
>  * starts with /my-app/current/ -> reject
>  * starts with /my-app/releases/ -> reject
>  * matches /my-app/something.php, or /myapp/something.php/anything ->
> fastcgi-process the file /srv/www/my-app/current/something.php
>  * matches /my-app/something -> just send the file
> /srv/www/my-app/current/something
>
> Is that correct? If so -- do exactly that.

Yes!
 

> For example (but mostly untested):
>
> ==
>   location ^~ /my-app/current/ { return 200 "nothing to see at /current/\n"; }
>   location ^~ /my-app/releases/ { return 200 "nothing to see at /releases/\n"; }
>   location ^~ /my-app/ {
>     location ~ \.php($|/) {
>       fastcgi_split_path_info ^/my-app(/.*php)(.*);
>       root /srv/www/my-app/current/;
>       include fastcgi.conf;
>       fastcgi_pass unix:php.sock;
>     }
>     alias /srv/www/my-app/current/;
>   }
> ==

Wow!  I had given up on getting this approach to work.  Honestly, I
don't think I would have thought of your solution.  Brilliant!

I tried it, and it worked after making a few tweaks!  Here's what
worked:

  location ^~ /my-app/current/ {
    return 404;
  }

  location ^~ /my-app/releases/ {
    return 404;
  }

  location ^~ /my-app/ {
    index index.php;
    location ~ [^/]\.php(?:/|$) {
      root /srv/www/my-app/current;
      fastcgi_split_path_info ^/my-app(/.+?\.php)(/.*)?$;
      include php-fpm-realpath.conf;
    }
    alias /srv/www/my-app/current/;
  }

I changed the root directive to come before the fastcgi_split_path_info,
but that was just aesthetic; it worked fine the other way
too.  Previously, I had the fastcgi_split_path_info call in
php-fpm-realpath.conf along with the following file-exists check after
it:

  if (!-f $realpath_root$fastcgi_script_name) {
    return 404;
  }

But I moved the fastcgi_split_path_info call out of
php-fpm-realpath.conf so that I could use the custom regex like in your
suggestion.

So, your solution solved my problem!  Thank you!

On a related note, while considering those return-404s for
/my-app/current/ and /my-app/releases/, I realized that those could, in
theory, conflict with the URIs of a web app if the web app actually used
those same URIs.  For example, maybe the web app is a GitLab-type app
and a URI of /my-app/releases/ might very well be part of the app's URI
set for displaying software releases or something.  This has nothing to
do with your solution and everything to do with my initial design choice
that I was trying to achieve, and it's something I hadn't considered
before.

For my current app, it doesn't use those URIs, so it's not a problem,
but as a general scheme, it's not perfect.  I think one solution would
be to move the app root directory to a different name so that it can't
conflict.  For example, have it live at

  /srv/www/my-app-deployment

or something like that.  Then I could just return a 404 for any request
on that, e.g.:

  location ^~ /my-app-deployment/ {
    return 404;
  }

Thanks again!

Lewis
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

Ian Hobson-3
In reply to this post by lists@lazygranch.com
Hi Mark,

On 30/08/19 22:23, lists wrote:
> I've been following this thread not really out of need but rather that it is really interesting. That said, I don't think for security you want to "escape" the web root. The risk is that might aid a traversal attack.
>
>
I am curious to know how this might work. Nginx itself is safe, so it
would have to be a script. And while those may indeed be vulnerable, is
the vulnerability changed by symlinking the root elsewhere? I don't see
any difference myself, but perhaps you know something I don't.

Regards

Ian

>
>
>
>
>
>   Original Message
>
>
>
> From: [hidden email]
> Sent: August 30, 2019 12:01 PM
> To: [hidden email]
> Reply-to: [hidden email]
> Subject: Re: Allow internal redirect to URI x, but deny external request for x?
>
>
> Hi Lewis,
>
> On 30/08/19 18:33, J. Lewis Muir wrote:
>> Hello!
>>
>> I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that
>> uses a deployment structure which uses an atomic symlink change for an
>> atomic app deploy, and I'm wishing to be able to do an internal redirect
>> in nginx to URL x, but deny an external request to the same URL x so
>> that I don't serve the same content at more than one URL.  Is there a
>> way to do that?
>>
> You could place the different versions away from the root so they cannot
> be obtained from the web. Then they can be served by setting up a
> symlink to the desired version.
>
> This can be changed using "ln -sfn version/dir serving/root" and then
> restarting nginx to pick up the new version.
>
> By not using redirects, this method should be more efficient.
>
> Regards
>
> Ian
>
> --
> Ian Hobson
>
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx
>

--
Ian Hobson
Tel (+351) 910 418 473
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

J. Lewis Muir
In reply to this post by Francis Daly
On 08/31, Francis Daly wrote:

> On Sat, Aug 31, 2019 at 12:21:40AM +0100, Francis Daly wrote:
>
> Hi there,
>
> A few further thoughts here...
>
> > It sounds like your desires are for requests:
> >
> >  * starts with /my-app/current/ -> reject
> >  * starts with /my-app/releases/ -> reject
> >  * matches /my-app/something.php, or /myapp/something.php/anything ->
>
> Typo there -- should be "/my-app/".

Yeah, no problem; I knew what you meant.

> But note that the "/my-app/" in the request and the "/my-app/" on the
> filesystem do not need to the same. (And also: the filesystem /my-app/
> for the php files and the filesystem /my-app/ for other files do not
> need to be the same; if you want to keep your "static" and "processed"
> content separate.)

Got it.

> > fastcgi-process the file /srv/www/my-app/current/something.php
> >  * matches /my-app/something -> just send the file
> > /srv/www/my-app/current/something
> >
> > Is that correct? If so -- do exactly that.
> >
> > For example (but mostly untested):
> >
> > ==
> >   location ^~ /my-app/current/ { return 200 "nothing to see at /current/\n"; }
> >   location ^~ /my-app/releases/ { return 200 "nothing to see at /releases/\n"; }
> >   location ^~ /my-app/ {
> >     location ~ \.php($|/) {
> >       fastcgi_split_path_info ^/my-app(/.*php)(.*);
>
> If there might be more than one "php" in the request, that will split on
> "the last one". Perhaps you want to split on "the first one followed by
> slash". In that case, adjust the regex:
>
>       fastcgi_split_path_info ^/my-app(/.*?php)($|/.*);

Yes, I ended up doing something like that:

  fastcgi_split_path_info ^/synchweb(/.+?\.php)(/.*)?$;

I think you'd want the "\." in yours before the "php", and then I think
the only meaningful difference between ours would be that yours would
match

  /my-app/.php

while mine would not because I used ".+?" to match one or more
reluctantly where you used ".*?" to match zero or more reluctantly.

I was aware of

  https://www.nginx.com/resources/wiki/start/topics/examples/phpfcgi/

which has a location of

  location ~ [^/]\.php(/|$) {

and a fastcgi_split_path_info of

  fastcgi_split_path_info ^(.+?\.php)(/.*)$;

I've always wondered exactly what the idea was with starting that
location regex with "[^/]", though.  Why require the ".php" to be
preceded by a character other than "/"? That means it would match a
request of

  /foo.php

but not

  /.php

Is it because that's considered an invalid PHP file name?  Is it because
some PHP web apps uses a directory named ".php" as a private directory
that should not be served?  I don't know.

> >       root /srv/www/my-app/current/;
>
> You did also show a "if (!-f" config, which is "404 if the matching php
> file is not present". That can be:
>
>       try_files $fastcgi_script_name =404;
>
> because we have root and the variable set correctly here.

Ah, I see!  Thank you for pointing that out!  I like that better.

> >       include fastcgi.conf;
>
> Possibly the only bits of that file that you care about are:
>
>       fastcgi_param PATH_INFO $fastcgi_path_info;
>       fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
>
> so you could just use those lines directly.

Yes, I could, but I was trying to make my php-fpm-realpath.conf reusable
for other apps which is why I was keeping those lines in it.  And, I
think I mentioned this earlier, but I changed them to use $realpath_root
instead of $document_root because of the symlinks and my desire to
support an atomic app deploy with no downtime and no nginx reload.

Thanks again!

Lewis
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

J. Lewis Muir
In reply to this post by Ian Hobson-3
On 08/30, Ian Hobson wrote:

> Hi Lewis,
>
> On 30/08/19 18:33, J. Lewis Muir wrote:
> > Hello!
> >
> > I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that
> > uses a deployment structure which uses an atomic symlink change for an
> > atomic app deploy, and I'm wishing to be able to do an internal redirect
> > in nginx to URL x, but deny an external request to the same URL x so
> > that I don't serve the same content at more than one URL.  Is there a
> > way to do that?
> >
> You could place the different versions away from the root so they cannot be
> obtained from the web. Then they can be served by setting up a symlink to
> the desired version.
>
> This can be changed using "ln -sfn version/dir serving/root" and then
> restarting nginx to pick up the new version.
>
> By not using redirects, this method should be more efficient.
>
> Regards
>
> Ian

Hi, Ian!

Thank you for the suggestion!  That's an interesting idea, and that
would avoid needing to exclude those URIs from being served and I think
avoid needing to change the document root and maybe some other things.

I toyed around with some designs and came up with four; I think I like
the third one best, but I'm not sure, maybe the fourth one is better.

= Design #1

Web app root:

  /srv/www/app/my-app
    current -> releases/1.0.2
    releases
      1.0.0
      1.0.1
      1.0.2

nginx server document root:

  /srv/www/host/localhost
    my-app -> /srv/www/app/my-app/current

Comments:

  This tries to share apps between virtual hosts.  It provides no way
  for different virtual hosts to serve the same app but with the app
  configured differently.  Also, all virtual hosts have to serve the
  same version of the app.

= Design #2

Web app root:

  /srv/www/app/my-app
    1.0.0
    1.0.1
    1.0.2

nginx server document root:

  /srv/www/host/localhost
    my-app -> /srv/www/app/my-app/1.0.2

Comments:

  Similar to design #1 except now the virtual hosts can server different
  versions of the app, but for a given version, the app is still
  configured in the exact same way for all virtual hosts.

= Design #3

Web app root:

  /srv/www/localhost/app
    my-app
      1.0.0
      1.0.1
      1.0.2

nginx server document root:

  /srv/www/localhost/root
    my-app -> ../app/my-app/1.0.2

Comments:

  I like this because now each virtual host has its own set of deployed
  apps, so the apps can be configured specifically for that particular
  virtual host.

= Design #4

Web app root:

  /srv/www/localhost/app
    my-app
      current -> releases/1.0.2
      releases
        1.0.0
        1.0.1
        1.0.2

nginx server document root:

  /srv/www/localhost/root
    my-app -> ../app/my-app/current

Comments:

  Similar to design #3 but moves where the "current" symlink lives.  So,
  the deployed version of the app is controlled under the web app root
  rather under the nginx server document root.  Maybe this is a little
  better because the nginx config reference to "my-app" can be sure to
  be correct and can be changed if desired, and be unrelated to which
  version of the app is deployed via the "current" symlink in the web
  app root.  The downside is two symlinks (i.e., a symlink to a symlink)
  as opposed to one.

Regards,

Lewis
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

lists@lazygranch.com
In reply to this post by Ian Hobson-3
Nginx does detect these traversal attacks. They come up as a 400 error. I got two yesterday. But out of paranoia, I wouldn't leave the web root. There is always some zero day.

That traversal attack was from some new to me Hong Kong hosting company and earned a place on my firewall block. Blocking just keeps the log file size down. There will be others.

https://null-byte.wonderhowto.com/how-to/perform-directory-traversal-extract-sensitive-information-0185558/

I have run dotdotpwn. Lots of false positives. It takes forever.

On nearly a daily basis, some entity gets hacked because of a misconfiguration. So I make sure I have secured the low hanging fruit. I watch file ownership and permissions. That is free. I don't have a WAF but I use Nginx maps and pattern match common hacks, given them the 444. Simple stuff like if you request some WordPress feature you get flagged because I don't run WordPress. I found a list of bad user agents on GitHub that I flag on.





  Original Message  



From: [hidden email]
Sent: August 31, 2019 7:41 AM
To: [hidden email]
Reply-to: [hidden email]
Subject: Re: Allow internal redirect to URI x, but deny external request for x?


Hi Mark,

On 30/08/19 22:23, lists wrote:
> I've been following this thread not really out of need but rather that it is really interesting. That said, I don't think for security you want to "escape" the web root. The risk is that might aid a traversal attack.
>
>
I am curious to know how this might work. Nginx itself is safe, so it
would have to be a script. And while those may indeed be vulnerable, is
the vulnerability changed by symlinking the root elsewhere? I don't see
any difference myself, but perhaps you know something I don't.

Regards

Ian

>
>
>
>
>
>   Original Message
>
>
>
> From: [hidden email]
> Sent: August 30, 2019 12:01 PM
> To: [hidden email]
> Reply-to: [hidden email]
> Subject: Re: Allow internal redirect to URI x, but deny external request for x?
>
>
> Hi Lewis,
>
> On 30/08/19 18:33, J. Lewis Muir wrote:
>> Hello!
>>
>> I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that
>> uses a deployment structure which uses an atomic symlink change for an
>> atomic app deploy, and I'm wishing to be able to do an internal redirect
>> in nginx to URL x, but deny an external request to the same URL x so
>> that I don't serve the same content at more than one URL.  Is there a
>> way to do that?
>>
> You could place the different versions away from the root so they cannot
> be obtained from the web. Then they can be served by setting up a
> symlink to the desired version.
>
> This can be changed using "ln -sfn version/dir serving/root" and then
> restarting nginx to pick up the new version.
>
> By not using redirects, this method should be more efficient.
>
> Regards
>
> Ian
>
> --
> Ian Hobson
>
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx
>

--
Ian Hobson
Tel (+351) 910 418 473
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

Francis Daly
In reply to this post by J. Lewis Muir
On Sat, Aug 31, 2019 at 09:10:09AM -0500, J. Lewis Muir wrote:
> On 08/31, Francis Daly wrote:

Hi there,

> >  * starts with /my-app/current/ -> reject
> >  * starts with /my-app/releases/ -> reject

Actually -- those two "rejects" should not be needed.

The app probably should not be installed in the general nginx document
root directory. The "alias" mentioning the "app/current" directory means
that that is the only part that nginx will try to serve files from. The
"root" mentioning the "app/current" directory means that that is the only
part that nginx will look in (try_files) and mention to the fastcgi server
(fastcgi_param).

So the "app/releases" directory will not be web-accessible; and the
"app/current" directory will only be accessible by the explicit url that
you define.

So the full config should be of the form

  location ^~ /app-url/ {
    alias /active-app-dir/;
    location ~ \.php(/|$) {
      root /active-app-dir;
      fastcgi_split_path_info ^/app-url(/.*?php)($|/.*);
      try_files $fastcgi_script_name =404;
      include fastcgi.conf;
      fastcgi_pass unix:php.sock;
    }
  }


Adjust regexes based on what you want.

"app-url" can be "my-app". "/active-app-dir" can be "/opt/app/releases/3"
or "/opt/my-app/current" or anything else.

> I changed the root directive to come before the fastcgi_split_path_info,
> but that was just aesthetic; it worked fine the other way
> too.

Yes. For many directives in nginx, the order in the config file does
not matter.

("rewrite" module directives use the order. And regex locations use
their order. I think that most others do not. Your fastcgi server might
care about the order that the fastcgi_param directives had, but nginx
does not.)

>  Previously, I had the fastcgi_split_path_info call in
> php-fpm-realpath.conf along with the following file-exists check after

Using "realpath" should not affect nginx at all. nginx invites the
fastcgi server to use pathname2 instead of pathname1; so the fastcgi
server is the only thing that should care.

> For my current app, it doesn't use those URIs, so it's not a problem,
> but as a general scheme, it's not perfect.  I think one solution would
> be to move the app root directory to a different name so that it can't
> conflict.  For example, have it live at
>
>   /srv/www/my-app-deployment

As above -- that shouldn't matter. If the app is not deployed in the web
server document root, only the specific alias/root directory is accessible,
and the entire url-space below that is available.

(And you can have one url prefix /my-app/, and a separate url prefix
/my-app-prev/, which uses the next most recent version. Restrict access
to that location{} to your source IP address, and you can do regression
testing between the two.)

> or something like that.  Then I could just return a 404 for any request
> on that, e.g.:
>
>   location ^~ /my-app-deployment/ {
>     return 404;
>   }

If you don't want nginx to serve content from the my-app-deployment
directory, it is probably easier for it to be somewhere other than
/srv/www.

It is hard to misconfigure nginx in that case.

Cheers,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

J. Lewis Muir
On 08/31, Francis Daly wrote:

> On Sat, Aug 31, 2019 at 09:10:09AM -0500, J. Lewis Muir wrote:
> > On 08/31, Francis Daly wrote:
>
> Hi there,
>
> > >  * starts with /my-app/current/ -> reject
> > >  * starts with /my-app/releases/ -> reject
>
> Actually -- those two "rejects" should not be needed.
>
> The app probably should not be installed in the general nginx document
> root directory. The "alias" mentioning the "app/current" directory means
> that that is the only part that nginx will try to serve files from. The
> "root" mentioning the "app/current" directory means that that is the only
> part that nginx will look in (try_files) and mention to the fastcgi server
> (fastcgi_param).
>
> So the "app/releases" directory will not be web-accessible; and the
> "app/current" directory will only be accessible by the explicit url that
> you define.
>
> So the full config should be of the form
>
>   location ^~ /app-url/ {
>     alias /active-app-dir/;
>     location ~ \.php(/|$) {
>       root /active-app-dir;
>       fastcgi_split_path_info ^/app-url(/.*?php)($|/.*);
>       try_files $fastcgi_script_name =404;
>       include fastcgi.conf;
>       fastcgi_pass unix:php.sock;
>     }
>   }

I can't believe this!  Another great insight!  Thank you!  I haven't
tried it, but yes, that looks way better, and your observation about not
needing the two rejects puts to rest my incorrect belief that there was
a URI namespace problem with the app directory structure (i.e., with the
"/current/" and "/releases/" URI components).  And yes, I will move the
app root out of the nginx document root to avoid the unnecessary risk of
an nginx misconfiguration.

> Adjust regexes based on what you want.
>
> "app-url" can be "my-app". "/active-app-dir" can be "/opt/app/releases/3"
> or "/opt/my-app/current" or anything else.
 
Got it.

> >  Previously, I had the fastcgi_split_path_info call in
> > php-fpm-realpath.conf along with the following file-exists check after
>
> Using "realpath" should not affect nginx at all. nginx invites the
> fastcgi server to use pathname2 instead of pathname1; so the fastcgi
> server is the only thing that should care.

Hmm, I might not be understanding this.  The rationale of using
$realpath_root instead of $document_root was to make it so that a
new version of the web app could be deployed atomically at any time
by changing the "current" symlink, meaning that, for example, if the
"current" symlink were changed right in the middle of a request being
handled in PHP, it wouldn't be possible for one part of the request to
execute in PHP in the old version of the app and another part to execute
in the new version.  By using $realpath_root, the idea was to ensure
that for any given request being handled in PHP, it would execute in its
entirety in the same version of the web app.

That's why I was doing in php-fpm-realpath.conf

  if (!-f $realpath_root$fastcgi_script_name) {
    return 404;
  }

and

  fastcgi_param DOCUMENT_ROOT $realpath_root;
  fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;

So, does that make sense, or am I still not understanding this?  I don't
know what you mean by "nginx invites the fastcgi server to use pathname2
instead of pathname1."  What are pathname1 and pathname2?
 

> > For my current app, it doesn't use those URIs, so it's not a problem,
> > but as a general scheme, it's not perfect.  I think one solution would
> > be to move the app root directory to a different name so that it can't
> > conflict.  For example, have it live at
> >
> >   /srv/www/my-app-deployment
>
> As above -- that shouldn't matter. If the app is not deployed in the web
> server document root, only the specific alias/root directory is accessible,
> and the entire url-space below that is available.
 
Understood.

> (And you can have one url prefix /my-app/, and a separate url prefix
> /my-app-prev/, which uses the next most recent version. Restrict access
> to that location{} to your source IP address, and you can do regression
> testing between the two.)
>
> > or something like that.  Then I could just return a 404 for any request
> > on that, e.g.:
> >
> >   location ^~ /my-app-deployment/ {
> >     return 404;
> >   }
>
> If you don't want nginx to serve content from the my-app-deployment
> directory, it is probably easier for it to be somewhere other than
> /srv/www.
>
> It is hard to misconfigure nginx in that case.

Agreed; I will move it out of there.

Thank you!

Lewis
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Allow internal redirect to URI x, but deny external request for x?

Francis Daly
On Sat, Aug 31, 2019 at 04:55:26PM -0500, J. Lewis Muir wrote:
> On 08/31, Francis Daly wrote:
> > On Sat, Aug 31, 2019 at 09:10:09AM -0500, J. Lewis Muir wrote:

Hi there,

> > Using "realpath" should not affect nginx at all. nginx invites the
> > fastcgi server to use pathname2 instead of pathname1; so the fastcgi
> > server is the only thing that should care.
>
> Hmm, I might not be understanding this.  The rationale of using
> $realpath_root instead of $document_root was to make it so that a
> new version of the web app could be deployed atomically at any time
> by changing the "current" symlink,
...
>   fastcgi_param DOCUMENT_ROOT $realpath_root;
>   fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
>
> So, does that make sense, or am I still not understanding this?  I don't
> know what you mean by "nginx invites the fastcgi server to use pathname2
> instead of pathname1."  What are pathname1 and pathname2?

nginx does not "do" php. nginx does not care what your fastcgi server
will do with the key/value pairs that it sends. nginx cares that the
fastcgi server gives a valid response to the request that nginx makes.

Typically, your fastcgi server will use the value associated with
SCRIPT_FILENAME as "the name of the file to execute". If your fastcgi
server fails to find / read / execute that file, it will return its own
error indication.

(So your "if", or the more common "try_files", is just an early-out,
to sometimes avoid involving the fastcgi server. It may happen that the
file is present when nginx looks for it, but is absent when the fastcgi
server looks for it -- so that case does have to be handled anyway.)


In this case, if $document_root is /srv/www/my-app/current/ and
$realpath_root is /srv/www/my-app/releases/1.0.2/, and the script
name is test.php, then with one config, nginx would send the string
"/srv/www/my-app/current/test.php", and with the other config nginx
would send the string "/srv/www/my-app/releases/1.0.2/test.php".

(That is "pathname1" vs "pathname2".)

So if "one request" involves the fastcgi server reading
"/srv/www/my-app/current/test.php", and then reading a bunch of other
files in the same directory -- then I guess that unfortunate timing
could lead to it reading some files from releases/1.0.1 and some from
releases/1.0.2. (Assuming that it opens the directory afresh each time --
which can't be ruled out.)

But if "the app" involves a http request to part1.php and then a http
request to part2.php (or: a second http request to part1.php), I don't
think that the symlink+realpath thing will prevent those two requests
going to different release versions.

All the best,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
12