Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

deoren
Hi,

Thanks for reading this.

My apologies if this has been answered before, but after much reading
(official docs, mailing list discussions, etc.) I'm still not completely
clear on whether this is supported. I know it's a hangup on my part, but
I've not managed to get past the stumbling point yet.

#1) Have a web app answer to https://subdomain.example.com/ with app
related urls like '/login', '/issues', and requests for static resources
with URL paths like '/static/styles.css'. The app runs on localhost at
http://127.0.0.1:3000 and is proxied by nginx with a direct mapping of
http://subdomain.example.com/ to http://127.0.0.1:3000/ and it works
well. As indicated, this setup does not use a sub-URI, but treats '/' as
its root URL path.

#2) I'd like to move the web application to
https://subdomain.example.com/sub-uri/ by setting up location block like
so (spacing condensed for display purposes):

     location /flask-demo {
         root /var/www/passenger-python-flask-demo;
         proxy_set_header X-Forwarded-Proto $scheme;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header X-Forwarded-Host $host;
         proxy_pass http://127.0.0.1:3000/;
     }

The trailing slash was added in an attempt to map /flask-demo/SOMETHING
to /SOMETHING in the application's point of view.

That works well for a test web app where everything is contained in a
single file, but when the static assets are referenced by the HTML
output the user's browser attempts to pull static content from '/'
instead of '/flask-demo/'.

I've found that for this and other web applications (demo and
production) that I've tested thus you can configure the base URL in the
web application itself.

Does nginx support redirecting those requests for static resources
requests to the associated sub-URI without modifying the web
application? If it was only one web application I could setup location
blocks for specific patterns, but if I plan on running multiple web
applications on a single FQDN (perhaps even different instances of the
same web app), each in a separate sub-URI, those web applications might
all make requests to '/static/styles.css' based on their original
configuration.

I'm hoping there is a way to isolate each web application based on the
initial location block match, thereby catching follow-up requests for
static resources related to the first request and prepend the sub-URI.
Thus a request for '/static/styles.css' becomes
'/flask-demo/static/styles.css' if I visit '/flask-demo' and if I visit
'/other-app' the static request becomes '/other-app/static/styles.css',
all without modifying the web application to know it is being run from a
sub-URI.

I assume the answer is "no, this is not supported", but I wanted to ask
just to make sure I'm not overlooking something. Francis Daly's remarks
on the "URL-Rewriting not working" thread that I've quoted from below
seems to suggest it might be possible, but probably isn't worth the
headache:

 > Note that if you want to reverse-proxy a back-end web
 > service at a different part of the url hierarchy to
 > where it believes it is installed, in general you need
 > the web service to help.
 >
 > That is, if you want the back-end / to correspond to
 > the front-end /x/, then if the back-end ever links to
 > something like /a, you will need that to become
 > translated to /x/a before it leaves the front-end. In
 > general, the front-end cannot do that translation.
 >
 > So you may find it easier to configure the back-end to
 > be (or to act as if it is) installed below /x/ directly.
 >
 > Otherwise things can go wrong.

I found the 'proxy_redirect' directive, but it doesn't appear to do what
I'm looking for. Instead, it appears to be designed specifically to do
things like prevent having the client access http://127.0.0.1:3000/ 
instead of http://127.0.0.1:80/ (as is shown in my example).

I've used nginx for years, but only in very basic configurations. This
is something new to me and I'm struggling to wrap my head around it.
Thank you for reading this and any advice you can offer.
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

Michael Salmon
Yes you can but you need to rewrite the text as it passes through nginx using a sub_filter. It isn't that hard but it is tedious.

/Michael Salmon

> On 9 May 2017, at 01:18, deoren <[hidden email]> wrote:
>
> Hi,
>
> Thanks for reading this.
>
> My apologies if this has been answered before, but after much reading (official docs, mailing list discussions, etc.) I'm still not completely clear on whether this is supported. I know it's a hangup on my part, but I've not managed to get past the stumbling point yet.
>
> #1) Have a web app answer to https://subdomain.example.com/ with app related urls like '/login', '/issues', and requests for static resources with URL paths like '/static/styles.css'. The app runs on localhost at http://127.0.0.1:3000 and is proxied by nginx with a direct mapping of http://subdomain.example.com/ to http://127.0.0.1:3000/ and it works well. As indicated, this setup does not use a sub-URI, but treats '/' as its root URL path.
>
> #2) I'd like to move the web application to https://subdomain.example.com/sub-uri/ by setting up location block like so (spacing condensed for display purposes):
>
>    location /flask-demo {
>        root /var/www/passenger-python-flask-demo;
>        proxy_set_header X-Forwarded-Proto $scheme;
>        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>        proxy_set_header X-Forwarded-Host $host;
>        proxy_pass http://127.0.0.1:3000/;
>    }
>
> The trailing slash was added in an attempt to map /flask-demo/SOMETHING to /SOMETHING in the application's point of view.
>
> That works well for a test web app where everything is contained in a single file, but when the static assets are referenced by the HTML output the user's browser attempts to pull static content from '/' instead of '/flask-demo/'.
>
> I've found that for this and other web applications (demo and production) that I've tested thus you can configure the base URL in the web application itself.
>
> Does nginx support redirecting those requests for static resources requests to the associated sub-URI without modifying the web application? If it was only one web application I could setup location blocks for specific patterns, but if I plan on running multiple web applications on a single FQDN (perhaps even different instances of the same web app), each in a separate sub-URI, those web applications might all make requests to '/static/styles.css' based on their original configuration.
>
> I'm hoping there is a way to isolate each web application based on the initial location block match, thereby catching follow-up requests for static resources related to the first request and prepend the sub-URI. Thus a request for '/static/styles.css' becomes '/flask-demo/static/styles.css' if I visit '/flask-demo' and if I visit '/other-app' the static request becomes '/other-app/static/styles.css', all without modifying the web application to know it is being run from a sub-URI.
>
> I assume the answer is "no, this is not supported", but I wanted to ask just to make sure I'm not overlooking something. Francis Daly's remarks on the "URL-Rewriting not working" thread that I've quoted from below seems to suggest it might be possible, but probably isn't worth the headache:
>
> > Note that if you want to reverse-proxy a back-end web
> > service at a different part of the url hierarchy to
> > where it believes it is installed, in general you need
> > the web service to help.
> >
> > That is, if you want the back-end / to correspond to
> > the front-end /x/, then if the back-end ever links to
> > something like /a, you will need that to become
> > translated to /x/a before it leaves the front-end. In
> > general, the front-end cannot do that translation.
> >
> > So you may find it easier to configure the back-end to
> > be (or to act as if it is) installed below /x/ directly.
> >
> > Otherwise things can go wrong.
>
> I found the 'proxy_redirect' directive, but it doesn't appear to do what I'm looking for. Instead, it appears to be designed specifically to do things like prevent having the client access http://127.0.0.1:3000/ instead of http://127.0.0.1:80/ (as is shown in my example).
>
> I've used nginx for years, but only in very basic configurations. This is something new to me and I'm struggling to wrap my head around it. Thank you for reading this and any advice you can offer.
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx

smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

rnburn
location /flask-demo/ {
         root /var/www/passenger-python-flask-demo;
         proxy_set_header X-Forwarded-Proto $scheme;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header X-Forwarded-Host $host;
         rewrite    /flask-demo/([^/]+) /$1 break;
         proxy_pass http://127.0.0.1:3000/;
     }

And then add additional location blocks/rewrites to handle static content.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,274104,274106#msg-274106

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

Francis Daly
In reply to this post by deoren
On Mon, May 08, 2017 at 06:17:02PM -0500, deoren wrote:

Hi there,

> I'm still
> not completely clear on whether this is supported.

nginx does the nginx side as much as it can. Whether it all works is
almost entirely down to the web app.

You cannot reliably reverse-proxy an arbitrary web app to a different
place in the url hierarchy than that at which it believes it is installed.

You can unreliably do it, if that is good enough for you.

> #2) I'd like to move the web application to
> https://subdomain.example.com/sub-uri/ by setting up location block
> like so (spacing condensed for display purposes):
>
>     location /flask-demo {
>         root /var/www/passenger-python-flask-demo;
>         proxy_set_header X-Forwarded-Proto $scheme;
>         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>         proxy_set_header X-Forwarded-Host $host;
>         proxy_pass http://127.0.0.1:3000/;
>     }
>
> The trailing slash was added in an attempt to map
> /flask-demo/SOMETHING to /SOMETHING in the application's point of
> view.

Just as a small point -- it is usually worth making sure that the numbers
of / characters at the end of the "proxy_pass" argument and at the end
of the "location" prefix value, are the same.

> That works well for a test web app where everything is contained in
> a single file, but when the static assets are referenced by the HTML
> output the user's browser attempts to pull static content from '/'
> instead of '/flask-demo/'.

I would suggest that the HTML output of the web app is wrong.

It should never link to /static/styles.css, if it wants to be
reverse-proxied.

It should link to static/styles.css or to ../static/styles.css or to
../../static/styles.css or to whatever value is appropriate relative to
the current resource.

If the web app did that, then reverse-proxying would probably Just Work.

"Not wanting to be reverse-proxied" is a valid choice for a web app. And
you can choose not to use ones that make that choice.

> I'm hoping there is a way to isolate each web application based on
> the initial location block match, thereby catching follow-up
> requests for static resources related to the first request and
> prepend the sub-URI.

In http, every request is independent of every other request, unless
you do something to try to tie them together (such as with a Cookie).

Stock nginx will receive a request for /static/styles.css, and will
handle it according to its configuration.

If you want your nginx to receive a request for /static/styles.css, and
to do some processing based on the Referer: header or based on a Cookie:
header or something else, in order that the final processed request will
be /flask-demo/static/styles.css, then you will probably have to write
some code in one of the embedded languages in nginx.conf.

I'm not aware of code like that existing.

> I assume the answer is "no, this is not supported", but I wanted to

If you write the code, you can make it work (unreliably) for you.

It cannot work in stock nginx just using nginx.conf normal directives.

> ask just to make sure I'm not overlooking something. Francis Daly's
> remarks on the "URL-Rewriting not working" thread that I've quoted
> from below seems to suggest it might be possible, but probably isn't
> worth the headache:

Your outline above is that nginx should not do the translation in the
html sent to the client, but that nginx should interpret the following
requests based on something not directly specified in the request line.

That is possibly harder than doing the translation in the html.

> > That is, if you want the back-end / to correspond to
> > the front-end /x/, then if the back-end ever links to
> > something like /a, you will need that to become
> > translated to /x/a before it leaves the front-end. In
> > general, the front-end cannot do that translation.

You can try to get the front-end to do that translation; it will also
be unreliable and probably inefficient.

Basically, any text that the client might interpret as a url that starts
with http:// or https:// or / would potentially have to be modified. And
possibly some other text as well, depending on the exact contents.

> > So you may find it easier to configure the back-end to
> > be (or to act as if it is) installed below /x/ directly.

That is still true.

If you won't do that, and if you have two web apps that each want to
be at /, you may be better off having two server{} blocks in nginx with
different server_name values.

> I found the 'proxy_redirect' directive, but it doesn't appear to do
> what I'm looking for.

proxy_redirect deals with http headers, not with the body content.

> I've used nginx for years, but only in very basic configurations.
> This is something new to me and I'm struggling to wrap my head
> around it. Thank you for reading this and any advice you can offer.

It's a http and url thing, rather than an nginx thing.

Good luck with it,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

rnburn
Francis Daly Wrote:
-------------------------------------------------------
> It cannot work in stock nginx just using nginx.conf normal directives.

You can with (many) proper rewrites but this does require a very close eye
on the logfiles until you have rewritten all the 40x's, and for the sake of
performance eventually have them converted to a sequence of maps.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,274104,274109#msg-274109

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

Francis Daly
On Tue, May 09, 2017 at 09:03:13AM -0400, itpp2012 wrote:
> Francis Daly Wrote:

Hi there,

> > It cannot work in stock nginx just using nginx.conf normal directives.
>
> You can with (many) proper rewrites but this does require a very close eye
> on the logfiles until you have rewritten all the 40x's, and for the sake of
> performance eventually have them converted to a sequence of maps.

From the original mail:

"""
If it was only one web application I could setup location
blocks for specific patterns, but if I plan on running multiple web
applications on a single FQDN (perhaps even different instances of the
same web app), each in a separate sub-URI, those web applications
might all make requests to '/static/styles.css' based on their
original configuration.
"""

nginx gets a request for /static/styles.css.

I'm not aware of a rewrite which will let you decide which of web-app-1
and web-app-2 is the correct one to proxy_pass this request to this
time. Both web apps use the same url.

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

rnburn
Francis Daly Wrote:
-------------------------------------------------------
> nginx gets a request for /static/styles.css.
>
> I'm not aware of a rewrite which will let you decide which of
> web-app-1
> and web-app-2 is the correct one to proxy_pass this request to this
> time. Both web apps use the same url.

There is a distinction between web apps, the request url, which you use as a
map variable.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,274104,274116#msg-274116

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

deoren
In reply to this post by Francis Daly
Michael, itpp2012, Francis,

My apologies for the terribly short reply, but I wanted to go ahead and
reply back and thank you for the detailed responses. I've looked over
them briefly and plan to go back over them in detail soon.

The takeaway appears to be that the best results come from a web
application that is both aware of being called from a sub-URI and
provides sufficient "hooks" to easily keep generated requests within the
desired sub-URI.

The tip re using the request url as a map variable is an interesting
one. I've used the map directive before, but it is a weak spot for me.
I'll do additional research in that direction.

Many thanks again to all of you for taking the time to respond to my
questions.
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Can you migrate a web app available via '/' to a proxied sub-URI without modifying the web app?

Peter Booth
In reply to this post by Francis Daly
There's "can you?" and there's "should you?" My attitude is that life is short, so I want to avoid building any opportunities to break.

Imagine that you deploy your N web apps. There can be a real value in being able to access the web app directly when debugging, and avoiding the web server layer. (for example, if your web server is also a caching reverse proxy) That means that your web app emits relative links that are valid in the context of the app. Best of all is if they also work on web server without URL rewriting.

What you're asking for is unreliably possible, but a bad idea. Imagine that you want to deploy a dev, QA, uat and demo version of the site - what urls would theee be?

Peter

Sent from my iPhone

> On May 9, 2017, at 8:37 AM, Francis Daly <[hidden email]> wrote:
>
> On Mon, May 08, 2017 at 06:17:02PM -0500, deoren wrote:
>
> Hi there,
>
>> I'm still
>> not completely clear on whether this is supported.
>
> nginx does the nginx side as much as it can. Whether it all works is
> almost entirely down to the web app.
>
> You cannot reliably reverse-proxy an arbitrary web app to a different
> place in the url hierarchy than that at which it believes it is installed.
>
> You can unreliably do it, if that is good enough for you.
>
>> #2) I'd like to move the web application to
>> https://subdomain.example.com/sub-uri/ by setting up location block
>> like so (spacing condensed for display purposes):
>>
>>    location /flask-demo {
>>        root /var/www/passenger-python-flask-demo;
>>        proxy_set_header X-Forwarded-Proto $scheme;
>>        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>>        proxy_set_header X-Forwarded-Host $host;
>>        proxy_pass http://127.0.0.1:3000/;
>>    }
>>
>> The trailing slash was added in an attempt to map
>> /flask-demo/SOMETHING to /SOMETHING in the application's point of
>> view.
>
> Just as a small point -- it is usually worth making sure that the numbers
> of / characters at the end of the "proxy_pass" argument and at the end
> of the "location" prefix value, are the same.
>
>> That works well for a test web app where everything is contained in
>> a single file, but when the static assets are referenced by the HTML
>> output the user's browser attempts to pull static content from '/'
>> instead of '/flask-demo/'.
>
> I would suggest that the HTML output of the web app is wrong.
>
> It should never link to /static/styles.css, if it wants to be
> reverse-proxied.
>
> It should link to static/styles.css or to ../static/styles.css or to
> ../../static/styles.css or to whatever value is appropriate relative to
> the current resource.
>
> If the web app did that, then reverse-proxying would probably Just Work.
>
> "Not wanting to be reverse-proxied" is a valid choice for a web app. And
> you can choose not to use ones that make that choice.
>
>> I'm hoping there is a way to isolate each web application based on
>> the initial location block match, thereby catching follow-up
>> requests for static resources related to the first request and
>> prepend the sub-URI.
>
> In http, every request is independent of every other request, unless
> you do something to try to tie them together (such as with a Cookie).
>
> Stock nginx will receive a request for /static/styles.css, and will
> handle it according to its configuration.
>
> If you want your nginx to receive a request for /static/styles.css, and
> to do some processing based on the Referer: header or based on a Cookie:
> header or something else, in order that the final processed request will
> be /flask-demo/static/styles.css, then you will probably have to write
> some code in one of the embedded languages in nginx.conf.
>
> I'm not aware of code like that existing.
>
>> I assume the answer is "no, this is not supported", but I wanted to
>
> If you write the code, you can make it work (unreliably) for you.
>
> It cannot work in stock nginx just using nginx.conf normal directives.
>
>> ask just to make sure I'm not overlooking something. Francis Daly's
>> remarks on the "URL-Rewriting not working" thread that I've quoted
>> from below seems to suggest it might be possible, but probably isn't
>> worth the headache:
>
> Your outline above is that nginx should not do the translation in the
> html sent to the client, but that nginx should interpret the following
> requests based on something not directly specified in the request line.
>
> That is possibly harder than doing the translation in the html.
>
>>> That is, if you want the back-end / to correspond to
>>> the front-end /x/, then if the back-end ever links to
>>> something like /a, you will need that to become
>>> translated to /x/a before it leaves the front-end. In
>>> general, the front-end cannot do that translation.
>
> You can try to get the front-end to do that translation; it will also
> be unreliable and probably inefficient.
>
> Basically, any text that the client might interpret as a url that starts
> with http:// or https:// or / would potentially have to be modified. And
> possibly some other text as well, depending on the exact contents.
>
>>> So you may find it easier to configure the back-end to
>>> be (or to act as if it is) installed below /x/ directly.
>
> That is still true.
>
> If you won't do that, and if you have two web apps that each want to
> be at /, you may be better off having two server{} blocks in nginx with
> different server_name values.
>
>> I found the 'proxy_redirect' directive, but it doesn't appear to do
>> what I'm looking for.
>
> proxy_redirect deals with http headers, not with the body content.
>
>> I've used nginx for years, but only in very basic configurations.
>> This is something new to me and I'm struggling to wrap my head
>> around it. Thank you for reading this and any advice you can offer.
>
> It's a http and url thing, rather than an nginx thing.
>
> Good luck with it,
>
>    f
> --
> Francis Daly        [hidden email]
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx