How do I add text to a response from a remote URL in NGINX?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

How do I add text to a response from a remote URL in NGINX?

zakirenish
Hi,

I have the following server in NGINX and it works fine. But, I am wondering
is it possible to add text to a response from a remote URL where hosts my
before_body.txt and after_body.txt? Is there any way to tackle this? Is it
possible at all?

server {
    listen 80;

    root /storage/path;
    index index.html;

    server_name  test.domain.com;

    location / {
       try_files $uri $uri/ =404;
       add_before_body /src/before_body.txt;
       add_after_body /src/after_body.txt;
       autoindex on;
    }

   location /src/ {
       alias /storage/path/content/;
   }
}

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,289227,289227#msg-289227

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

RE: How do I add text to a response from a remote URL in NGINX?

Reinis Rozitis
> I have the following server in NGINX and it works fine. But, I am wondering is
> it possible to add text to a response from a remote URL where hosts my
> before_body.txt and after_body.txt? Is there any way to tackle this? Is it
> possible at all?

According to documentation (http://nginx.org/en/docs/http/ngx_http_addition_module.html) add_before_body/add_after_body does a subrequest.

So something like that might work (I don't have an nginx instance with particular module compiled so you'll have to test yourself) - eg have a (internal) named location with a proxy_pass to the remote url:


location @beforebody {
    proxy_pass http://externalserver/somefile.txt;
}

location / {
        try_files $uri $uri/ =404;
        add_before_body @beforebody;
}


rr


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: RE: How do I add text to a response from a remote URL in NGINX?

zakirenish
Thanks for the reply. :-)

I have added a location as follows:

location ~/src/(.*) {
        proxy_pass http://externalserver.com;
}

It works when the entire URL is loaded in a browser. However, in the
"autoindex" page, "Object Moved
This document may be found _here_" show up before and after the files list.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,289227,289261#msg-289261

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: How do I add text to a response from a remote URL in NGINX?

Francis Daly
On Tue, Sep 01, 2020 at 01:12:03AM -0400, moyamos wrote:

Hi there,

It looks like you had:

   location /src/ {
       alias /storage/path/content/;
   }

and the url /src/before_body.txt would provide the content of the local
file /storage/path/content/before_body.txt.

Now instead you want the content of the url
http://externalserver.com/before_body.txt?

In that case, just change the "alias" line to "proxy_pass
http://externalserver.com/;", and change nothing else.

> I have added a location as follows:
>
> location ~/src/(.*) {
>         proxy_pass http://externalserver.com;
> }

This should be broadly similar, except it will fetch the url
http://externalserver.com/src/before_body.txt -- maybe that is what
you want?

> It works when the entire URL is loaded in a browser. However, in the
> "autoindex" page, "Object Moved
> This document may be found _here_" show up before and after the files list.

It works for me, providing the content of
http://externalserver.com/src/before_body.txt, before the
autoindex-generated content.

Can you show one complete config that does not do what you want?

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: How do I add text to a response from a remote URL in NGINX?

zakirenish
Francis Daly Wrote:
-------------------------------------------------------
> On Tue, Sep 01, 2020 at 01:12:03AM -0400, moyamos wrote:
>
> Hi there,

Thanks Francis for your reply. :-)

> It looks like you had:
>
>    location /src/ {
>        alias /storage/path/content/;
>    }
> and the url /src/before_body.txt would provide the content of the
> local
> file /storage/path/content/before_body.txt.
>
> Now instead you want the content of the url
> http://externalserver.com/before_body.txt?

Yes, that's right.

[...]

> Can you show one complete config that does not do what you want?

server {
    listen 80;

    root /storage/path;
    index index.html;

    server_name  test.domain.com;

    location / {
      try_files $uri $uri/ =404;
      add_before_body /src/before_body.txt;
      add_after_body /src/after_body.txt;
      autoindex on;
    }

   location /src/ {
#    alias /storage/path/content/;
     proxy_pass http://externalserver.com;
   }
}

The result is the same as my previous location. "Object Moved This document
may be found _here_" is showing up before and after the files list. When I
am clicking on the "_here_" link, the content is loaded correctly. But, on
"https". There is a redirect from "http" to "https" on externalserver.com.

As soon as I am changing to

   location /src/ {
     proxy_pass https://externalserver.com;
   }

following errors are logged:

2020/09/01 16:48:41 [error] 8445#8445: *33789440 peer closed connection in
SSL handshake (104: Connection reset by peer) while SSL handshaking to
upstream, client: YY.YYY.YYY.YY, server: test.domain.com, request: "GET /
HTTP/1.1", subrequest: "/src/after_body.txt", upstream:
"https://XXX.XXX.XXX.XXX:443/src/after_body.txt", host: "test.domain.com"
2020/09/01 16:48:41 [error] 8445#8445: *33789440 peer closed connection in
SSL handshake (104: Connection reset by peer) while sending to client,
client: YY.YYY.YYY.YY, server: test.domain.com, request: "GET / HTTP/1.1",
subrequest: "/src/after_body.txt", upstream:
"https://XXX.XXX.XXX.XXX:443/src/after_body.txt", host: "test.domain.com"

>
> f
> --
> Francis Daly        [hidden email]
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,289227,289266#msg-289266

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

RE: How do I add text to a response from a remote URL in NGINX?

Reinis Rozitis
> > Now instead you want the content of the url
> > http://externalserver.com/before_body.txt?
>
> Yes, that's right.

Can you actually open the file on the external server -  http://externalserver.com/src/before_body.txt and does it have the content you expect (without redirects)?

Note that since you have proxy_pass without the trailing slash nginx will send the request as  '/src/before_body.txt'.

If there is no such /src/ directory on the remote server but only /before_body.txt then you have to add the slash at the end:

location /src/ {
     proxy_pass http://externalserver.com/;
}

rr

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: How do I add text to a response from a remote URL in NGINX?

Francis Daly
In reply to this post by zakirenish
On Tue, Sep 01, 2020 at 08:40:02AM -0400, moyamos wrote:
> Francis Daly Wrote:
> > On Tue, Sep 01, 2020 at 01:12:03AM -0400, moyamos wrote:

Hi there,

>     location / {
>       try_files $uri $uri/ =404;
>       add_before_body /src/before_body.txt;
>       add_after_body /src/after_body.txt;
>       autoindex on;
>     }
>
>    location /src/ {
> #    alias /storage/path/content/;
>      proxy_pass http://externalserver.com;

That will end up with nginx requesting
http://externalserver.com/src/before_body.txt and
http://externalserver.com/src/after_body.txt

> The result is the same as my previous location. "Object Moved This document
> may be found _here_" is showing up before and after the files list. When I
> am clicking on the "_here_" link, the content is loaded correctly. But, on
> "https". There is a redirect from "http" to "https" on externalserver.com.

That would explain the message.

Have you the option to change externalserver.com not to issue that redirect?

(Alternatively, you could save the content into the local files and just
serve those; but I guess you want to use the external content to stay
current when it changes.)

>    location /src/ {
>      proxy_pass https://externalserver.com;
>    }
>
> following errors are logged:
>
> 2020/09/01 16:48:41 [error] 8445#8445: *33789440 peer closed connection in
> SSL handshake (104: Connection reset by peer) while SSL handshaking to
> upstream, client: YY.YYY.YYY.YY, server: test.domain.com, request: "GET /
> HTTP/1.1", subrequest: "/src/after_body.txt", upstream:
> "https://XXX.XXX.XXX.XXX:443/src/after_body.txt", host: "test.domain.com"

That says that the nginx client and the upstream server are not able to
agree a ssl session.

Is "XXX.XXX.XXX.XXX" the same as "externalserver.com"? If
not, can you use "curl" or something similar to fetch
"https://XXX.XXX.XXX.XXX:443/src/after_body.txt" without errors?

Maybe you need some extra ssl config in nginx, such as
http://nginx.org/r/proxy_ssl_server_name and friends?

Good luck with it,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: How do I add text to a response from a remote URL in NGINX?

zakirenish
In reply to this post by Francis Daly
Hi Reinis and Francis,

Thanks for your hands.

I resolved the issue. I've moved before_body.txt and after_body.txt to
another domain without https forced redirect. It works fine with a simple
proxy_pass even over https.

Thanks for all your help.

Best regards

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,289227,289286#msg-289286

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx