Why does nginx strip trailing headers from a proxied backend? How can I prevent it?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Why does nginx strip trailing headers from a proxied backend? How can I prevent it?

Alan Chandler
I have nginx acting as the static file server for a single page web app
I am developing. It acts as a proxy server for the "/api" portion on my
url space.

The backend server is running on a different port on local host and is
nodejs based.. I'm using nginx as an http2 front end and using http 1/1
between nginx and the backend. In the main this is working well.

But I have one problem. I would like to make use of a trailing header.
My outgoing request has the header "TE: trailers", and the response has
a header "Trailers: API-Status" and then after the body it adds (using
nodejs response.addTrailers({'API-Status': 'OK'})).

But nginx is stripping them out.

I can use curl to prove it

curl -b "MBFMVISIT=emailverify; expires=Sun, 07 Jun 2020 13:14:06
GMT;path=/;" -H "Content-Type: application/json" -H "TE: trailers" -X
GET -c cookie.jar -i https://footdev.chandlerfamily.org.uk/api/config/config

goes via nginx and outputs the response (including the initial
'Trailers: API-Status' header, but not the trailing header

HTTP/2 200
server: nginx/1.18.0
date: Mon, 08 Jun 2020 19:39:41 GMT
content-type: application/json
trailer: API-Status
cache-control: no-cache

{"dcid":17,"pointsMap":"[1,2,4,6,8,12,16]","underdogMap":"[0,1,2,4,6,8]","playoffMap":"[1,2,4,6,8]","bonusMap":"[1,2,4,6,8,12,16]","defaultBonus":2,"clientLog":"ALL","clientLogUid":0,"version":"v4.0.0-alpha3","copyrightYear":2020,"cookieName":"MBBall","cookieVisitName":"MBFMVISIT","mainMenuIcon":"menu","status":true}


curl -b "MBFMVISIT=emailverify; expires=Sun, 07 Jun 2020 13:14:06
GMT;path=/;" -H "Content-Type: application/json" -H "TE: trailers" -X
GET -c cookie.jar -i http://localhost:2040/api/config/config

goes directly to the backend. in this curl outputs the initial headers,
the response and then after the response the trailing header
'API-Status: OK'

HTTP/1.1 200 OK
Trailer: API-Status
Content-Type: application/json
Cache-Control: no-cache
Date: Mon, 08 Jun 2020 19:40:14 GMT
Connection: keep-alive
Transfer-Encoding: chunked

{"dcid":17,"pointsMap":"[1,2,4,6,8,12,16]","underdogMap":"[0,1,2,4,6,8]","playoffMap":"[1,2,4,6,8]","bonusMap":"[1,2,4,6,8,12,16]","defaultBonus":2,"clientLog":"ALL","clientLogUid":0,"version":"v4.0.0-alpha3","copyrightYear":2020,"cookieName":"MBBall","cookieVisitName":"MBFMVISIT","mainMenuIcon":"menu","status":true}API-Status:
OK

(The API-Status: OK is bolded by curl along with the pre reply headers)

My nginx config for the proxy is

location /api/ {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_pass <a href="http://localhost:2040;">http://localhost:2040;
proxy_redirect default;
proxy_buffering on;
proxy_cache off;
}

So how do I tell nginx to pass the trailing header?  I have buffering
on, but doesn't seem to have changed anything - it didn't work when I
had it off.

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Why does nginx strip trailing headers from a proxied backend? How can I prevent it?

Maxim Dounin
Hello!

On Mon, Jun 08, 2020 at 08:57:56PM +0100, Alan Chandler wrote:

> I have nginx acting as the static file server for a single page
> web app I am developing. It acts as a proxy server for the
> "/api" portion on my url space.
>
> The backend server is running on a different port on local host
> and is nodejs based.. I'm using nginx as an http2 front end and
> using http 1/1 between nginx
> and the backend. In the main this is working well.
>
> But I have one problem. I would like to make use of a trailing
> header. My outgoing request has the header "TE: trailers", and
> the response has a header
> "Trailers: API-Status" and then after the body it adds (using
> nodejs response.addTrailers({'API-Status': 'OK'})).
>
> But nginx is stripping them out.

[...]

> My nginx config for the proxy is
>
> location /api/ {
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header Host $http_host;
> proxy_set_header X-NginX-Proxy true;
> proxy_http_version 1.1;
> proxy_set_header Connection "";
> proxy_pass <a href="http://localhost:2040;">http://localhost:2040;
> proxy_redirect default;
> proxy_buffering on;
> proxy_cache off;
> }
>
> So how do I tell nginx to pass the trailing header?

Trailers are only supported in gRPC proxying (grpc_pass), where
they are required for gRPC.  Trailers are not supported by
proxy_pass.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Why does nginx strip trailing headers from a proxied backend? How can I prevent it?

Alan Chandler

On 08/06/2020 21:58, Maxim Dounin wrote:

> Hello!
>
> On Mon, Jun 08, 2020 at 08:57:56PM +0100, Alan Chandler wrote:
>
>> I have nginx acting as the static file server for a single page
>> web app I am developing. It acts as a proxy server for the
>> "/api" portion on my url space.
> Trailers are only supported in gRPC proxying (grpc_pass), where
> they are required for gRPC.  Trailers are not supported by
> proxy_pass.
>
That is a shame.

Time to think up an alternative way

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Why does nginx strip trailing headers from a proxied backend? How can I prevent it?

Maxim Dounin
Hello!

On Mon, Jun 08, 2020 at 10:29:23PM +0100, Alan Chandler wrote:

> On 08/06/2020 21:58, Maxim Dounin wrote:
> >
> > On Mon, Jun 08, 2020 at 08:57:56PM +0100, Alan Chandler wrote:
> >
> > > I have nginx acting as the static file server for a single page
> > > web app I am developing. It acts as a proxy server for the
> > > "/api" portion on my url space.
> > Trailers are only supported in gRPC proxying (grpc_pass), where
> > they are required for gRPC.  Trailers are not supported by
> > proxy_pass.
> >
> That is a shame.

You are welcome to work on this if you think this is needed.  This
should be relatively easy given that trailers infrastructure was
added a while ago to support gRPC proxying.

Note though that there are security concerns about HTTP trailers
in general, and these shouldn't be passed by default unless
explicitly enabled in the configuration.

Just in case, an overview of trailers support in browsers can be
found here (TL;DR: Firefox supports Server-Timing in trailers
since 2018, and that's all):

https://stackoverflow.com/questions/13371367/do-any-browsers-support-trailers-sent-in-chunked-encoding-responses

Some lengthy discussion on trailers support happened in the past
here (TL;DR: net effect is that there are no trailers support in
the Fetch Standard now):

https://github.com/whatwg/fetch/issues/34

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Why does nginx strip trailing headers from a proxied backend? How can I prevent it?

Alan Chandler

On 09/06/2020 02:51, Maxim Dounin wrote:

> Hello!
>
> On Mon, Jun 08, 2020 at 10:29:23PM +0100, Alan Chandler wrote:
>
>> On 08/06/2020 21:58, Maxim Dounin wrote:
>>> On Mon, Jun 08, 2020 at 08:57:56PM +0100, Alan Chandler wrote:
>>>
>>>> I have nginx acting as the static file server for a single page
>>>> web app I am developing. It acts as a proxy server for the
>>>> "/api" portion on my url space.
>>> Trailers are only supported in gRPC proxying (grpc_pass), where
>>> they are required for gRPC.  Trailers are not supported by
>>> proxy_pass.
>>>
>> That is a shame.
> You are welcome to work on this if you think this is needed.

HA:  I'm still working on a Single Page Application I started in 2013 to
move a client from Microsoft Access/Sqlserver app to

a web app (still with sqlserver).  Until I do so I cannot retire, and
that client is the only one I have left and I am currently 69.

The work I was trying to use trailers for is a hobby, but with a
deadline of September this year. Trailers was just me exploring

options to support my api handler throwing an error mid stream. I can
think of other options to handle my use case.

Besides it has been almost 40 years (early 1980s)  since I last coded in
C or C++

> Some lengthy discussion on trailers support happened in the past
> here (TL;DR: net effect is that there are no trailers support in
> the Fetch Standard now):
>
> https://github.com/whatwg/fetch/issues/34
>
Having read through that, trailers is definitely not a rabbit hole I
want to go down.

Alan Chandler






_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx