nginx removes strong etags on gzip compression

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

nginx removes strong etags on gzip compression

satay
Hi, I'm using nginx as a reverse proxy and found a behavior that I wouldn't
expect.

According to some references below, I assume nginx would downgrade strong
etags to weak ones when it modifies response content (e.g. gzip
compression). But nginx removes strong etags on gzip compression instead of
a downgrade.

- http://nginx.org/en/CHANGES (See "Changes with nginx 1.7.3  08 Jul 2014")
  - > *) Feature: weak entity tags are now preserved on response
modifications, and strong ones are changed to weak.
-
https://github.com/nginx/nginx/commit/def16742a1ec22ece8279185eb2b798eb5ffa031
  - > Entity tags: downgrade strong etags to weak ones as needed.

I created a gist to reproduce the behavior with minimum requirements, please
see https://gist.github.com/ohbarye/86f2d5b464f5e88821133c43a9cf4956

So my question is: Is it expected behavior that nginx removes strong etags
on gzip compression?

Regards

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,286645,286645#msg-286645

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: nginx removes strong etags on gzip compression

Francis Daly
On Wed, Jan 01, 2020 at 07:19:48AM -0500, ohbarye wrote:

Hi there,

> Hi, I'm using nginx as a reverse proxy and found a behavior that I wouldn't
> expect.

> So my question is: Is it expected behavior that nginx removes strong etags
> on gzip compression?

No, but: the thing that your upstream sends is not a thing that nginx
recognizes as a strong etag.

The HTTP/1.1 RFC (https://tools.ietf.org/html/rfc7232#section-2.3)
says that the etag header must be of the form

  ETag: "abc"

or

  ETag: W/"abc"

while your example sends something of the form

  ETag: abc

and current nginx recognizes a weak etag when the first two characters
are W/, and a strong etag when the first character is ".

(Arguably: nginx could become more strict, and insist on W/" at the
start and " at the end of a weak etag; and insist on " at the start and
end of a strong etag; but I suspect that that is unnecessary.)

The best fix in your case is probably to change your upstream to send
valid headers.

If that is not doable, then possibly you could patch your nginx to accept
this invalid header; or possibly you could try some other config-based
manipulation to make things work the way that you want. I suspect that
either of those is likely to be more work in the long run than fixing
the upstream.

Cheers,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: nginx removes strong etags on gzip compression

J.R.
In reply to this post by satay
> If that is not doable, then possibly you could patch your nginx to accept
> this invalid header; or possibly you could try some other config-based
> manipulation to make things work the way that you want. I suspect that
> either of those is likely to be more work in the long run than fixing
> the upstream.

Depending on the order of how everything runs, *maybe* it can be captured
with the "headers more" module?

https://github.com/openresty/headers-more-nginx-module

Just a thought... No idea if it would work.
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: nginx removes strong etags on gzip compression

satay
In reply to this post by Francis Daly
To francis

Thank you for your answer.

> the thing that your upstream sends is not a thing that nginx recognizes as
a strong etag.
> The HTTP/1.1 RFC (https://tools.ietf.org/html/rfc7232#section-2.3) says
that the etag header must be of the form

Oh, I wasn't aware of the thing.

> The best fix in your case is probably to change your upstream to send
valid headers.

I tried making my upstream's header comply with the form that nginx
recognize as a strong etag like below, then nginx got working to downgrade
the strong etag to a weak one as I expected.

```diff
    location /strong_etag {
-      add_header Etag d41d8cd98f00b204e9800998ecf8427e;
+      add_header Etag '"d41d8cd98f00b204e9800998ecf8427e"';
      default_type application/json;
      return 200 '{"message": "Hello, this is from upstream!"}';
    }
```

```shell
$ curl http://localhost:80/strong_etag -i -H "Accept-Encoding: gzip"
HTTP/1.1 200 OK
Server: nginx/1.17.6
Date: Thu, 02 Jan 2020 16:49:06 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
Etag: W/"d41d8cd98f00b204e9800998ecf8427e"
Content-Encoding: gzip

�V�M-.NLOU�RP�H����Q(��,V����\�҂⒢��\E�Z5[,%
```

The riddle was resolved.

---

To J.R.

Thanks for introducing
https://github.com/openresty/headers-more-nginx-module
I'll check it.

---

Again, many thanks!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,286645,286650#msg-286650

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx