nginx cache issue (http upstream cache: -5)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

nginx cache issue (http upstream cache: -5)

orsolya.magos
I am new to nginx caching but have worked with nginx a lot. I tried enabling
caching feature in our repository but it never worked so I thought I will
pull a fresh copy of nginx and turn it on. I ended with the same issue. For
some reason, nginx is not able to create the cache file in the cache dir.  I
have already turned on proxy buffering and set full rw permission for all
users on the cache dir. I also gdb'ed the code and it seems like it gets
into ngx_open_and_stat_file from ngx_open_cached_file (
http://lxr.nginx.org/source/src/core/ngx_open_file_cache.c?v=nginx-1.12.0#0144
) and it tries to open an non existent file in RDONLY mode if the of->log is
not set (
http://lxr.nginx.org/source/src/core/ngx_open_file_cache.c?v=nginx-1.12.0#0869
)

Any help here would be awesome.  I have also pasted my relevant config and
debug log below.

Here's my configuration.. (only the relevant part)

http {

    proxy_cache_path  /tmp/nginx/cache  levels=1:2    keys_zone=STATIC:10m
    inactive=24h  max_size=1g;

    server {
        listen       80;
        server_name  localhost;
        proxy_buffering on;

        location / {
            proxy_pass http://52.216.66.40$request_uri;
            proxy_set_header Host images.clipartpanda.com;
            proxy_buffering on;
            proxy_cache            STATIC;
            add_header X-Proxy-Cache $upstream_cache_status;
            root   html;
            index  index.html index.htm;
        }
    }

And here's the debug log showing whats that nginx is failing to create the
cache file.
2018/02/05 08:57:26 [debug] 22509#0: *7 http cache key:
"http://52.216.66.40"
2018/02/05 08:57:26 [debug] 22509#0: *7 http cache key:
"/teddy-clip-art-teddy-md.png"
2018/02/05 08:57:26 [debug] 22509#0: *7 add cleanup: 00000000006D3560
2018/02/05 08:57:26 [debug] 22509#0: shmtx lock
2018/02/05 08:57:26 [debug] 22509#0: slab alloc: 120 slot: 4
2018/02/05 08:57:26 [debug] 22509#0: slab alloc: 00007FD2C31AA080
2018/02/05 08:57:26 [debug] 22509#0: shmtx unlock
2018/02/05 08:57:26 [debug] 22509#0: *7 http file cache exists: -5 e:0
2018/02/05 08:57:26 [debug] 22509#0: *7 cache file:
"/tmp/nginx/cache/3/4f/ecbc29d001e852b40f09e913b5ced4f3"
2018/02/05 08:57:26 [debug] 22509#0: *7 add cleanup: 00000000006D35B0
2018/02/05 08:57:26 [debug] 22509#0: *7 http upstream cache: -5

And for now i have set "777" perm to the nginx cache dir.

>> ls -lrt /tmp/nginx/                                                      
                                       
drwxrwxrwx 2 test root 4096 Feb  5 08:20 cache

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,278346,278346#msg-278346

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: nginx cache issue (http upstream cache: -5)

Pratyush Kumar
What's the permission for /tmp/nginx ?

On 05-Feb-2018 14:38, loopback_proxy <[hidden email]> wrote:

I am new to nginx caching but have worked with nginx a lot. I tried enabling
caching feature in our repository but it never worked so I thought I will
pull a fresh copy of nginx and turn it on. I ended with the same issue. For
some reason, nginx is not able to create the cache file in the cache dir.  I
have already turned on proxy buffering and set full rw permission for all
users on the cache dir. I also gdb'ed the code and it seems like it gets
into ngx_open_and_stat_file from ngx_open_cached_file (
http://lxr.nginx.org/source/src/core/ngx_open_file_cache.c?v=nginx-1.12.0#0144
) and it tries to open an non existent file in RDONLY mode if the of->log is
not set (
http://lxr.nginx.org/source/src/core/ngx_open_file_cache.c?v=nginx-1.12.0#0869
)

Any help here would be awesome.  I have also pasted my relevant config and
debug log below.

Here's my configuration.. (only the relevant part)

http {

    proxy_cache_path  /tmp/nginx/cache  levels=1:2    keys_zone=STATIC:10m
    inactive=24h  max_size=1g;

    server {
        listen       80;
        server_name  localhost;
        proxy_buffering on;

        location / {
    proxy_pass http://52.216.66.40$request_uri;
            proxy_set_header Host images.clipartpanda.com;
    proxy_buffering on;
            proxy_cache            STATIC;
    add_header X-Proxy-Cache $upstream_cache_status;
            root   html;
            index  index.html index.htm;
        }
    }

And here's the debug log showing whats that nginx is failing to create the
cache file.
2018/02/05 08:57:26 [debug] 22509#0: *7 http cache key:
"http://52.216.66.40"
2018/02/05 08:57:26 [debug] 22509#0: *7 http cache key:
"/teddy-clip-art-teddy-md.png"
2018/02/05 08:57:26 [debug] 22509#0: *7 add cleanup: 00000000006D3560
2018/02/05 08:57:26 [debug] 22509#0: shmtx lock
2018/02/05 08:57:26 [debug] 22509#0: slab alloc: 120 slot: 4
2018/02/05 08:57:26 [debug] 22509#0: slab alloc: 00007FD2C31AA080
2018/02/05 08:57:26 [debug] 22509#0: shmtx unlock
2018/02/05 08:57:26 [debug] 22509#0: *7 http file cache exists: -5 e:0
2018/02/05 08:57:26 [debug] 22509#0: *7 cache file:
"/tmp/nginx/cache/3/4f/ecbc29d001e852b40f09e913b5ced4f3"
2018/02/05 08:57:26 [debug] 22509#0: *7 add cleanup: 00000000006D35B0
2018/02/05 08:57:26 [debug] 22509#0: *7 http upstream cache: -5

And for now i have set "777" perm to the nginx cache dir.

>> ls -lrt /tmp/nginx/                                                     
                                      
drwxrwxrwx 2 test root 4096 Feb  5 08:20 cache

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,278346,278346#msg-278346

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx



_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: nginx cache issue (http upstream cache: -5)

Maxim Dounin
In reply to this post by orsolya.magos
Hello!

On Mon, Feb 05, 2018 at 04:08:33AM -0500, loopback_proxy wrote:

> I am new to nginx caching but have worked with nginx a lot. I tried enabling
> caching feature in our repository but it never worked so I thought I will
> pull a fresh copy of nginx and turn it on. I ended with the same issue. For
> some reason, nginx is not able to create the cache file in the cache dir.  I
> have already turned on proxy buffering and set full rw permission for all
> users on the cache dir. I also gdb'ed the code and it seems like it gets
> into ngx_open_and_stat_file from ngx_open_cached_file (
> http://lxr.nginx.org/source/src/core/ngx_open_file_cache.c?v=nginx-1.12.0#0144
> ) and it tries to open an non existent file in RDONLY mode if the of->log is
> not set (
> http://lxr.nginx.org/source/src/core/ngx_open_file_cache.c?v=nginx-1.12.0#0869
> )

The code in question is not related to your problem.

Instead, you should check what your backend returns.  There are
number of cases when nginx won't cache a response, see description
of the proxy_cache_valid directive for the details:

http://nginx.org/r/proxy_cache_valid

[...]

> And here's the debug log showing whats that nginx is failing to create the
> cache file.
> 2018/02/05 08:57:26 [debug] 22509#0: *7 http cache key:
> "http://52.216.66.40"
> 2018/02/05 08:57:26 [debug] 22509#0: *7 http cache key:
> "/teddy-clip-art-teddy-md.png"
> 2018/02/05 08:57:26 [debug] 22509#0: *7 add cleanup: 00000000006D3560
> 2018/02/05 08:57:26 [debug] 22509#0: shmtx lock
> 2018/02/05 08:57:26 [debug] 22509#0: slab alloc: 120 slot: 4
> 2018/02/05 08:57:26 [debug] 22509#0: slab alloc: 00007FD2C31AA080
> 2018/02/05 08:57:26 [debug] 22509#0: shmtx unlock
> 2018/02/05 08:57:26 [debug] 22509#0: *7 http file cache exists: -5 e:0
> 2018/02/05 08:57:26 [debug] 22509#0: *7 cache file:
> "/tmp/nginx/cache/3/4f/ecbc29d001e852b40f09e913b5ced4f3"
> 2018/02/05 08:57:26 [debug] 22509#0: *7 add cleanup: 00000000006D35B0
> 2018/02/05 08:57:26 [debug] 22509#0: *7 http upstream cache: -5

Your assumption that it is "failing to create" is wrong.  Instead,
this debug log snippet shows that nginx checked if there is a
cached response available, and the result is negative.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: nginx cache issue (http upstream cache: -5)

orsolya.magos
Ahhh interesting, that did the trick. Thank you so much.

I have been also trying to understand the internals of nginx caching and how
it works. I read the nginx blog about the overall architecture and the nginx
man page about proxy_cache_* directives. I am looking for the internal
architecture of the how the caching subsystem works. If you guys have any
documentation or article about it, that would be so useful.

- Karthik

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,278346,278370#msg-278370

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx