proxy_cache_path 'inactive' vs http cache-control / expires headers?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

proxy_cache_path 'inactive' vs http cache-control / expires headers?

J.R.
I've been doing some experimenting with nginx's proxy caching and
slowly working the kinks out.

From what I read, the cache-control & expires headers take precedence
over the 'proxy_cache_valid' setting, which is great as certain pages
are valid for several hours at a time.

However, I am noticing still a high amount of cache misses... Upon
further investigation I'm thinking (haven't tested it yet) that the
'proxy_cache_path' inactive setting (currently at its default of 10m)
is taking precedence over the above cache-control settings...

Is there any way to tie the 'inactive' time to the cache-control
header expiration time so that pages that are cached in a certain
time-window are always kept and not deleted until after the header
expiration time?
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

RE: proxy_cache_path 'inactive' vs http cache-control / expires headers?

Reinis Rozitis
> Is there any way to tie the 'inactive' time to the cache-control header
> expiration time so that pages that are cached in a certain time-window are
> always kept and not deleted until after the header expiration time?

You can just set the inactive time longer than your possible maximum expire time for the objects  then the cache manager won't purge the cache files even the object is still valid but not accessed.

rr

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: proxy_cache_path 'inactive' vs http cache-control / expires headers?

J.R.
In reply to this post by J.R.
> You can just set the inactive time longer than your possible maximum
> expire time for the objects  then the cache manager won't purge the
> cache files even the object is still valid but not accessed.

That's what I ended up doing. Thanks for the suggestion though.
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: proxy_cache_path 'inactive' vs http cache-control / expires headers?

Roger Fischer
In reply to this post by Reinis Rozitis

You can just set the inactive time longer than your possible maximum expire time for the objects  then the cache manager won't purge the cache files even the object is still valid but not accessed.

That may only have a small impact.

As far as I understand:
  • NGINX will remove an item only when the cache is full (ie. it needs space for a new item).
  • Items are removed based on the least-recently used (LRU) queue.
  • The least-recently-used (last) item in the LRU queue is unconditionally removed.
  • The second and third last items are removed if they are past the invalid time.

The expiry of an item has no influence on the removal of items. It only affects if the item is delivered from the cache, or revalidated with an upstream request.

Restarting NGINX does have an impact. The LRU queue is not persisted (for performance reasons). On a restart, the LRU queue is based on the order that the cache loader finds the cached files in the file system.

Roger


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: proxy_cache_path 'inactive' vs http cache-control / expires headers?

Maxim Dounin
Hello!

On Fri, Apr 03, 2020 at 08:33:43AM -0700, Roger Fischer wrote:

> > You can just set the inactive time longer than your possible maximum expire time for the objects  then the cache manager won't purge the cache files even the object is still valid but not accessed.
>
> That may only have a small impact.
>
> As far as I understand:
> NGINX will remove an item only when the cache is full (ie. it needs space for a new item).
> Items are removed based on the least-recently used (LRU) queue.
> The least-recently-used (last) item in the LRU queue is unconditionally removed.
> The second and third last items are removed if they are past the invalid time.

Your understanding is wrong.  Cache manager always removes all
items which were not access for the "inactive=" period of time.  
Quoting docs (http://nginx.org/r/proxy_cache_path):

: Cached data that are not accessed during the time specified by the
: inactive parameter get removed from the cache regardless of their
: freshness. By default, inactive is set to 10 minutes.

See ngx_http_file_cache_expire() function for details.

Additionally, cache items can be removed based on the "max_size="
parameter of the "proxy_cache_path" directive, or if there isn't
enough room in the "keys_zone=" shared memory zone.

> The expiry of an item has no influence on the removal of items.
> It only affects if the item is delivered from the cache, or
> revalidated with an upstream request.

That's correct, as long as "expire" is meant to be the time from the
Expires / X-Accel-Expires / Cache-Control / proxy_cache_valid.  In
nginx documentation this is called "caching time".

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: proxy_cache_path 'inactive' vs http cache-control / expires headers?

Roger Fischer
Thanks, Maxim, for correcting my misunderstanding.

With what frequency is the cache manger run?

Roger

> On Apr 3, 2020, at 9:26 AM, Maxim Dounin <[hidden email]> wrote:
>
> Hello!
>
> On Fri, Apr 03, 2020 at 08:33:43AM -0700, Roger Fischer wrote:
>
>>> You can just set the inactive time longer than your possible maximum expire time for the objects  then the cache manager won't purge the cache files even the object is still valid but not accessed.
>>
>> That may only have a small impact.
>>
>> As far as I understand:
>> NGINX will remove an item only when the cache is full (ie. it needs space for a new item).
>> Items are removed based on the least-recently used (LRU) queue.
>> The least-recently-used (last) item in the LRU queue is unconditionally removed.
>> The second and third last items are removed if they are past the invalid time.
>
> Your understanding is wrong.  Cache manager always removes all
> items which were not access for the "inactive=" period of time.  
> Quoting docs (http://nginx.org/r/proxy_cache_path):
>
> : Cached data that are not accessed during the time specified by the
> : inactive parameter get removed from the cache regardless of their
> : freshness. By default, inactive is set to 10 minutes.
>
> See ngx_http_file_cache_expire() function for details.
>
> Additionally, cache items can be removed based on the "max_size="
> parameter of the "proxy_cache_path" directive, or if there isn't
> enough room in the "keys_zone=" shared memory zone.
>
>> The expiry of an item has no influence on the removal of items.
>> It only affects if the item is delivered from the cache, or
>> revalidated with an upstream request.
>
> That's correct, as long as "expire" is meant to be the time from the
> Expires / X-Accel-Expires / Cache-Control / proxy_cache_valid.  In
> nginx documentation this is called "caching time".
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: proxy_cache_path 'inactive' vs http cache-control / expires headers?

Maxim Dounin
Hello!

On Fri, Apr 03, 2020 at 08:25:14PM -0700, Roger Fischer wrote:

> With what frequency is the cache manger run?

Cache manager monitors least recently used cache items, and wakes
up when there are items to be removed, at the expected removal time.  
Additionally, even if there are no inactive items to remove, it
still wakes up every 10 seconds to monitor "max_size=".

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx