Cache Volume utilized at around 50 % with proxy_cache_min_uses

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Cache Volume utilized at around 50 % with proxy_cache_min_uses

zakirenish
With use of proxy_cache_min_uses volume of cache is getting settled up at
around 50% utilization.
No matter what is the volume allocated in max_size its not filling up
further beyond 50%.
If the proxy_cache_min_uses is removed the cache gets filled up with
max_size allocated volume.

No of files in cache directory is far less beyond the size allocated in key
zone. Its getting capped up near 20 Lakhs whereas allocated key zone could
have accommodate around 80 L files with below configuration  

proxy_cache_path    /cache/contentcache keys_zone=content:1000m levels=1:2
max_size=1000g inactive=7d use_temp_path=off;

proxy_cache_min_uses 2;

Cache volume is utilized with above configuration is around 550 GB which is
not growing beyond and as inactive is set to 7d so this would have been
effective only after 7 days when content should have got deleted if not
accessed within 7 days time period.

Writing all the objects on disk is causing high i/o so using
proxy_cache_min_uses would have been beneficial with utilizing cache
optimally and high cache hit ratio

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,289185,289185#msg-289185

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Cache Volume utilized at around 50 % with proxy_cache_min_uses

Maxim Dounin
Hello!

On Tue, Aug 25, 2020 at 07:41:43AM -0400, anish10dec wrote:

> With use of proxy_cache_min_uses volume of cache is getting settled up at
> around 50% utilization.
> No matter what is the volume allocated in max_size its not filling up
> further beyond 50%.
> If the proxy_cache_min_uses is removed the cache gets filled up with
> max_size allocated volume.
>
> No of files in cache directory is far less beyond the size allocated in key
> zone. Its getting capped up near 20 Lakhs whereas allocated key zone could
> have accommodate around 80 L files with below configuration  

It is important to understand that number of files in the cache
directory is not directly related to the keys zone size when using
proxy_cache_min_uses.  Instead, when using proxy_cache_min_uses,
keys zone needs to keep information about all resources requested,
to correctly trac usage numbers, and this usually much higher than
the number of files saved to disk.  This is what
proxy_cache_min_uses does: it saves disk space and disk bandwidth
by tracking information only in the keys zone.

Given the above, I see two possible reasons why the cache volume
is only filled at 50%:

1. You've run out of keys_zone size.

2. You've run out of resources requested frequent enough to be
cached with proxy_cache_min_uses set to 2.

It should be easy enough to find out what happens in your case.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Cache Volume utilized at around 50 % with proxy_cache_min_uses

zakirenish
> Given the above, I see two possible reasons why the cache volume
> is only filled at 50%:
>
> 1. You've run out of keys_zone size.
>
> 2. You've run out of resources requested frequent enough to be
> cached with proxy_cache_min_uses set to 2.
>
> It should be easy enough to find out what happens in your case.
>

It seems possible reason is keys_zone size. Will look into by increasing the
same and trying different permutations.

As in general 1M stores around 8000 Keys, what could be probable formula for
keys_zone size with proxy_cache_min_uses.

Since it keeps information of all requested resource so it would highly
depend upon number of requested resources.

In my case number of request per sec is around 1000 i.e. 36 Lakhs per hour
during peak hours

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,289185,289189#msg-289189

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx