Most of the time all is fine and working as expected. There is some
specialty in the deployment setup where some expected spikes in requests
(end clients updating daily data) to few locations occur. Response size
varies 1M-1.5M non-gziped. Log snippet from such spike:
Monitoring du on cache location shows max 1.1G, like:
After 2minutes response 'stabilizes' with correct size (in this example
1526025). Problem is also amplified due clients validate response and retry
progressively if corrupted.
There are no weird log lines in error log or linux (centos) messages, also
there is no cache 'updating', just hits (I guess this omits upstream servers
issue). Is it possible we have issue with reading cached entries from
/dev/shm during peak times?
I would kindly ask for hints where possibly to start looking and debugging?
Big thanks in advance
Re: Strange behavior on proxy cache at high load spike
> After 2minutes response 'stabilizes' with correct size (in this example
> 1526025). Problem is also amplified due clients validate response and retry
> progressively if corrupted.
What is the response your upstream is sending back? If the 'corrupted'
data is still a 200, then nginx will cache that... You need to make
sure it's sending back a 5xx if it's overloaded or whatever error
would be relevant.
You might want to consider expanding your 'use_stale' like: