Full logging

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Full logging

Hans Liss
Hi!

I need a HTTP proxy that can handle requests for a single upstream
server, but also log request headers, raw request body, response headers
and raw response body for each request. Preferably this should be logged
to a separate daily logfile (with date-stamped filename), with
timestamps, but the exact format isn't important.

I understand that nginx doesn't provide full logging like this out of
the box, so I was wondering if it would be difficult to add. Is there a
single well-defined point in the code where an upstream request is ready
to be sent, and a point where the response has been received? I might be
able to take it from there, but it would save a lot of time if I knew
where to do this.

I'm well aware of the performance penalties involved, but for this
particular case, it doesn't matter.

Best regards,

Hans









När du har kontakt med oss på Uppsala universitet med e-post så innebär det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/

E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: http://www.uu.se/en/about-uu/data-protection-policy
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

RE: Full logging

Reinis Rozitis
> I need a HTTP proxy that can handle requests for a single upstream server,
> but also log request headers, raw request body, response headers and raw
> response body for each request. Preferably this should be logged to a
> separate daily logfile (with date-stamped filename), with timestamps, but the
> exact format isn't important.


By default nginx can log the request body ( http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_body ),
for the response body an option is to use the lua module https://github.com/openresty/lua-nginx-module (either compile yourself or use Openresty)  

But it might be way more simple just to plug a proxy (developed for exact purposes) between nginx and upstream. Like for example https://mitmproxy.org/



I have an old test example (not sure if still working) which shows the idea, but maybe there is a more elegant way to do it:

http {

   log_format bodylog '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time "$req_headers" "$req_body" "$resp_body"';


sevrver {

        lua_need_request_body on;

        set $resp_body "";
        set $req_body "";
        set $req_headers "";

        rewrite_by_lua_block {
            local req_headers = "Headers: ";
            ngx.var.req_body = ngx.req.get_body_data();
            local h, err = ngx.req.get_headers()
            for k, v in pairs(h) do
                req_headers = req_headers .. k .. ": " .. v .. "\n";
            end

            ngx.var.req_headers = req_headers;
        }

        body_filter_by_lua '
        local resp_body = string.sub(ngx.arg[1], 1, 1000)
        ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body
        if ngx.arg[2] then
          ngx.var.resp_body = ngx.ctx.buffered
        end
        ';
     }
}



rr

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx