nginx as nonroot - setsockopt not permitted

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

nginx as nonroot - setsockopt not permitted

wld75
Hi,

we use nginx which load-balances toward our snmptrapd. Everything is working
fine if we start nginx with root. We would like to change it so nginx
(workers) would start with nginx user. I couldn't make it work, do you have
any idea what additional thing can I set/check?

nginx -V
nginx version: nginx/1.12.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx
--modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log
--http-client-body-temp-path=/var/lib/nginx/tmp/client_body
--http-proxy-temp-path=/var/lib/nginx/tmp/proxy
--http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi
--http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi
--http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid
--lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx
--with-file-aio --with-ipv6 --with-http_auth_request_module
--with-http_ssl_module --with-http_v2_module --with-http_realip_module
--with-http_addition_module --with-http_xslt_module=dynamic
--with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic
--with-http_sub_module --with-http_dav_module --with-http_flv_module
--with-http_mp4_module --with-http_gunzip_module
--with-http_gzip_static_module --with-http_random_index_module
--with-http_secure_link_module --with-http_degradation_module
--with-http_slice_module --with-http_stub_status_module
--with-http_perl_module=dynamic --with-mail=dynamic --with-mail_ssl_module
--with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module
--with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4 -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic'
--with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-Wl,-E'

uname -a
Linux c-1 4.14.62-1.el7.centos.ncir.1.x86_64 #1 SMP Wed Aug 15 04:24:17 EEST
2018 x86_64 x86_64 x86_64 GNU/Linux


--------------------------------------------------------------------------------------------------

observation 0) with root user (master+workers)
everything works fine, snmptrapd gets the traps

--------------------------------------------------------------------------------------------------

observation 1)
idea: playing with setcap

config:
  * with nginx user (master is root, workers are started with nginx user, so
in /etc/nginx/nginx.conf 'user nginx;' line is included)
root     2703077  0.0  0.0  59028  2280 ?        Ss   11:34   0:00 nginx:
master process /usr/sbin/nginx
nginx    2703078  0.0  0.0  59476  4160 ?        S    11:34   0:00 nginx:
worker process
nginx    2703079  0.0  0.0  59476  4840 ?        S    11:34   0:00 nginx:
worker process
nginx    2703080  0.0  0.0  59476  4840 ?        S    11:34   0:00 nginx:
worker process
... etc.
  * upstream port is 162, snmptrapd is listening there
  * I've tried both capacities:
       setcap cap_net_bind_service=+ep /sbin/nginx
       setcap cap_net_admin+ep /sbin/nginx
  * /etc/nginx/conf.d/stream/snmptrap.conf
     upstream snmptrap_upstream {
        #server x.y.z.226:162; #commented out for easier testing
        #server x.y.z:227162; #commented out for easier testing
        server x.y.z.228:162;
    }
    server {
        listen z.y.z.225:162 udp;
        proxy_pass snmptrap_upstream;
        proxy_timeout 1s;
        proxy_responses 0;
        proxy_bind $remote_addr transparent;
        error_log /var/log/nginx/snmptrap.log;
    }
  * also tried out switching off iptables



netstat -ulpn | grep 162
udp        0      0 x.y.z.228:162         0.0.0.0:*                        
 2748327/snmptrapd  
udp        0      0 x.y.z.225:162         0.0.0.0:*                        
 2743096/nginx: mast


/var/log/nginx/snmptrap.log:
2018/09/12 11:55:04 [alert] 2739785#0: *23 setsockopt(IP_TRANSPARENT) failed
(1: Operation not permitted) while connecting to upstream, udp client:
x.y.z.225, server: x.y.z.225:162, upstream: "x.y.z.228:162", bytes from/to
client:5/0, bytes from/to upstream:0/0

/var/log/nginx/stream.log: error 500 is coming
2018-09-12T11:55:04+03:00 x.y.z.225 UDP 500 0 5 0.000 "0" "0" "0.000"

--------------------------------------------------------------------------------------------------
observation 2)
idea: trying an other upstream port (>1024), but still the same:

config:
  * with nginx user (master is root, workers are started with nginx user, so
in /etc/nginx/nginx.conf 'user nginx;' line is included)
  * upstream port is 4162
  * /etc/nginx/conf.d/stream/snmptrap.conf
     upstream snmptrap_upstream {
        #server x.y.z.226:162; #commentedout for easier testing
        #server x.y.z:227162; #commented out for easier testing
        server x.y.z.228:4162;
    }
    server {
        listen z.y.z.225:162 udp;
        proxy_pass snmptrap_upstream;
        proxy_timeout 1s;
        proxy_responses 0;
        proxy_bind $remote_addr transparent;
        error_log /var/log/nginx/snmptrap.log;
    }
  * also tried out switching off iptables


netstat -ulpn | grep 162
udp        0      0 x.y.z.228:4162         0.0.0.0:*                        
  2748327/snmptrapd  
udp        0      0 x.y.z.225:162         0.0.0.0:*                        
 2743096/nginx: mast


/var/log/nginx/snmptrap.log:
2018/09/12 11:08:03 [alert] 121472#0: *112642 setsockopt(IP_TRANSPARENT)
failed (1: Operation not permitted) while connecting to upstream, udp
client: x.y.z.225, server: x.y.z.225:162, upstream: "x.y.z.228:4162", bytes
from/to client:5/0, bytes from/to upstream:0/0

/var/log/nginx/stream.log: error 500 is coming
2018-09-12T11:08:03+03:00 x.y.z.225 UDP 500 0 5 0.000 "0" "0" "0.000"


Thanks in advance:
Orsi

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,281226,281226#msg-281226

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: nginx as nonroot - setsockopt not permitted

Maxim Dounin
Hello!

On Fri, Sep 14, 2018 at 03:52:03AM -0400, orsolya.magos wrote:

> we use nginx which load-balances toward our snmptrapd. Everything is working
> fine if we start nginx with root. We would like to change it so nginx
> (workers) would start with nginx user. I couldn't make it work, do you have
> any idea what additional thing can I set/check?
>
> nginx -V
> nginx version: nginx/1.12.2
> built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)

Update to nginx 1.13.8+, it should be able to use transparent
proxying on Linux without workers being run as root:

    *) Feature: now nginx automatically preserves the CAP_NET_RAW capability
       in worker processes when using the "transparent" parameter of the
       "proxy_bind", "fastcgi_bind", "memcached_bind", "scgi_bind", and
       "uwsgi_bind" directives.

Alternatively, consider not using "proxy_bind ... transparent".  
See docs here for additional details:

http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_bind

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: nginx as nonroot - setsockopt not permitted

wld75
Wow great, thanks Maxim for the super fast answer!
We are using epel version, still investigating the possibilities of version
change.
Br,
Orsi

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,281226,281230#msg-281230

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx