UDP reverse proxying for OpenVPN isn't working using Nginx streams

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

UDP reverse proxying for OpenVPN isn't working using Nginx streams

rnburn
Hi.

I was just wondering whether UDP stream proxying on Nginx is in its infacy
or there is something which I am doing wrong. I have this simple config:

events { worker_connections  1024; }

worker_processes  1;
error_log /dev/stderr debug;
daemon off;

stream {
server {
    listen X.X.X.X:1194 udp;
    proxy_pass 127.0.0.1:1195;
}
}

to make Nginx a reverse proxy for my OpenVPN server listening on UDP port
1195 on localhost. But it just doesn't work. When a client connects, Nginx
keeps logging these lines on stderr:

2017/04/26 12:14:43 [notice] 17125#0: using the "epoll" event method
2017/04/26 12:14:43 [notice] 17125#0: nginx/1.11.13
2017/04/26 12:14:43 [notice] 17125#0: built by gcc 4.9.2 (Debian 4.9.2-10)
2017/04/26 12:14:43 [notice] 17125#0: OS: Linux 3.16.0-4-amd64
2017/04/26 12:14:43 [notice] 17125#0: getrlimit(RLIMIT_NOFILE): 1024:4096
2017/04/26 12:14:43 [notice] 17125#0: start worker processes
2017/04/26 12:14:43 [notice] 17125#0: start worker process 17126
2017/04/26 12:14:47 [info] 17126#0: *1 udp client Y.Y.Y.Y:40332 connected to
X.X.X.X:1194
2017/04/26 12:14:47 [info] 17126#0: *1 udp proxy 127.0.0.1:55424 connected
to 127.0.0.1:1195
2017/04/26 12:14:47 [info] 17126#0: *3 udp client Y.Y.Y.Y:40332 connected to
X.X.X.X:1194
2017/04/26 12:14:47 [info] 17126#0: *3 udp proxy 127.0.0.1:48958 connected
to 127.0.0.1:1195
2017/04/26 12:14:47 [info] 17126#0: *5 udp client Y.Y.Y.Y:40332 connected to
X.X.X.X:1194
2017/04/26 12:14:47 [info] 17126#0: *5 udp proxy 127.0.0.1:56732 connected
to 127.0.0.1:1195
2017/04/26 12:14:47 [info] 17126#0: *7 udp client Y.Y.Y.Y:40332 connected to
X.X.X.X:1194
2017/04/26 12:14:47 [info] 17126#0: *7 udp proxy 127.0.0.1:60363 connected
to 127.0.0.1:1195
2017/04/26 12:14:50 [info] 17126#0: *9 udp client Y.Y.Y.Y:56226 connected to
X.X.X.X:1194
2017/04/26 12:14:50 [info] 17126#0: *9 udp proxy 127.0.0.1:52499 connected
to 127.0.0.1:1195
2017/04/26 12:14:50 [info] 17126#0: *11 udp client Y.Y.Y.Y:56226 connected
to X.X.X.X:1194
2017/04/26 12:14:50 [info] 17126#0: *11 udp proxy 127.0.0.1:48850 connected
to 127.0.0.1:1195
2017/04/26 12:14:50 [info] 17126#0: *13 udp client Y.Y.Y.Y:56226 connected
to X.X.X.X:1194
2017/04/26 12:14:50 [info] 17126#0: *13 udp proxy 127.0.0.1:60125 connected
to 127.0.0.1:1195
2017/04/26 12:14:50 [info] 17126#0: *15 udp client Y.Y.Y.Y:56226 connected
to X.X.X.X:1194
2017/04/26 12:14:50 [info] 17126#0: *15 udp proxy 127.0.0.1:54133 connected
to 127.0.0.1:1195
2017/04/26 12:14:52 [info] 17126#0: *17 udp client Y.Y.Y.Y:56226 connected
to X.X.X.X:1194
2017/04/26 12:14:52 [info] 17126#0: *17 udp proxy 127.0.0.1:50184 connected
to 127.0.0.1:1195
2017/04/26 12:14:52 [info] 17126#0: *19 udp client Y.Y.Y.Y:56226 connected
to X.X.X.X:1194
2017/04/26 12:14:52 [info] 17126#0: *19 udp proxy 127.0.0.1:48836 connected
to 127.0.0.1:1195
2017/04/26 12:14:53 [info] 17126#0: *21 udp client Y.Y.Y.Y:56226 connected
to X.X.X.X:1194
2017/04/26 12:14:53 [info] 17126#0: *21 udp proxy 127.0.0.1:42665 connected
to 127.0.0.1:1195
2017/04/26 12:14:56 [info] 17126#0: *23 udp client Y.Y.Y.Y:56226 connected
to X.X.X.X:1194
.......................
.......................

Whereas the OpenVPN client is stuck on:

Wed Apr 26 12:14:50 2017 OpenVPN 2.3.4 x86_64-pc-linux-gnu [SSL (OpenSSL)]
[LZO] [EPOLL] [PKCS11] [MH] [IPv6] built on Nov 12 2015
Wed Apr 26 12:14:50 2017 library versions: OpenSSL 1.0.1t  3 May 2016, LZO
2.08
Wed Apr 26 12:14:50 2017 Control Channel Authentication: tls-auth using
INLINE static key file
Wed Apr 26 12:14:50 2017 Outgoing Control Channel Authentication: Using 160
bit message hash 'SHA1' for HMAC authentication
Wed Apr 26 12:14:50 2017 Incoming Control Channel Authentication: Using 160
bit message hash 'SHA1' for HMAC authentication
Wed Apr 26 12:14:50 2017 Socket Buffers: R=[212992->212992]
S=[212992->212992]
Wed Apr 26 12:14:50 2017 UDPv4 link local: [undef]
Wed Apr 26 12:14:50 2017 UDPv4 link remote: [AF_INET]X.X.X.X:1194
Wed Apr 26 12:14:50 2017 TLS: Initial packet from [AF_INET]X.X.X.X:1194,
sid=afcea479 758711e0

Even there trivial setups work as expected:

pen X.X.X.X:1194 127.0.0.1:1195  -U

OR

nc -u -l -p 1194 -c "nc -u 127.0.0.1 1195"

But I fail to understand why isn't Nginx working. By the way, if everything
is replaced with TCP in both nginx and OpenVPN file, it works. Also UDP
proxying for DNS:

listen X.X.X.X:53 udp;
proxy_pass 8.8.8.8:53;

works. The Nginx version is: 1.11.13. Will really appreciate any advice on
this.

Thanks & Regards.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,273875,273875#msg-273875

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: UDP reverse proxying for OpenVPN isn't working using Nginx streams

Roman Arutyunyan
Hi,

On Wed, Apr 26, 2017 at 08:32:08AM -0400, akb-nginx wrote:

> Hi.
>
> I was just wondering whether UDP stream proxying on Nginx is in its infacy
> or there is something which I am doing wrong. I have this simple config:
>
> events { worker_connections  1024; }
>
> worker_processes  1;
> error_log /dev/stderr debug;
> daemon off;
>
> stream {
> server {
>     listen X.X.X.X:1194 udp;
>     proxy_pass 127.0.0.1:1195;
> }
> }
>
> to make Nginx a reverse proxy for my OpenVPN server listening on UDP port
> 1195 on localhost. But it just doesn't work. When a client connects, Nginx
> keeps logging these lines on stderr:
>
> 2017/04/26 12:14:43 [notice] 17125#0: using the "epoll" event method
> 2017/04/26 12:14:43 [notice] 17125#0: nginx/1.11.13
> 2017/04/26 12:14:43 [notice] 17125#0: built by gcc 4.9.2 (Debian 4.9.2-10)
> 2017/04/26 12:14:43 [notice] 17125#0: OS: Linux 3.16.0-4-amd64
> 2017/04/26 12:14:43 [notice] 17125#0: getrlimit(RLIMIT_NOFILE): 1024:4096
> 2017/04/26 12:14:43 [notice] 17125#0: start worker processes
> 2017/04/26 12:14:43 [notice] 17125#0: start worker process 17126
> 2017/04/26 12:14:47 [info] 17126#0: *1 udp client Y.Y.Y.Y:40332 connected to
> X.X.X.X:1194
> 2017/04/26 12:14:47 [info] 17126#0: *1 udp proxy 127.0.0.1:55424 connected
> to 127.0.0.1:1195
> 2017/04/26 12:14:47 [info] 17126#0: *3 udp client Y.Y.Y.Y:40332 connected to
> X.X.X.X:1194
> 2017/04/26 12:14:47 [info] 17126#0: *3 udp proxy 127.0.0.1:48958 connected
> to 127.0.0.1:1195
> 2017/04/26 12:14:47 [info] 17126#0: *5 udp client Y.Y.Y.Y:40332 connected to
> X.X.X.X:1194
> 2017/04/26 12:14:47 [info] 17126#0: *5 udp proxy 127.0.0.1:56732 connected
> to 127.0.0.1:1195

Stream UDP proxy creates a session for every client packet.
That packet is proxied separately from other client packets with a new
proxy client port each time and a response for this packet is proxied back.

While this works fine for protocols like DNS, long sessions with multiple
client packets will not work properly.

[..]

--
Roman Arutyunyan
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: UDP reverse proxying for OpenVPN isn't working using Nginx streams

rnburn
Thanks for your responce. I think I am out of luck then as far as proxying
UDP openvpn is concerned. Any particular reason that Nginx took this
approach instead of how very basic load balancers like "Pen" etc do it.

I was able to proxy using simpler tools like pen and nc but a more loaded
Nginx fails for the reasons you mentioned.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,273875,273878#msg-273878

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: UDP reverse proxying for OpenVPN isn't working using Nginx streams

Roman Arutyunyan
On Wed, Apr 26, 2017 at 09:42:22AM -0400, akb-nginx wrote:
> Thanks for your responce. I think I am out of luck then as far as proxying
> UDP openvpn is concerned. Any particular reason that Nginx took this
> approach instead of how very basic load balancers like "Pen" etc do it.
>
> I was able to proxy using simpler tools like pen and nc but a more loaded
> Nginx fails for the reasons you mentioned.

When operating in multi-worker mode, a UDP packet from a client may be received
by any worker.  Using SO_REUSEPORT can help with that on Linux, but not always.
Proxying becomes complicated if every packet of a UDP packet sequence is
received by a different worker.

--
Roman Arutyunyan
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx