Load Balancing TCP directive mail {}

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Load Balancing TCP directive mail {}

vergil
Hi folks,

I'm trying to do a round-robin load balancing for outgoing connections from
my MTA servers. At first I tried it as follows:

[root@proxy-lb02 email]# pwd
/etc/nginx/email
[root@proxy-lb02 email]# cat balanceador.conf
stream {
        upstream stream_backend_mail {
                least_conn;
                server mta-01.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                server mta-02.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                server mta-03.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                server mta-04.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                server mta-05.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                }


        server {
        listen     0.0.0.0:25;
        proxy_pass stream_backend_mail;
        }
}

[root@proxy-lb02 nginx]# pwd
/etc/nginx
[root@proxy-lb02 nginx]# cat nginx.conf

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request"
'
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;




}

mail {

        include /etc/nginx/email/*.conf;

}


[root@proxy-lb02 nginx]# systemctl restart nginx
Job for nginx.service failed because the control process exited with error
code. See "systemctl status nginx.service" and "journalctl -xe" for
details.

[root@proxy-lb02 nginx]# systemctl status nginx -l
● nginx.service - nginx - high performance web server
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor
preset: disabled)
   Active: failed (Result: exit-code) since Sáb 2020-06-06 18:18:19 EDT; 58s
ago
     Docs: http://nginx.org/en/docs/
  Process: 51695 ExecStop=/bin/kill -s TERM $MAINPID (code=exited,
status=0/SUCCESS)
  Process: 52777 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf
(code=exited, status=1/FAILURE)
 Main PID: 51686 (code=exited, status=0/SUCCESS)

Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: Starting nginx -
high performance web server...
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br nginx[52777]: nginx: [emerg]
"stream" directive is not allowed here in
/etc/nginx/email/balanceador.conf:1
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: nginx.service:
control process exited, code=exited status=1
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: Failed to start
nginx - high performance web server.
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: Unit nginx.service
entered failed state.
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: nginx.service
failed.


[root@proxy-lb02 nginx]# nginx -v
nginx version: nginx/1.18.0

Could someone help me find the solution to this error?

Thanks.

Anderson Serra

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,288282,288282#msg-288282

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Load Balancing TCP directive mail {}

Thomas Ward

That's a pretty self-explanatory error actually:

Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br nginx[52777]: nginx: [emerg]
"stream" directive is not allowed here in
/etc/nginx/email/balanceador.conf:1

Your mail configuration file is imported inside a mail block.  That won't work.

Stream operates on the same level as an http or mail block - that is, it's not *part* of the mail{} block but instead its own stream block.  You would need to import the stream function directly at /etc/nginx/nginx.conf root level and NOT as part of the mail{} block.  Details on *that* are in the nginx documentation: http://nginx.org/en/docs/stream/ngx_stream_core_module.html#stream

Basically, you're trying to include the stream configuration at the wrong level - the stream{} block you are configuring needs to be at the nginx.conf base level and NOT as part of the mail{} block as your nginx.conf is trying to do.


Thomas

On 6/6/20 6:51 PM, andersonsserra wrote:
Hi folks,

I'm trying to do a round-robin load balancing for outgoing connections from
my MTA servers. At first I tried it as follows:

[root@proxy-lb02 email]# pwd
/etc/nginx/email
[root@proxy-lb02 email]# cat balanceador.conf
stream {
        upstream stream_backend_mail {
                least_conn;
                server mta-01.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                server mta-02.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                server mta-03.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                server mta-04.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                server mta-05.srvmail.com.br:26  max_fails=2
fail_timeout=15s;
                }


        server {
        listen     0.0.0.0:25;
        proxy_pass stream_backend_mail;
        }
}

[root@proxy-lb02 nginx]# pwd
/etc/nginx
[root@proxy-lb02 nginx]# cat nginx.conf

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request"
'
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;




}

mail {

        include /etc/nginx/email/*.conf;

}


[root@proxy-lb02 nginx]# systemctl restart nginx
Job for nginx.service failed because the control process exited with error
code. See "systemctl status nginx.service" and "journalctl -xe" for
details.

[root@proxy-lb02 nginx]# systemctl status nginx -l
● nginx.service - nginx - high performance web server
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor
preset: disabled)
   Active: failed (Result: exit-code) since Sáb 2020-06-06 18:18:19 EDT; 58s
ago
     Docs: http://nginx.org/en/docs/
  Process: 51695 ExecStop=/bin/kill -s TERM $MAINPID (code=exited,
status=0/SUCCESS)
  Process: 52777 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf
(code=exited, status=1/FAILURE)
 Main PID: 51686 (code=exited, status=0/SUCCESS)

Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: Starting nginx -
high performance web server...
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br nginx[52777]: nginx: [emerg]
"stream" directive is not allowed here in
/etc/nginx/email/balanceador.conf:1
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: nginx.service:
control process exited, code=exited status=1
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: Failed to start
nginx - high performance web server.
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: Unit nginx.service
entered failed state.
Jun 06 18:18:19 proxy-lb02.srvmail.ma.gov.br systemd[1]: nginx.service
failed.


[root@proxy-lb02 nginx]# nginx -v
nginx version: nginx/1.18.0

Could someone help me find the solution to this error?

Thanks.

Anderson Serra

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,288282,288282#msg-288282

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Load Balancing TCP directive mail {}

vergil
In reply to this post by vergil
andersonsserra Wrote:
-------------------------------------------------------
> nginx: [emerg] "stream" directive is not allowed here in
/etc/nginx/email/balanceador.conf:1

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,288282,288291#msg-288291

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Load Balancing TCP directive mail {}

vergil
In reply to this post by Thomas Ward
Thomas,

Thanks for the answer! I made the corrections and the nginx service worked.
Thank you.
Although it worked, I saw that the backend servers are receiving the IP
request from the proxy, is there any way to pass the source IP of the proxy
request preserving the source IP and port?
take a look at this example:
from an IP test server 10.22.51.16 I make a request on port 25 to the proxy
server with address 10.22.8.153, as follows:
anderson @ support-seati: ~ $ telnet 10.22.8.153 25
Trying 10.22.8.153 ...
Connected to 10.22.8.153.
Escape character is '^]'.
220 mta-01.example.com
502 5.5.2 Error: command not recognized


look how it appears on the destination server ...
root @ mta-03: ~ # tail -f /var/log/mail.log | egrep -e "(10.22.51. 16 |
10.22.8.153)"
Jun 8 10:49:21 mta-03 postfix / smtpd [12607]: connect from unknown
[10.22.8.153]


How can I preserve the source port and IP address?

Regards.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,288282,288294#msg-288294

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx