I need my “bad user agent” map not to block my rss xml file

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

I need my “bad user agent” map not to block my rss xml file

lists@lazygranch.com
Stripping down the nginx.conf file:

server{
location / {
 root   /usr/share/nginx/html/mydomain/public_html;
        if ($badagent)     { return 403; }
}
location = /feeds {
        try_files $uri $uri.xml $uri/ ;
       }
}
The "=" should force an exact match, but the badagent map is checked.
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: I need my “bad user agent” map not to block my rss xml file

Francis Daly
On Tue, Jan 08, 2019 at 07:30:44PM -0800, [hidden email] wrote:

Hi there,

> Stripping down the nginx.conf file:
>
> server{
> location / {
>  root   /usr/share/nginx/html/mydomain/public_html;
>         if ($badagent)     { return 403; }
> }
> location = /feeds {
>         try_files $uri $uri.xml $uri/ ;
>        }
> }
> The "=" should force an exact match, but the badagent map is checked.

What file on your filesystem is your rss xml file?

Is it something other than /usr/local/nginx/html/feeds or
/usr/local/nginx/html/feeds.xml?

And what request do you make to fetch your rss xml file?

Do things change if you move the "root" directive out of the "location"
block so that it is directly in the "server" block?

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: I need my “bad user agent” map not to block my rss xml file

lists@lazygranch.com
On Wed, 9 Jan 2019 08:20:05 +0000
Francis Daly <[hidden email]> wrote:

> On Tue, Jan 08, 2019 at 07:30:44PM -0800, [hidden email] wrote:
>
> Hi there,
>
> > Stripping down the nginx.conf file:
> >
> > server{
> > location / {
> >  root   /usr/share/nginx/html/mydomain/public_html;
> >         if ($badagent)     { return 403; }
> > }
> > location = /feeds {
> >         try_files $uri $uri.xml $uri/ ;
> >        }
> > }
> > The "=" should force an exact match, but the badagent map is
> > checked.  
>
> What file on your filesystem is your rss xml file?
>
> Is it something other than /usr/local/nginx/html/feeds or
> /usr/local/nginx/html/feeds.xml?
>
> And what request do you make to fetch your rss xml file?
>
> Do things change if you move the "root" directive out of the
> "location" block so that it is directly in the "server" block?
>
> f

Good catch on the root declaration. Actually I declared it twice. Once
under server and once under location. I got rid of the declaration
under location since that is the wrong place.

So it is now:
 server{
  root   /usr/share/nginx/html/mydomain/public_html;
 location / {
         if ($badagent)     { return 403; }
 }
 location = /feeds {
         try_files $uri $uri.xml $uri/ ;
        }
 }

 The "=" should force an exact match, but the badagent map is
 checked.  

Absolutely the badagent check under location / is being triggered.
Everything works if I comment out the check.

The URL to request the XML file is domain.com/feeds/file.xml .
It is located in /usr/share/nginx/html/mydomain/public_html/feeds .

Here is the access.log file. First line is with the badagent check
skipped. Second line is with it enable.
200 xxx.58.22.151 - - [10/Jan/2019:02:07:42 +0000] "GET /feeds/file.xml HTTP/1.1" 3614 "-" "-" "-"
403 xxx.58.22.151 - - [10/Jan/2019:02:08:38 +0000] "GET /feeds/file.xml HTTP/1.1" 169 "-" "-" "-"
I'm using the RSS reader Akregator in this case. Some readers work fine
since they act more like browsers.



_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: I need my “bad user agent” map not to block my rss xml file

Francis Daly
On Wed, Jan 09, 2019 at 06:14:04PM -0800, [hidden email] wrote:

Hi there,

>  location / {
>          if ($badagent)     { return 403; }
>  }
>  location = /feeds {
>          try_files $uri $uri.xml $uri/ ;
>         }

>  The "=" should force an exact match, but the badagent map is
>  checked.  
>
> Absolutely the badagent check under location / is being triggered.
> Everything works if I comment out the check.
>
> The URL to request the XML file is domain.com/feeds/file.xml .

If the request is /feeds/file.xml, that will not exactly match "/feeds".

  location = /feeds/file.xml {}

should serve the file feeds/file.xml below the document root.

Or, if you want to handle all requests that start with /feeds/ in a
similar way,

  location /feeds/ {}

or

  location ^~ /feeds/ {}

should do that. (The two are different if you have regex locations in
the config.)

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: I need my “bad user agent” map not to block my rss xml file

lists@lazygranch.com
On Thu, 10 Jan 2019 08:50:33 +0000
Francis Daly <[hidden email]> wrote:

> On Wed, Jan 09, 2019 at 06:14:04PM -0800, [hidden email] wrote:
>
> Hi there,
>
> >  location / {
> >          if ($badagent)     { return 403; }
> >  }
> >  location = /feeds {
> >          try_files $uri $uri.xml $uri/ ;
> >         }  
>
> >  The "=" should force an exact match, but the badagent map is
> >  checked.  
> >
> > Absolutely the badagent check under location / is being triggered.
> > Everything works if I comment out the check.
> >
> > The URL to request the XML file is domain.com/feeds/file.xml .  
>
> If the request is /feeds/file.xml, that will not exactly match
> "/feeds".
>
>   location = /feeds/file.xml {}
>
> should serve the file feeds/file.xml below the document root.
>
> Or, if you want to handle all requests that start with /feeds/ in a
> similar way,
>
>   location /feeds/ {}
>
> or
>
>   location ^~ /feeds/ {}
>
> should do that. (The two are different if you have regex locations in
> the config.)
>
> f

There is a certain irony in that I first started out with
location /feeds/ {}
BUT I had the extra root statement. This appears to work. Thanks.

Here are a few tests:
claws rssyl plugin
200 xxx.58.22.151 - - [11/Jan/2019:03:48:25 +0000] "GET /feeds/feed.xml HTTP/1.1" 3614 "-" "libfeed 0.1" "-"

akragator
200 xxx.58.22.151 - - [11/Jan/2019:03:50:39 +0000] "GET /feeds/feed.xml HTTP/1.1" 3614 "-" "-" "-"

liferea
200 xxx.58.22.151 - - [11/Jan/2019:03:51:40 +0000] "GET /feeds/feed.xml HTTP/1.1" 3614 "-" "Liferea/1.10.19 (Linux; en_US.UTF-8; http://liferea.sf.net/) AppleWebKit (KHTML, like Gecko)" "-"

read on android
304 myvpnip - - [11/Jan/2019:03:55:44 +0000] "GET /feeds/feed.xml HTTP/1.1" 0 "-" "Mozilla/5.0 (Linux; Android 8.1.0; myphone) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/71.0.3578.99 Mobile Safari/537.36" "-"

feedbucket (a web based reader) They use a proxy
200 162.246.57.122 - - [11/Jan/2019:04:01:06 +0000] "GET /feeds/feed.xml HTTP/1.1" 3614 "-" "FeedBucket/1.0 \x5C(+http://www.feedbucket.com\x5C)" "-"

 
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx