Quantcast

No referrer header on leacher's site !!

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

No referrer header on leacher's site !!

shahzaib mushtaq
Hi,

We came across a website who is playing our video links remotely. Since we've hotlinking protection enabled based on referrer headers so i checked the request header by playing that video & found out that referrer header was missing in the browser's requests header tab. 

Then to generate same issue on our end, i statically added the video link in player on different domain & tried to play that video remotely which was successfully forbidden & browser had referrer header as well. 

Please have a note that he didn't embedded the video from our website, he's putting direct mp4 links & they are being played without any referrer header in the requests.

Thanks for your help in advance !!

Regards.
Shahzaib


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No referrer header on leacher's site !!

Edho Arief-2
Hi,

On Tue, Apr 4, 2017, at 20:24, shahzaib mushtaq wrote:
> Hi,
>
> We came across a website who is playing our video links remotely. Since
> we've hotlinking protection enabled based on referrer headers so i
> checked
> the request header by playing that video & found out that *referrer
> header
> was missing* in the browser's requests header tab.
>

If your site isn't https but his site is, some browsers by default don't
send referrer header. There are also various other referrer policies
with varying level of support:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy

http://caniuse.com/#search=referrer%20policy
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No referrer header on leacher's site !!

shahzaib mushtaq
Hi,

Thanks for quick response. Well its reverse, he's putting our HTTPS video link on his HTTP website. Could that create issue as well? If yes, what's the fix of it.

Again thanks for your help.

On Tue, Apr 4, 2017 at 4:32 PM, nanaya <[hidden email]> wrote:
Hi,

On Tue, Apr 4, 2017, at 20:24, shahzaib mushtaq wrote:
> Hi,
>
> We came across a website who is playing our video links remotely. Since
> we've hotlinking protection enabled based on referrer headers so i
> checked
> the request header by playing that video & found out that *referrer
> header
> was missing* in the browser's requests header tab.
>

If your site isn't https but his site is, some browsers by default don't
send referrer header. There are also various other referrer policies
with varying level of support:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy

http://caniuse.com/#search=referrer%20policy
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No referrer header on leacher's site !!

Richard Stanway
With the controls sites have over the referrer header, it's not very
effective as an access control mechanism. You can use something like
http://nginx.org/en/docs/http/ngx_http_secure_link_module.html
instead.

On Tue, Apr 4, 2017 at 1:39 PM, shahzaib mushtaq <[hidden email]> wrote:

> Hi,
>
> Thanks for quick response. Well its reverse, he's putting our HTTPS video
> link on his HTTP website. Could that create issue as well? If yes, what's
> the fix of it.
>
> Again thanks for your help.
>
> On Tue, Apr 4, 2017 at 4:32 PM, nanaya <[hidden email]> wrote:
>>
>> Hi,
>>
>> On Tue, Apr 4, 2017, at 20:24, shahzaib mushtaq wrote:
>> > Hi,
>> >
>> > We came across a website who is playing our video links remotely. Since
>> > we've hotlinking protection enabled based on referrer headers so i
>> > checked
>> > the request header by playing that video & found out that *referrer
>> > header
>> > was missing* in the browser's requests header tab.
>> >
>>
>> If your site isn't https but his site is, some browsers by default don't
>> send referrer header. There are also various other referrer policies
>> with varying level of support:
>>
>> https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy
>>
>> http://caniuse.com/#search=referrer%20policy
>> _______________________________________________
>> nginx mailing list
>> [hidden email]
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>
> _______________________________________________
> nginx mailing list
> [hidden email]
> http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No referrer header on leacher's site !!

Francis Daly
In reply to this post by shahzaib mushtaq
On Tue, Apr 04, 2017 at 04:39:23PM +0500, shahzaib mushtaq wrote:

Hi there,

> Thanks for quick response. Well its reverse, he's putting our HTTPS video
> link on his HTTP website. Could that create issue as well? If yes, what's
> the fix of it.

nginx does not know (or care) what the linking site does. All it can
see is the request made to it.

The browser entirely controls what request headers the browser sends.

If you want to deny all requests that have no Referer header, you can
do that.

If you want to deny only some requests that have no Referer header,
you will need to tell nginx which requests to deny and which requests to
allow. But before you can do that, you will have to know how to identify
the requests in one of the sets.

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No referrer header on leacher's site !!

shahzaib mushtaq
>>With the controls sites have over the referrer header, it's not very
effective as an access control mechanism. You can use something like
http://nginx.org/en/docs/http/ngx_http_secure_link_module.html
instead.

We're also using Nginx secure link module based on HASH + expiry but somehow this secure link is exploited by that website. The video link hash on his website is exactly matching with ours means no matter if hash get expire & new takes it place that leacher is also getting the new hash & we're unable to find how he exploited us. Though on digging more into this we found that he's using following script to fetch video links from our website : 


His website name is also dizibox1.


On Wed, Apr 5, 2017 at 1:54 AM, Francis Daly <[hidden email]> wrote:
On Tue, Apr 04, 2017 at 04:39:23PM +0500, shahzaib mushtaq wrote:

Hi there,

> Thanks for quick response. Well its reverse, he's putting our HTTPS video
> link on his HTTP website. Could that create issue as well? If yes, what's
> the fix of it.

nginx does not know (or care) what the linking site does. All it can
see is the request made to it.

The browser entirely controls what request headers the browser sends.

If you want to deny all requests that have no Referer header, you can
do that.

If you want to deny only some requests that have no Referer header,
you will need to tell nginx which requests to deny and which requests to
allow. But before you can do that, you will have to know how to identify
the requests in one of the sets.

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No referrer header on leacher's site !!

tory
Hello There,

I had this same issue and fixed it by the following method.

For example in HTML :
<source src="file.mp4?md5=jobIVRUfgH6USADuWsqJHr818vw&expires=1478192353"
type="video/mp4" />

That is what your media stream link would look like.

But if you use JavaScript like the following example :

<script type="text/javascript">
window.onload = MediaReplacement()
function MediaReplacement() {
var _video =
"/file.mp4?md5=jobIVRUfgH6USADuWsqJHr818vw&expires=1478192353";

var videoTags = document.getElementsByTagName("source");
videoTags[0].src = _video;
}
</script>
<video width="320" height="240" controls="controls">
<!-- MP4 for Safari, IE9, iPhone, iPad, Android, and Windows Phone 7 -->
<source src="" type="video/mp4" /> <!-- NOTICE HOW I MADE THIS EMPTY BECAUSE
JAVASCRIPT WILL INSERT THIS NOW -->
</video>

And you insert your stream link into the page using JavaScript it unlocks
the ability to make it hard for their python script to
scrape/hotlink/content leech of your web pages.

You can obfuscate JavaScript you can change the var names you can make it
incredibly dynamic and difficult breaking their apps completely the more
dynamic it is the harder and harder it is for them to obtain your stream
links.


Also you should blocked the following two user agents that those apps use.

Kodi
XBMC

(I would suggest making them non case sensitive matches too)

Where I posted in regards to this.
https://forum.nginx.org/read.php?2,270705,270739#msg-270739
https://github.com/C0nw0nk/Nginx-Lua-Secure-Link-Anti-Hotlinking

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,273405,273447#msg-273447

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No referrer header on leacher's site !!

Dmitry S. Polyakov
In reply to this post by shahzaib mushtaq


On Thu, Apr 6, 2017, 10:50 shahzaib mushtaq <[hidden email]> wrote:
>>With the controls sites have over the referrer header, it's not very
effective as an access control mechanism. You can use something like
http://nginx.org/en/docs/http/ngx_http_secure_link_module.html
instead.

We're also using Nginx secure link module based on HASH + expiry but somehow this secure link is exploited by that website. The video link hash on his website is exactly matching with ours means no matter if hash get expire & new takes it place that leacher is also getting the new hash & we're unable to find how he exploited us. Though on digging more into this we found that he's using following script to fetch video links from our website : 


His website name is also dizibox1.
IT happens because your secure  links hash doesn't have any end user unique attributes like ip address 
If you'll include enduser ip to the secure link hash, secure link become unique for the end user. Any direct video link grabbed and shared by the enduser or some script become useless. 



On Wed, Apr 5, 2017 at 1:54 AM, Francis Daly <[hidden email]> wrote:
On Tue, Apr 04, 2017 at 04:39:23PM +0500, shahzaib mushtaq wrote:

Hi there,

> Thanks for quick response. Well its reverse, he's putting our HTTPS video
> link on his HTTP website. Could that create issue as well? If yes, what's
> the fix of it.

nginx does not know (or care) what the linking site does. All it can
see is the request made to it.

The browser entirely controls what request headers the browser sends.

If you want to deny all requests that have no Referer header, you can
do that.

If you want to deny only some requests that have no Referer header,
you will need to tell nginx which requests to deny and which requests to
allow. But before you can do that, you will have to know how to identify
the requests in one of the sets.

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No referrer header on leacher's site !!

tory
Dmitry S. Polyakov Wrote:
-------------------------------------------------------

> On Thu, Apr 6, 2017, 10:50 shahzaib mushtaq <[hidden email]>
> wrote:
>
> > >>With the controls sites have over the referrer header, it's not
> very
> > effective as an access control mechanism. You can use something like
> > http://nginx.org/en/docs/http/ngx_http_secure_link_module.html
> > instead.
> >
> > We're also using Nginx secure link module based on HASH + expiry but
> > somehow this secure link is exploited by that website. The video
> link hash
> > on his website is exactly matching with ours means no matter if hash
> get
> > expire & new takes it place that leacher is also getting the new
> hash &
> > we're unable to find how he exploited us. Though on digging more
> into this
> > we found that he's using following script to fetch video links from
> our
> > website :
> >
> >
> >
> https://github.com/XvBMC/repository.xvbmc/blob/master/plugin.video.sal
> tsrd.lite/scrapers/dizibox_scraper.py
> >
> > His website name is also dizibox1.
> >
> IT happens because your secure  links hash doesn't have any end user
> unique
> attributes like ip address
> If you'll include enduser ip to the secure link hash, secure link
> become
> unique for the end user. Any direct video link grabbed and shared by
> the
> enduser or some script become useless.


You would think that but with Kodi/XBMC that is not the case their App grabs
and sends a HTML request on a per user basis.

So each and every request comes from a users Kodi box or app on their phone
etc what when the page generates the HTML response to that user it also
generated the response for their IP address.

It is like real web traffic.

I prevented them as I explained here
https://forum.nginx.org/read.php?2,273405,273447#msg-273447

Also if you browse and view pornhub, pornsocket, youtube what ever streaming
sites etc you will see they now hide and obfuscate their stream links in
JavaScript to break these kodi box users as I explained in the link above.

Here is proof :
<script type="text/javascript">
        /*This entire area would be their broken up url link obfuscated to be put
back together again by JavaScript making it unreadable for these kodi/xbmc
users */ = quality_720p;
       
        loadScriptUniqueId.push('111418492');
        loadScriptVar.push(flashvars_111418492);

        playerObjList.playerDiv_111418492 = {
                'flashvars' : {"embedId":111418492},
                'embedSWF' :
{"url":"https:\/\/bi.phncdn.com\/www-static\/flash\/","element":"playerDiv_111418492","width":"100%","height":"100%","version":"9.0.0"} };
</script>
        <div id="playerDiv_111418492" class="playerFlvContainer" data-enlarge="1"
data-showautoplayoption="1" data-share="1">
                <noscript>
                        <video style="width:100%; height:100%;" controls="controls"
autobuffer="autobuffer" class="player-html5" preload="metadata">
                                <source src="" type="video/mp4">
                        </video>
                </noscript>

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,273405,273449#msg-273449

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Loading...