Removing Null Character from Query Parameter

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Removing Null Character from Query Parameter

vergil
Nginx Upstream returning 400 Bad Request if null character is being passed
in the request as part of uri or query params.

Is there a way Null Character can be removed from request before proxying
it to upstream.

Its only known from access logs that null character is being passed in
request as \x00 and causing the failure

How to identify the Null Character and remove it ?

Tried below options but its not able to identify the null character

if ($args ~* (.*)(\x00)(.*)) {
     set $args $1$3;
}


Nginx returning below error

Error Log

2020/06/25 20:20:43 [info] 19838#19838: *11985 client sent invalid request
while reading client request line, client: 10.49.120.61, server: test.com,
request: "HEAD /folder/Test.m3u8?uid=abc123 HTTP/1.0"


Access log

 10.49.120.61 | - | test.com | [25/Jun/2020:20:20:43 +0530] | - | "HEAD
/folder/Test.m3u8?uid=abc123\x00 HTTP/1.0" | 400 | 0 | "-" | "-" | 0.001 | -
| - | - | "- - - -" | http | - | -| "-"

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,288455,288455#msg-288455

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Removing Null Character from Query Parameter

Maxim Dounin
Hello!

On Thu, Jun 25, 2020 at 11:33:29AM -0400, anish10dec wrote:

> Nginx Upstream returning 400 Bad Request if null character is being passed
> in the request as part of uri or query params.
>
> Is there a way Null Character can be removed from request before proxying
> it to upstream.
>
> Its only known from access logs that null character is being passed in
> request as \x00 and causing the failure

The null character is not allowed in the HTTP request line, and
hence nginx returns 400 (Bad Request) error.

> How to identify the Null Character and remove it ?

You can't.  Instead, consider fixing the client to generate HTTP
requests correctly.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Removing Null Character from Query Parameter

vergil
Thanks Maxim

Actually null character is not being generated by Client .

We are using below module to validate the tokens
https://github.com/kaltura/nginx-akamai-token-validate-module

This is being caused by akamai_token_validate_strip_token directive which
strips the token and forwards request to upstream server.

While striping the token and passing the remaining request  to upstream
stream its appending null character at the end.
If there is no any additional query param in request apart from token , then
there is no issue in handling.

http://10.49.120.61/folder/Test.m3u8?token=st=1593095161~exp=1593112361~acl=/*~hmac=60d9c29a65d837b203225318d1c69e205037580a08bf4417d4a1e237e5a2f5b6&uid=abc123

Request passed to upstream is as below which is causing problem

GET /folder/Test.m3u8?uid=abc123\x00

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,288455,288462#msg-288462

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Removing Null Character from Query Parameter

Maxim Dounin
Hello!

On Thu, Jun 25, 2020 at 02:02:35PM -0400, anish10dec wrote:

> Thanks Maxim
>
> Actually null character is not being generated by Client .
>
> We are using below module to validate the tokens
> https://github.com/kaltura/nginx-akamai-token-validate-module
>
> This is being caused by akamai_token_validate_strip_token directive which
> strips the token and forwards request to upstream server.
>
> While striping the token and passing the remaining request  to upstream
> stream its appending null character at the end.
> If there is no any additional query param in request apart from token , then
> there is no issue in handling.
>
> http://10.49.120.61/folder/Test.m3u8?token=st=1593095161~exp=1593112361~acl=/*~hmac=60d9c29a65d837b203225318d1c69e205037580a08bf4417d4a1e237e5a2f5b6&uid=abc123
>
> Request passed to upstream is as below which is causing problem
>
> GET /folder/Test.m3u8?uid=abc123\x00

So the module is broken and needs to be fixed.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Removing Null Character from Query Parameter

Jeff Dyke
no offense to the OP, but i love Maxim.  Direct and to the point, and in this case, as usual, he is correct.  You should not look at what the requester wants, before understanding what the sender should provide.

On Thu, Jun 25, 2020 at 2:18 PM Maxim Dounin <[hidden email]> wrote:
Hello!

On Thu, Jun 25, 2020 at 02:02:35PM -0400, anish10dec wrote:

> Thanks Maxim
>
> Actually null character is not being generated by Client .
>
> We are using below module to validate the tokens
> https://github.com/kaltura/nginx-akamai-token-validate-module
>
> This is being caused by akamai_token_validate_strip_token directive which
> strips the token and forwards request to upstream server.
>
> While striping the token and passing the remaining request  to upstream
> stream its appending null character at the end.
> If there is no any additional query param in request apart from token , then
> there is no issue in handling.
>
> http://10.49.120.61/folder/Test.m3u8?token=st=1593095161~exp=1593112361~acl=/*~hmac=60d9c29a65d837b203225318d1c69e205037580a08bf4417d4a1e237e5a2f5b6&uid=abc123
>
> Request passed to upstream is as below which is causing problem
>
> GET /folder/Test.m3u8?uid=abc123\x00

So the module is broken and needs to be fixed.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Removing Null Character from Query Parameter

vergil
Thanks Maxim
Will fix the module , just was looking a way around if it can be handled by
just removing the null character

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,288455,288472#msg-288472

_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|

Re: Removing Null Character from Query Parameter

vergil
In reply to this post by Maxim Dounin