Quantcast

Serve index.html file if exists try_files + proxy_pass?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Serve index.html file if exists try_files + proxy_pass?

Lucas Rolff-2
Hi guys,

I have a small scenario where I have a backend (s3 compatible storage), which by default generates a directory listing overview of the files stored.
I want to be able to serve an "index.html" file if the file exists, else just proxy_pass as normally.


My nginx config for somewhat reason doesn't work – or maybe it's because I misunderstand how try_files actually work.

So I have URLs such as:

minio.box.com/bucket1/
minio.box.com/bucket43253/


When I request these URL's I want nginx to check if index.html exists in the directory (it's an actual file on the filesystem) - if it does, serve this one, else go to @minio location.

For any other file within the directory, I will just go to @minio location so if I request unicorn.png it should go in @minio location as well.

Is there any decent (non-evil) way of doing this?

I assume I have to define the root directive to make try_files work, but what would I actually have to define, to make nginx use try_files for index.html *within* the specific bucket?

Thanks in advance


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Serve index.html file if exists try_files + proxy_pass?

Francis Daly
On Sun, Apr 30, 2017 at 10:44:21AM +0000, Lucas Rolff wrote:

Hi there,

> I have a small scenario where I have a backend (s3 compatible storage), which by default generates a directory listing overview of the files stored.
> I want to be able to serve an "index.html" file if the file exists, else just proxy_pass as normally.

I think that it will be very useful to be utterly clear on the distinction
between a file and a url here. If you can describe what you want to happen
in exact terms, there is a much better chance that the configuration
you want will be clear.

A file is a thing available on the local nginx filesystem. Its full name
will be some thing like /usr/local/nginx/html/one/two.

A url is a thing available by proxy_pass:ing to a http server. (That's
good enough for these purposes.) Its full name will be something like
http://upstream/one/two.

(The http server on upstream may have a direct mapping between urls it
receives and files it knows about; that's because those files are on
upstream's local filesystem. Similarly, nginx receives requests which
are urls, and it may map them to files or to other urls. This can get
confusing. That's why it is useful to be explicit.)

> https://gist.github.com/lucasRolff/c7ea13305e9bff40eb6729246cd7eb39
>
> My nginx config for somewhat reason doesn't work – or maybe it's because I misunderstand how try_files actually work.

try_files checks for the existence of a file. In the common case, the full
name of the file that it checks is the concatenation of $document_root
with the argument to try_files.

> So I have URLs such as:
>
> minio.box.com/bucket1/
> minio.box.com/bucket43253/
>
>
> When I request these URL's I want nginx to check if index.html exists in the directory (it's an actual file on the filesystem) - if it does, serve this one, else go to @minio location.

Can you be specific here, with a worked example?

The request to nginx is for /one/two/. What do you want nginx to do? (If
you mention the word "file", please use the full name of the file that
you are interested in.)

Then, a separate request to nginx is for /one/three. Same question.

> For any other file within the directory, I will just go to @minio location so if I request unicorn.png it should go in @minio location as well.
>
> Is there any decent (non-evil) way of doing this?
>
> I assume I have to define the root directive to make try_files work, but what would I actually have to define, to make nginx use try_files for index.html *within* the specific bucket?

nginx does not know about buckets. It knows about incoming requests,
and it knows about files and directories.


I *suspect* that you can do what you want with one "location ~ /$"
inside your "location /"; but I'm not fully clear on what you want.

I also suspect that the correct solution is to just configure the
upstream http server to serve the contents of index.html if it exists,
when it gets a request ending in / -- presumably there's a reason why
that isn't done instead.

Good luck with it,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Serve index.html file if exists try_files + proxy_pass?

Lucas Rolff-2
Hi Francis,

Thanks for your reply.

A little about what I'm doing/trying to make work.

I use Minio (https://www.minio.io/) - which is a S3 compatible object storage server - it's a simple Go binary, that you pass an argument, simply a directory which you want to store your buckets and files in.

In my case, I've created a user called minio, with a homedir of /home/minio
I've configured nginx to run under the user "minio" as well, to ensure correct permissions.

Minio by default listens to 0.0.0.0:9000, and to simplify working with SSL certificates, I ended up putting nginx on the same machine, and all it does is basically a proxy_pass on localhost:9000

When I access https://minio.box.com/<bucket>/ Minio will generate an XML containin a list of objects within a specific bucket (as per S3 API standards).

Example: https://gist.github.com/lucasRolff/7a0afb95103f6c93d8bc448f5c1c35f4

Since I do not want to expose this bucket object list, I want to do so if a bucket has the file "index.html" that it will serve this, instead of showing the bucket object list.

Minio run from /home/minio with the directory /home/minio being the storage directory.
This means, when I create a bucket called "storage", the directory
/home/minio/storage will be created - within the storage directory, objects will be placed, as it was normal files, so if I decide to upload index.html, then I will be able to find the exact file, with that name at path /home/minio/storage/index.html

Now on nginx, if I have the domain https://minio.box.com/storage/ - what I want to load is /home/minio/storage/index.html if the file exists, else load the bucket object list

If I access https://minio.box.com/images/ - it should look for the file /home/minio/images/index.html and serve if existing else load the bucket object list (basically, just proxy_pass as normal).

Any other request I do such as https://minio.box.com/images/nginx-rocks.png should go to my upstream server (localhost:9000)

> I also suspect that the correct solution is to just configure the upstream http server to serve the contents of index.html if it exists

If I could, I would have done that, but it returns a bucket object list as defined in the S3 API standard.

nginx itself can have a root /home/minio; defined - and the 'bucket' is just an actual folder on the file-system, with normal files.

The only problem I have is to serve index.html from within the current 'bucket', so /images/ would load /home/minio/images/index.html

If I do try_files index.html @upstream;

Then try_files will base it on the root directive defined, in this case it would try look for /home/minio/index.html if I set the root directive to "/home/minio", correct?

I guess I could take try_files "${uri}index.html" @upstream; which would produce something like /home/minio/storage/index.html if you have /storage/ as the URI, but if URI is /storage/image1.png it would try to look for "/home/minio/storage/image1.pngindex.html" and for me that doesn't seem very efficient, since it would have to stat for a file on the file system for every request before actually going to my upstream.

I could maybe do:

location / {
  location ~ /$ {
    try_files "${uri}index.html" @upstream;

  }

  // continue normal code here
}

location @upstream {
  proxy_pass http://127.0.0.1:9000;
}

I'm not sure if the above made it more clear.

Best Regards,
Lucas R


Francis Daly wrote:

On Sun, Apr 30, 2017 at 10:44:21AM +0000, Lucas Rolff wrote:

Hi there,

I have a small scenario where I have a backend (s3 compatible storage), which by default generates a directory listing overview of the files stored.
I want to be able to serve an "index.html" file if the file exists, else just proxy_pass as normally.

I think that it will be very useful to be utterly clear on the distinction
between a file and a url here. If you can describe what you want to happen
in exact terms, there is a much better chance that the configuration
you want will be clear.

A file is a thing available on the local nginx filesystem. Its full name
will be some thing like /usr/local/nginx/html/one/two.

A url is a thing available by proxy_pass:ing to a http server. (That's
good enough for these purposes.) Its full name will be something like
http://upstream/one/two.

(The http server on upstream may have a direct mapping between urls it
receives and files it knows about; that's because those files are on
upstream's local filesystem. Similarly, nginx receives requests which
are urls, and it may map them to files or to other urls. This can get
confusing. That's why it is useful to be explicit.)

https://gist.github.com/lucasRolff/c7ea13305e9bff40eb6729246cd7eb39

My nginx config for somewhat reason doesn't work – or maybe it's because I misunderstand how try_files actually work.

try_files checks for the existence of a file. In the common case, the full
name of the file that it checks is the concatenation of $document_root
with the argument to try_files.

So I have URLs such as:

minio.box.com/bucket1/
minio.box.com/bucket43253/


When I request these URL's I want nginx to check if index.html exists in the directory (it's an actual file on the filesystem) - if it does, serve this one, else go to @minio location.

Can you be specific here, with a worked example?

The request to nginx is for /one/two/. What do you want nginx to do? (If
you mention the word "file", please use the full name of the file that
you are interested in.)

Then, a separate request to nginx is for /one/three. Same question.

For any other file within the directory, I will just go to @minio location so if I request unicorn.png it should go in @minio location as well.

Is there any decent (non-evil) way of doing this?

I assume I have to define the root directive to make try_files work, but what would I actually have to define, to make nginx use try_files for index.html *within* the specific bucket?

nginx does not know about buckets. It knows about incoming requests,
and it knows about files and directories.


I *suspect* that you can do what you want with one "location ~ /$"
inside your "location /"; but I'm not fully clear on what you want.

I also suspect that the correct solution is to just configure the
upstream http server to serve the contents of index.html if it exists,
when it gets a request ending in / -- presumably there's a reason why
that isn't done instead.

Good luck with it,

	f


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Serve index.html file if exists try_files + proxy_pass?

Francis Daly
On Mon, May 01, 2017 at 01:50:10PM +0200, Lucas Rolff wrote:

Hi there,

Thanks for the extra explanation. It is clear to me now.

> When I access https://minio.box.com/<bucket>/ Minio will generate an
> XML containin a list of objects within a specific bucket (as per S3
> API standards).
>
> Example: https://gist.github.com/lucasRolff/7a0afb95103f6c93d8bc448f5c1c35f4
>
> Since I do not want to expose this bucket object list, I want to do
> so if a bucket has the file "index.html" that it will serve this,
> instead of showing the bucket object list.

Ok.

For info: that *will* expose the bucket object list if there is no
index.html.

You may prefer a static fallback page, or an error indication instead.

> If I access https://minio.box.com/images/ - it should look for the
> file /home/minio/images/index.html and serve if existing else load
> the bucket object list (basically, just proxy_pass as normal).
>
> Any other request I do such as
> https://minio.box.com/images/nginx-rocks.png should go to my
> upstream server (localhost:9000)

> If I do try_files index.html @upstream;
>
> Then try_files will base it on the root directive defined, in this
> case it would try look for /home/minio/index.html if I set the root
> directive to "/home/minio", correct?

Correct.

> I guess I could take try_files "${uri}index.html" @upstream; which
> would produce something like /home/minio/storage/index.html if you
> have /storage/ as the URI, but if URI is /storage/image1.png it
> would try to look for "/home/minio/storage/image1.pngindex.html" and
> for me that doesn't seem very efficient, since it would have to stat
> for a file on the file system for every request before actually
> going to my upstream.

Also correct.

> I could maybe do:
>
> location / {
>   location ~ /$ {
>     try_files "${uri}index.html" @upstream;
>
>   }
>
>   // continue normal code here
> }
>
> location @upstream {
>   proxy_pass <a href="http://127.0.0.1:9000;">http://127.0.0.1:9000;
> }

That is what I would suggest. Where "// continue normal code here" is
"proxy_pass <a href="http://127.0.0.1:9000;">http://127.0.0.1:9000;". And "root /home/minio;" is set
somewhere so that it applies where try_files is.

Good luck with it,

        f
--
Francis Daly        [hidden email]
_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx
Loading...