This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the web-development category.
Last Updated: 2024-11-23
"One reason to proxy to other servers from Nginx is the ability to scale out your infrastructure. Nginx is built to handle many concurrent connections at the same time. This makes it ideal for being the point-of-contact for clients. The server can pass requests to any number of backend servers to handle the bulk of the work, which spreads the load across your infrastructure. This design also provides you with flexibility in easily adding backend servers or taking them down as needed for maintenance." -
E.g. nginx might be able to handle 1000 connections when your server can only handle 32
The servers that Nginx proxies requests to are known as upstream servers.
Proxy pass is usually found in a location directive. When a request matches a location with a proxy_pass directive inside, the request is forwarded to the URL given by the directive.
location /match/here {
// This would pass the URL as is upstream (i.e. with /match/here/X where x can be anything appended)
proxy_pass http://example.com;
// This would replace /match/here with /new/prefix
proxy_pass http://example.com/new/prefix;
}
The request coming from Nginx on behalf of a client will look different than a request coming directly from a client. A big part of this is the headers that go along with the request.
When Nginx proxies a request, it automatically makes some adjustments to the request headers it receives from the client:
- Nginx gets rid of any empty headers. There is no point of passing along empty values to another server; it would only serve to bloat the request.
- Nginx, by default, will consider any header that contains underscores as invalid. It will remove these from the proxied request. If you wish to have Nginx interpret these as valid, you can set the underscores_in_headers directive to “on”, otherwise your headers will never make it to the backend server. NB: Therefore if you set custom heads, you need to ensure they are compatible with nginx.
Here is us adding headers via NGINX, headers that will add useful info. They use variables liek `$Host` and `$scheme` and `$remote_addr` that nginx makes available to the user.
```c
location /match/here {
proxy_set_header HOST $host; / $host variable, which should contain information about the original host being requested
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr; / IP address of the client so that the proxy can correctly make decisions or log
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; / X-Forwarded-For header is a list containing the IP addresses of every server the client has been proxied through up to this point.
}
How does nginx do the load balancing? It has a bunch of algorithms - round robin (default) - each server receives sequentially in turn - leastconn - given to the backend that has the least number of active connections. esp useful in situations where connections persist for time - iphash: This balancing algorithm distributes requests to different servers based on the client’s IP address. The first three octets are used as a key to decide on the server to handle the request. The result is that clients tend to be served by the same server each time, which can assist in session consistency.
You can also set weights for each server
upstream backend_hosts {
least_conn;
server host1.example.com weight 3;
server host2.example.com;
server host3.example.com;
}
Ok so what about these buffers?
Nginx defaults to a buffering design since clients tend to have vastly different connection speeds. This can be changed with proxy_buffering off; at top-level. You can configure the number of buffers with proxy_buffers 24 4k; (number and size -- 4k is one memory page here - fixed-length contiguous block of virtual memory, described by a single entry in the page table. You can find out with python -c "import resource; print(resource.getpagesize())")
In general buffering is into memroy, but when the response is too big, it goes and gets stored on disk at proxy_temp_path.
--
A high availability (HA) setup is an infrastructure without a single point of failure, and your load balancers are a part of this configuration. By having more than one load balancer, you prevent potential downtime if your load balancer is unavailable or if you need to take them down for maintenance.
Nginx proxying can be made more robust by adding in a redundant set of load balancers, creating a high availability infrastructure.
--
Nginx also provides a way to cache content from backend servers, eliminating the need to connect to the upstream at all for many requests. To set up a cache to use for proxied content, we can use the proxycachepath directive. This will create an area where data returned from the proxied servers can be kept. The proxycachepath directive must be set in the http context.
proxy_cache_path /var/lib/nginx/cache levels=1:2 keys_zone=backcache:8m max_size=50m; // directory on file system. needs to be created and have correct permissions
// sudo mkdir -p /var/lib/nginx/cache ; sudo chown www-data /var/lib/nginx/cache; sudo chmod 700 /var/lib/nginx/cache
// levels is how the cached data is store - subdir with one letter then two letters
// keys_zone is how much meta-data (8mb = approx 64k keys)
// max_size is cache size
proxy_cache_key "$scheme$request_method$host$request_uri$is_args$args"; // The key setting this to a combination of the scheme (http or https), the HTTP request method, as well as the requested host and URI.
proxy_cache_valid 200 302 10m; // keep 200/302s for 10m
proxy_cache_valid 404 1m; // keep 404 for 1m
To USE the cache you need more config
location /proxy-me {
proxy_cache backcache; // which cache to use -- key_zone above
proxy_cache_bypass $http_cache_control; // This will contain an indicator as to whether the client is explicitly requesting a fresh, non-cached version of the resource. Setting this directive allows Nginx to correctly handle these types of client requests
add_header X-Proxy-Cache $upstream_cache_status; // Basically, this sets a header that allows us to see if the request resulted in a cache hit, a cache miss, or if the cache was explicitly bypassed. GREAT for debugging
proxy_pass http://backend;
}
tips a: use keepalive connections between nginx and upstream (otherwise waste resources). Note that the keepalive directive does not limit the total number of connections to upstream servers that an NGINX worker process can open – it's just about having some idle connections handy. Set to twice the number of upstream servers as a start.