nginx 缓存-沃梦达编程网

浏览器缓存与nginx缓存

浏览器缓存

优点：使用有效缓存时，没有网络消耗，速度快；即使有网络消耗，但对失效缓存使用304响应做到网络消耗最小化

缺点：仅提升一个用户的体验

nginx 缓存

优点：提升所有用户体验，相比浏览器缓存，有效降低上游服务的负载，通过304响应减少nginx与上游服务间的流量消耗

缺点：用户仍然保持网络消耗

同时使用浏览器与nginx缓存

nginx 缓存

Etag 头部

ETagHTTP 响应头是资源的特定版本的标识符。可以是缓存更高效，并节省带宽，因为如果内容没有改变。web服务不需要发送完整的响应。如果内容发生变化，使用ETag有助于防止资源的同时更新互换覆盖；如果给定URL中的资源更改,则一定要生成新的Etag值。因此Etge类似于指纹，也可能被某些服务器用于追踪。比较etags能快速确定此资源是否变化，但也可能被追踪服务器永久保存

W/可选

‘W/’（大小写敏感）表示使用弱验证器，弱验证器很容易生成不利于比较，强验证器是比较的理想选择，但很难生成，相同资源的两个弱Etag值可能语义相同，但不是每个字节都相同

etag 指令

Syntax: etag on | off;
Default: etag on; 
Context: http, server, location

　　生成规则

ngx_sprintf(etag->value.data, "\"%xT-%xO\"",
r->headers_out.last_modified_time,
r->headers_out.content_length_n)

　　If-None-Match

nginx 缓存

If-Modified-Since 头部

not_modified过滤模块

expires 指令

Syntax: expires [modified] time;
expires epoch | max | off;   #指的一个绝对时间，表示缓存在这段时间内一直有效
Default: expires off; 
Context: http, server, location, if in location

　 nginx 缓存

配置

[root@python vhast]# cat cache.conf 
server {
	server_name cache.com;
	error_log logs/cacgeee.log;
	access_log logs/cache.log main;
	root html/;
	location /{
		etag on;  启用
		expires 1h;缓存1小时
		#expires -1h;
		#expires @20h30m;
		#if_modified_since off;
		#proxy_cache two;
		#proxy_cache_valid 100 10m;
		#add_header X-Cache-Status $upstream_cache_status;
		#proxy_cache_use_stale error timeout updating;
		#proxy_cache_revalidate on;
		#proxy_cache_background_update on;
		#proxy_hide_header	Set-Cookie;
		#proxy_ignore_headers	Set-Cookie;
		#proxy_force_ranges on;
		#proxy_pass http://127.0.0.1:8012;
		}
}

　　测试

[root@python vhast]# date -u
2019年 07月 17日 星期三 16:12:16 UTC
[root@python vhast]# curl cache.com/ -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 16:12:19 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
Connection: keep-alive
ETag: "5d262d06-264"
Expires: Wed, 17 Jul 2019 17:12:19 GMT
Cache-Control: max-age=3600
Accept-Ranges: bytes

　配置

[root@python vhast]# cat cache.conf 
server {
	server_name cache.com;
	error_log logs/cacgeee.log;
	access_log logs/cache.log main;
	root html/;
	location /{
		etag on;
		#expires 1h;
		expires -1h;  #设置前一小时失效
		#expires @20h30m;
		#if_modified_since off;
		#proxy_cache two;
		#proxy_cache_valid 100 10m;
		#add_header X-Cache-Status $upstream_cache_status;
		#proxy_cache_use_stale error timeout updating;
		#proxy_cache_revalidate on;
		#proxy_cache_background_update on;
		#proxy_hide_header	Set-Cookie;
		#proxy_ignore_headers	Set-Cookie;
		#proxy_force_ranges on;
		#proxy_pass http://127.0.0.1:8012;
		}
}

　　测试

[root@python vhast]# date -u
2019年 07月 17日 星期三 16:16:37 UTC
[root@python vhast]# curl cache.com/ -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 16:16:48 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
Connection: keep-alive
ETag: "5d262d06-264"
Expires: Wed, 17 Jul 2019 15:16:48 GMT
Cache-Control: no-cache  #缓存失效
Accept-Ranges: bytes

　　配置指的浏览器缓存到指定的时间

[root@python vhast]# cat cache.conf 
server {
	server_name cache.com;
	error_log logs/cacgeee.log;
	access_log logs/cache.log main;
	root html/;
	location /{
		etag on;
		#expires 1h;
		#expires -1h;
		expires @20h30m; 缓存到最近的20点半
		#if_modified_since off;
		#proxy_cache two;
		#proxy_cache_valid 100 10m;
		#add_header X-Cache-Status $upstream_cache_status;
		#proxy_cache_use_stale error timeout updating;
		#proxy_cache_revalidate on;
		#proxy_cache_background_update on;
		#proxy_hide_header	Set-Cookie;
		#proxy_ignore_headers	Set-Cookie;
		#proxy_force_ranges on;
		#proxy_pass http://127.0.0.1:8012;
		}
}

　　测试

[root@python vhast]# date -u
2019年 07月 17日 星期三 16:18:54 UTC
[root@python vhast]# curl cache.com/ -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 16:18:57 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
Connection: keep-alive
ETag: "5d262d06-264"
Expires: Thu, 18 Jul 2019 12:30:00 GMT
Cache-Control: max-age=72663
Accept-Ranges: bytes

not_modified 过滤模块

not_modified 指令

Syntax: if_modified_since off | exact | before;
Default: if_modified_since exact; 
Context: http, server, location

　　 nginx 缓存

测试

[root@python vhast]# curl cache.com/ -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 16:47:10 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
Connection: keep-alive
ETag: "5d262d06-264"
Expires: Thu, 18 Jul 2019 12:30:00 GMT
Cache-Control: max-age=70970
Accept-Ranges: bytes

[root@python vhast]# curl -H 'If_Modified_Since: Wed, 10 Jul 2019 18:23:02 GMT' -H 'If-None-Match: "5d262d06-264"' cache.com/ -I  
HTTP/1.1 304 Not Modified#命中缓存
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 16:47:22 GMT
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
Connection: keep-alive
ETag: "5d262d06-264"
Expires: Thu, 18 Jul 2019 12:30:00 GMT
Cache-Control: max-age=70958

[root@python vhast]# curl -H 'If_Modified_Since: Wed, 10 Jul 2019 18:23:02 GMT' -H 'If-None-Match: "5d262d06-267"' cache.com/ -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 16:48:12 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
Connection: keep-alive
ETag: "5d262d06-264"
Expires: Thu, 18 Jul 2019 12:30:00 GMT
Cache-Control: max-age=70908
Accept-Ranges: bytes

If-Match

If-Unmodified-Since

nginx 缓存：定义存放缓存的载体

Syntax: proxy_cache zone | off;
Default: proxy_cache off; 
Context: http, server, location



Syntax: 
proxy_cache_path path [levels=levels] [use_temp_path=on|off] 
keys_zone=name:size [inactive=time] [max_size=size] [manager_files=number] 
[manager_sleep=time] [manager_threshold=time] [loader_files=number] 
[loader_sleep=time] [loader_threshold=time] [purger=on|off] [purger_files=number] 
[purger_sleep=time] [purger_threshold=time];
Default: —
Context: http

proxy_cache_path 指令

nginx 缓存

缓存关键字

Syntax: proxy_cache_key string;
Default: proxy_cache_key $scheme$proxy_host$request_uri; 
Context: http, server, location

缓存什么样的响应

Syntax: proxy_cache_valid [code ...] time;
Default: —
Context: http, server, location

　　 nginx 缓存

那些内容不使用缓存

参数为真时响应不存入缓存

Syntax: proxy_no_cache string ...;
Default: —
Context: http, server, location

　　参数为真时，不使用缓存内容

Syntax: proxy_cache_bypass string ...;
Default: —
Context: http, server, location

　　变更HEAD方法

Syntax: proxy_cache_convert_head on | off;
Default: proxy_cache_convert_head on; 
Context: http, server, location

　　upstream_cache_status变量

upstream_cache_status

MISS：未命中缓存； HIT：命中缓存； EXPIRED：缓存已过期；STALE：命中陈旧的缓存；UPDATING:内容陈旧，正在更新；REVALIDATED：nginx验证陈旧的内容依然有效；BYPASS：内容是从原始服务器获得

缓存流程

nginx 缓存

对那个method方法使用缓存返回响应

Syntax: proxy_cache_methods GET | HEAD | POST ...;
Default: proxy_cache_methods GET HEAD; 
Context: http, server, location

　配置

server {
	listen 8012;
	default_type text/plain;
	root html;
	location /{
		#add_header X-Accel-Limit-Rate 10;
		
	}
	location /test {

		return 200 '8012 server response.
uri: $uri
method: $request_method
requset: $request
http_name: $http_name
\n';
	}
}

[root@python vhast]# cat cache.conf 
proxy_cache_path /data/web/cache levels=2:2 keys_zone=two:10m loader_threshold=300 loader_files=200 max_size=200m inactive=1m;
server {
	server_name cache.com;
	error_log logs/cacgeee.log;
	access_log logs/cache.log main;
	root html/;
	location /{
		etag on;
		#expires 1h;
		#expires -1h;
		#expires @20h30m;
		#if_modified_since off;
		proxy_cache two;
		proxy_cache_valid 200 1m;
		add_header X-Cache-Status $upstream_cache_status;
		#proxy_cache_use_stale error timeout updating;
		#proxy_cache_revalidate on;
		#proxy_cache_background_update on;
		#proxy_hide_header	Set-Cookie;
		#proxy_ignore_headers	Set-Cookie;
		#proxy_force_ranges on;
		proxy_pass http://127.0.0.1:8012;
		}
}

　　测试

[root@python vhast]# curl  cache.com/a.txt -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 17:23:40 GMT
Content-Type: text/plain
Content-Length: 23
Connection: keep-alive
Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT
ETag: "5d28942d-17"
X-Cache-Status: MISS   第一次
Accept-Ranges: bytes

[root@python vhast]# curl  cache.com/a.txt -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 17:23:44 GMT
Content-Type: text/plain
Content-Length: 23
Connection: keep-alive
Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT
ETag: "5d28942d-17"
X-Cache-Status: HIT  第二次
Accept-Ranges: bytes

X-Accel-Expires头部

nginx 缓存

配置

server {
        listen 8012;
        default_type text/plain;
        #client_body_in_single_buffer on;
        #add_header Cache_Control 'max-age=3,stale-while-revalidate=3';
        #add_header Vary *;
        add_header X-Accel-Expires 3; #定义代理服务器缓存为3秒
        root html;
        location /{
                #add_header X-Accel-Limit-Rate 10;

        }
        location /test {

                return 200 '8012 server response.
uri: $uri
method: $request_method
requset: $request
http_name: $http_name
\n';
        }
}

　　测试

[root@python vhast]# curl  cache.com/a.txt -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 17:35:20 GMT
Content-Type: text/plain
Content-Length: 23
Connection: keep-alive
Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT
ETag: "5d28942d-17"
X-Cache-Status: EXPIRED
Accept-Ranges: bytes

[root@python vhast]# curl  cache.com/a.txt -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 17:35:21 GMT
Content-Type: text/plain
Content-Length: 23
Connection: keep-alive
Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT
ETag: "5d28942d-17"
X-Cache-Status: HIT
Accept-Ranges: bytes

[root@python vhast]# curl  cache.com/a.txt -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 17:35:22 GMT
Content-Type: text/plain
Content-Length: 23
Connection: keep-alive
Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT
ETag: "5d28942d-17"
X-Cache-Status: HIT
Accept-Ranges: bytes

[root@python vhast]# curl  cache.com/a.txt -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 17:35:23 GMT
Content-Type: text/plain
Content-Length: 23
Connection: keep-alive
Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT
ETag: "5d28942d-17"
X-Cache-Status: HIT
Accept-Ranges: bytes

[root@python vhast]# curl  cache.com/a.txt -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 17:35:25 GMT
Content-Type: text/plain
Content-Length: 23
Connection: keep-alive
Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT
ETag: "5d28942d-17"
X-Cache-Status: EXPIRED
Accept-Ranges: bytes

vary 头部

nginx 缓存

配置

server {
	listen 8012;
	default_type text/plain;
	#client_body_in_single_buffer on;
	#add_header Cache_Control 'max-age=3,stale-while-revalidate=3';
	add_header Vary *;  #配置代理不缓存
	add_header X-Accel-Expires 3; #定义上游服务器缓存为3
	root html;
	location /{
		#add_header X-Accel-Limit-Rate 10;
		
	}
	location /test {

		return 200 '8012 server response.
uri: $uri
method: $request_method
requset: $request
http_name: $http_name
\n';
	}
}

　　测试

[root@python vhast]# curl  cache.com/a.txt -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 17:38:23 GMT
Content-Type: text/plain
Content-Length: 23
Connection: keep-alive
Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT
ETag: "5d28942d-17"
Vary: *
Accept-Ranges: bytes
X-Cache-Status: MISS

[root@python vhast]# curl  cache.com/a.txt -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 17:38:23 GMT
Content-Type: text/plain
Content-Length: 23
Connection: keep-alive
Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT
ETag: "5d28942d-17"
Vary: *
Accept-Ranges: bytes
X-Cache-Status: MISS

Set-Cookie头部

Set-Cookie: <cookie-name>=<cookie-value> 
Set-Cookie: <cookie-name>=<cookie-value>; Expires=<date> 
Set-Cookie: <cookie-name>=<cookie-value>; Max-Age=<non-zero-digit> 
Set-Cookie: <cookie-name>=<cookie-value>; Domain=<domain-value> 
Set-Cookie: <cookie-name>=<cookie-value>; Path=<path-value> 
Set-Cookie: <cookie-name>=<cookie-value>; Secure 
Set-Cookie: <cookie-name>=<cookie-value>; HttpOnly 
Set-Cookie: <cookie-name>=<cookie-value>; SameSite=Strict 
Set-Cookie: <cookie-name>=<cookie-value>; SameSite=Lax 
Set-Cookie: <cookie-name>=<cookie-value>; Domain=<domain-value>; Secure; HttpOnly

　　缓存流程：接收上游的响应

nginx 缓存

合并回源请求-减少峰值流量下的压力

Syntax: proxy_cache_lock on | off;
Default: proxy_cache_lock off; 
Context: http, server, location

同一时间，仅第一个请求发往上游，其他请求等待第一个响应返回或者超时候，使用缓存响应客户端
Syntax: proxy_cache_lock_timeout time;
Default: proxy_cache_lock_timeout 5s; 
Context: http, server, location
等待第一个请求返回响应的最大时间，到达后直接向上游发送请求，但不缓存响应
Syntax: proxy_cache_lock_age time;
Default: proxy_cache_lock_age 5s; 
Context: http, server, location
上一个请求返回响应的超时时间，到达后再放行一个请求发送向上游

　　减少回源请求试图stale陈旧的缓存

Syntax: proxy_cache_use_stale error | timeout | invalid_header | 
updating | http_500 | http_502 | http_503 | http_504 | 
http_403 | http_404 | http_429 | off ...;
Default: proxy_cache_use_stale off; 
Context: http, server, location




Syntax: proxy_cache_background_update on | off;
Default: proxy_cache_background_update off; 
Context: http, server, location

　　proxy_cache_use_stale指定陈旧缓存的用法

nginx 缓存

缓存有问题时的响应

Syntax: proxy_cache_background_update on | off;
Default: proxy_cache_background_update off; 
Context: http, server, location

　　当使用proxy_cache_use_stale允许使用过期响应时，将同步生成一个子请求，通过访问上游服务更新过期的缓存

Syntax: proxy_cache_revalidate on | off;
Default: proxy_cache_revalidate off; 
Context: http, server, location

　　更新缓存时，使用If-Modified-Since和If-None-Match作为请求头部，预期内容未发生变更时通过304来减少传输的内容

nginx 缓存

及时清除缓存

模块第三方模块ngx_cache_purge 地址 https://github.com/FRiCKLE/ngx_cache_purge

使用--add-module=指令编译添加到nginx中

功能：接收到指定HTTP请求后立刻清除缓存

•syntax: proxy_cache_purge on|off|<method> [from all|<ip> [.. <ip>]] 
•default: none 
•context: http, server, location




•syntax: proxy_cache_purge zone_name key 
•default: none 
•context: location

　　编译进第三方模块

git clone https://github.com/FRiCKLE/ngx_cache_purge.git
 cd nginx-1.15.9/
./configure --prefix=/data/web --sbin-path=/usr/bin --user=nginx --group=nginx --with-http_stub_status_module --with-http_auth_request_module --with-http_sub_module --add-module=/root/nginx-http-concat --with-http_addition_module --with-http_secure_link_module --with-http_geoip_module  --with-http_ssl_module  --add-module=/root/ngx_cache_purge
 make
mv /usr/bin/nginx{,.01.18.15.41}
cp objs/nginx /usr/bin/

　　配置

[root@python vhast]# cat cache.conf 
proxy_cache_path /data/web/cache levels=2:2 keys_zone=two:10m loader_threshold=300 loader_files=200 max_size=200m inactive=1m;
server {
	server_name cache.com;
	error_log logs/cacgeee.log;
	access_log logs/cache.log main;
	root html/;
	location ~/purge(/.*) {
		proxy_cache_purge two $scheme$1;  匹配上的时候，就清空这个缓存
	}
	location /{
		etag on;
		#expires 1h;
		#expires -1h;
		#expires @20h30m;
		#if_modified_since off;
		proxy_cache two;
		proxy_cache_valid 200 1m;
		add_header X-Cache-Status $upstream_cache_status;
		#proxy_cache_use_stale error timeout updating;
		#proxy_cache_revalidate on;
		#proxy_cache_background_update on;
		#proxy_hide_header	Set-Cookie;
		#proxy_ignore_headers	Set-Cookie;
		#proxy_force_ranges on;
		proxy_cache_key $scheme$uri;
		proxy_pass http://127.0.0.1:8012;
		}
}

　　测试

[root@python vhast]# curl  cache.com/index.html -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 20:16:30 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
ETag: "5d262d06-264"
X-Cache-Status: MISS #未命中
Accept-Ranges: bytes

[root@python vhast]# curl  cache.com/index.html -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 20:16:31 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
ETag: "5d262d06-264"
X-Cache-Status: HIT   #命中
Accept-Ranges: bytes

[root@python vhast]# curl  cache.com/index.html -I
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 20:16:35 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
ETag: "5d262d06-264"
X-Cache-Status: HIT #命中
Accept-Ranges: bytes

[root@python vhast]# curl  cache.com/purge/index.html -I  #清除缓存
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 20:16:40 GMT
Content-Type: text/html
Content-Length: 270
Connection: keep-alive

[root@python vhast]# curl  cache.com/index.html -I 
HTTP/1.1 200 OK
Server: nginx/1.15.9
Date: Wed, 17 Jul 2019 20:16:42 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT
ETag: "5d262d06-264"
X-Cache-Status: MISS #未命中
Accept-Ranges: bytes

七层反向代理对照：构造请求内容

nginx 缓存