说明

Web File Browser一个轻量级网盘(私有云)项目,github上star 19.6K，k8s竟然面临数据持久化的问题，本文介绍如何在k8s上部署并持久化数据

项目地址：filebrowser
Docker
File Browser is also available as a Docker image. The usage is as follows:

1
2
3
4
5
6
7
docker run \
    -v /path/to/root:/srv \
    -v /path/filebrowser.db:/database.db \
    -v /path/.filebrowser.json:/.filebrowser.json \
    -u $(id -u):$(id -g) \
    -p 8080:80 \
    filebrowser/filebrowser

但是实际使用中，你会发现这样直接运行会报错,-v /path/.filebrowser.json:/.filebrowser.json这个直接映射的话,会启动出错,这是第一个坑
如果docker部署去掉-v /path/.filebrowser.json:/.filebrowser.json应该问题不大

1. 相关清单及说明

  1. 这个不能直接改写为k8s的yaml文件
  2. 因为k8s的yaml文件中，volumeMounts的挂载路径不能为根目录，这是其一
  3. filebrowser.db是一个数据库二进制文件，不能直接挂载到pvc上，这是其二
  4. 不是因为是二进制文件不能进行挂载,而是文件是不能直接通过pvc挂载到pod上的，在k8s中文件只能使用secret或者configmap进行挂载,只有目录才能使用pvc进行挂载
  5. 但是如果使用secret或者configmap进行挂载是不现实的，因为这个文件是二进制文件，而且会随着使用而变化
  6. 所有我这里运用了下面的小技巧解决这个问题

1.1 命名空间

文章结尾有完整的yaml配置文件

1
2
3
4
5
apiVersion: v1
kind: Namespace
metadata:
  name: filebrowser
---

1.2 StorageClass

1
2
3
4
5
6
7
8
9
10
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: filebrowser-storage
provisioner: kubernetes.io/no-provisioner
parameters:
  type: pd-ssd # 存储参数，这里针对 SSD 存储设备进行优化
volumeBindingMode: WaitForFirstConsumer
---

1.3 PersistentVolume

这里我创建两个pv分别用来存储filebrowser.db、filebrowser.json等配置文件和srv目录(你要放文件的目录)

  1. 需要注意storage: 20Gi的单位,请使用Gi、Mi等单位,不要使用G、M、Gb、Mb等单位,会报错
  2. nodeAffinity是节点选择,这里我选择了k8s-master1节点,PV的节点需要提前创建好
  3. filebrowser-pv用来存放数据,可以给大一点,至于filebrowser-db-pv用来存放配置文件,可以给小一点
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: filebrowser-pv
  namespace: filebrowser
spec:
  storageClassName: filebrowser-storage
  claimRef:
    name: filebrowser-pvc
    namespace: filebrowser
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteOnce
  local:
    path: /mnt/filebrowser/file
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-master1          #节点选择
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: filebrowser-db-pv
  namespace: filebrowser
spec:
  storageClassName: filebrowser-storage
  claimRef:
    name: filebrowser-db-pvc
    namespace: filebrowser
  capacity:
    storage: 200Mi
  accessModes:
    - ReadWriteOnce
  local:
    path: /mnt/filebrowser/db
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-master1                 #同上节点选择
---
### 1.4 PersistentVolumeClaim
```yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: filebrowser-pvc
  namespace: filebrowser
spec:
  storageClassName: filebrowser-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 15Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: filebrowser-db-pvc
  namespace: filebrowser
spec:
  storageClassName: filebrowser-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 150Mi
---

1.5 Deployment特别重要!!!

  1. filebrowser中你是没法直接使用sh、bash等命令的,你只能使用./filebrowser的命令,然后在使用所提供的参数,有可能是我不会,如果你能执行诸如sh、bash的命令请告诉我
  2. 配置文件.filebrowser.json是根目录下的一个隐藏文件,其内容我会贴在下面
  3. 你是无法直接修改.filebrowser.json文件的,即使修改了也不会生效,这点可以去看官方的Issues
  4. 需要实现持久化,我们需要自定义一个config.json文件,将其创建在非根目录路径,然后让其生效替换掉.filebrowser.json的配置,最后将config.json和config.json中指定的数据库文件filebrowser.db一同挂载出来就能实现持久化了
  5. 使用command: ["./filebrowser"]和args: ["--config", "/db/config.json"]来实现让config.json生效
  6. 在config.json中指定的数据库文件filebrowser.db和所在目录替换掉根目录下的database.db

原.filebrowser.json 定义了端口号、地址、日志、数据库、文件根目录等信息

1
2
3
4
5
6
7
8
{
  "port": 80,
  "baseURL": "",
  "address": "",
  "log": "stdout",
  "database": "/database.db",
  "root": "/srv"
}

我们修改后的config.json文件,就是这么简单

1
2
3
4
5
6
7
8
{
  "port": 80,
  "baseURL": "",
  "address": "",
  "log": "stdout",
  "database": "/db/database.db",
  "root": "/srv"
}

如果你按照我的配置在上面pv部分定义的path为/mnt/filebrowser/db那么请创建该目录,并将config.json和database.db文件放入该目录中,database.db可以是无内容的空文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
apiVersion: apps/v1
kind: Deployment
metadata:
  name: filebrowser-deployment
  namespace: filebrowser
spec:
  replicas: 1
  selector:
    matchLabels:
      app: filebrowser
  template:
    metadata:
      labels:
        app: filebrowser
    spec:
      nodeSelector:
        filebrowser: enable             #注意此次label命令参考kubectl label nodes k8s-master1 filebrowser=enable
      containers:
        - name: filebrowser
          image: filebrowser/filebrowser
          command: ["./filebrowser"]
          args: ["--config", "/db/config.json"]
          imagePullPolicy: Always
          ports:
            - containerPort: 80
          env:
            - name: "FB_CONFIG"
              value: "/db/config.json"
            - name: "FB_DATABASE"
              value: "/db/database.db"
          volumeMounts:
            - name: srv-volume
              mountPath: /srv
            - name: db-volume
              mountPath: /db
      volumes:
        - name: srv-volume
          persistentVolumeClaim:
            claimName: filebrowser-pvc
        - name: db-volume
          persistentVolumeClaim:
            claimName: filebrowser-db-pvc
---

1.6 Service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
apiVersion: v1
kind: Service
metadata:
  name: filebrowser
  namespace: filebrowser
spec:
  selector:
    app: filebrowser
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: NodePort
---

1.7 Ingress

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 20m
  name: mirrors.gucat.vip
  namespace: filebrowser
spec:
  ingressClassName: nginx
  rules:
    - host: mirrors.gucat.vip
      http:
        paths:
          - backend:
              service:
                name: filebrowser
                port:
                  number: 80
            path: /
            pathType: Prefix
---

2. 架构及技术总结

2.1 总结

  1. filebrowser默认的配置文件和数据库都在根目录,而是是单一文件,无法直接在k8s挂载
  2. 我们需要自定义 config.json文件,将其创建在非根目录路径,然后让其生效替换掉.filebrowser.json的配置,最后将config.json和config.json中指定的数据库文件filebrowser.db一同挂载出来就能实现持久化了

2.2 其他说明

上传文件你可能会发现无法上传很大的文件,可能是nginx的限制,你可以在ingress中添加nginx.ingress.kubernetes.io/proxy-body-size: 20m来解决
如果是nginx的话,你可以在nginx.conf中添加client_max_body_size 20m;来解决

2.3 自定义logo和去除版权信息

设置->全局设置->品牌->品牌信息文件夹路径->/srv/style->保存
创建style目录,style->img->icons 创建这样的目录结构,style里创建img,然后img里创建icons目录
style里存放custom.css文件,img里存放logo.svg文件,icons里存放favicon.ico文件favicon-32x32.png和favicon-16x16.png文件
custom.css文件添加如下内容

1
2
3
.credits {
    display: none
}

2.4 完整版yaml文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
apiVersion: v1
kind: Namespace
metadata:
  name: filebrowser

#需要创建挂载目录
#当前配置/mnt/filebrowser/file   /mnt/filebrowser/db
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: filebrowser-storage
provisioner: kubernetes.io/no-provisioner
parameters:
  type: pd-ssd # 存储参数，这里针对 SSD 存储设备进行优化
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: filebrowser-pv
  namespace: filebrowser
spec:
  storageClassName: filebrowser-storage
  claimRef:
    name: filebrowser-pvc
    namespace: filebrowser
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteOnce
  local:
    path: /mnt/filebrowser/file
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-master1          #节点选择
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: filebrowser-pvc
  namespace: filebrowser
spec:
  storageClassName: filebrowser-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 15Gi
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: filebrowser-db-pv
  namespace: filebrowser
spec:
  storageClassName: filebrowser-storage
  claimRef:
    name: filebrowser-db-pvc
    namespace: filebrowser
  capacity:
    storage: 200Mi
  accessModes:
    - ReadWriteOnce
  local:
    path: /mnt/filebrowser/db
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-master1                 #同上节点选择
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: filebrowser-db-pvc
  namespace: filebrowser
spec:
  storageClassName: filebrowser-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 150Mi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: filebrowser-deployment
  namespace: filebrowser
spec:
  replicas: 1
  selector:
    matchLabels:
      app: filebrowser
  template:
    metadata:
      labels:
        app: filebrowser
    spec:
      nodeSelector:
        filebrowser: enable             #注意此次label命令参考kubectl label nodes k8s-master1 filebrowser=enable
      containers:
        - name: filebrowser
          image: filebrowser/filebrowser
          command: ["./filebrowser"]
          args: ["--config", "/db/config.json"]
          imagePullPolicy: Always
          ports:
            - containerPort: 80
          env:
            - name: "FB_CONFIG"
              value: "/db/config.json"
            - name: "FB_DATABASE"
              value: "/db/database.db"
          volumeMounts:
            - name: srv-volume
              mountPath: /srv
            - name: db-volume
              mountPath: /db
      volumes:
        - name: srv-volume
          persistentVolumeClaim:
            claimName: filebrowser-pvc
        - name: db-volume
          persistentVolumeClaim:
            claimName: filebrowser-db-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: filebrowser
  namespace: filebrowser
spec:
  selector:
    app: filebrowser
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: NodePort

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 20m
  name: mirrors.gucat.vip
  namespace: filebrowser
spec:
  ingressClassName: nginx
  rules:
    - host: mirrors.gucat.vip
      http:
        paths:
          - backend:
              service:
                name: filebrowser
                port:
                  number: 80
            path: /
            pathType: Prefix
status:
  loadBalancer: {}