By

Contents
  1. 1. 构建nodejs镜像
  2. 2. 构建Redis基础镜像
  3. 3. 构建Redis主镜像
  4. 4. 构建Redis副本镜像
  5. 5. 构建Redis后端集群
  6. 6. 创建Node容器
  7. 7. 捕获应用日志
  8. 8. 遇到的问题

本篇将构建如下镜像来支持部署多容器的应用

  • Node容器，用来服务于Node应用，该容器会处于express网络下，链接到Redis容器
  • Redis容器，用来保存和集群化应用状态，该容器会处于express网络下，链接到两个Redis副本容器，用于集群化应用状态
  • 日志容器，用于捕获应用日志

构建nodejs镜像

创建Node.js Dockerfile

1
2
3
4
5
6
7
8
mkdir nodejs
cd nodejs
mkdir -p nodeapp
cd nodeapp
wget https://raw.githubusercontent.com/jamtur01/dockerbook-code/master/code/6/node/nodejs/nodeapp/package.json
wget https://raw.githubusercontent.com/jamtur01/dockerbook-code/master/code/6/node/nodejs/nodeapp/server.js
cd ..
vim Dockerfile

Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
FROM ubuntu:14.04
MAINTAINER flag0 <root@flag0.com>
ENV REFRESHED_AT 2019-09-10

RUN apt-get -yqq update
RUN apt-get -y install npm wget curl 
RUN ln -s /usr/bin/nodejs /usr/bin/node
RUN mkdir -p /var/log/nodeapp

ADD nodeapp /opt/nodeapp/
WORKDIR /opt/nodeapp

RUN npm config set strict-ssl false
RUN npm install
RUN npm install -g n
RUN n stable

VOLUME [ "/var/log/nodeapp" ]
EXPOSE 3000

ENTRYPOINT [ "node","server.js" ]

默认安装的nodejs版本过老，无法支持server.js中的语法，所以我们这里使用n stable来安装最新版的nodejs

其中npm install用来安装server.js运行所需的模块

构建Node.js镜像

1
docker build -t flag0/nodejs .

构建Redis基础镜像

Redis主镜像和副本镜像都是依照基础镜像来创建的。

创建Redis基础镜像 Dockerfile

1
2
3
mkdir redis_base
cd redis_base
vim Dockerfile

Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
FROM ubuntu:14.04
MAINTAINER flag0 root@flag0.com
ENV REFRESHED_AT 2019-9-1

RUN apt-get -y update
RUN apt-get install -y software-properties-common python-software-properties
RUN add-apt-repository ppa:chris-lea/redis-server
RUN apt-get -y update
RUN apt-get -y install redis-server redis-tools

VOLUME ["/var/lib/redis","/var/log/redis/"]
EXPOSE 6379
CMD []

该Redis基础镜像安装了最新版本的Redis(从PPA库中进行安装，而不是使用ubuntu自带的较老的Redis包)

构建Redis 基础镜像

1
docker build -t flag0/redis .

构建Redis主镜像

创建Redis主服务的Dockerfile

1
2
3
mkdir redis_primary
cd redis_primary
vim Dockerfile

Redis主镜像

1
2
3
4
5
FROM flag0/redis
MAINTAINER flag0 <root@flag0.com>
ENV REFRESHED_AT 2019-09-10

ENTRYPOINT ["redis-server","--logfile /var/log/redis/redis-server.log"]

构建Redis主镜像

1
docker build -t flag0/redis_primary .

构建Redis副本镜像

创建Redis副本镜像的Dockerfile

1
2
3
mkdir redis_replica
cd redis_replica
vim Dockerfile

Redis副本镜像

1
2
3
4
5
FROM flag0/redis
MAINTAINER flag0 <root@flag0.com>
ENV REFRESHED_AT 2019-09-10

ENVTRYPOINT ["redis-server","--logfile /var/log/redis/redis-replica.log","--slaveof redis_primary 6379"]

构建Redis副本镜像

1
docker build -t flag0/redis_replica .

构建Redis后端集群

创建express网络

1
docker network create express

运行Redis容器

1
docker run -d -h redis_primary --net express --name redis_priary flag0/redis_primary
  • -h 设置容器的主机名，使用其可以确保容器使用指定的主机名，并被本地的DNS服务正常解析

读取Redis主日志

Redis服务会将日志记录到一个文件而不是记录到标准输出

1
2
3
4
5
6
7
8
9
10
11
docker run -it --rm --volumes-from redis_primary ubuntu cat /var/log/redis/redis-server.log

1:C 10 Sep 2019 04:24:53.310 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 10 Sep 2019 04:24:53.310 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 10 Sep 2019 04:24:53.310 # Configuration loaded
1:M 10 Sep 2019 04:24:53.313 * Running mode=standalone, port=6379.
1:M 10 Sep 2019 04:24:53.313 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 10 Sep 2019 04:24:53.313 # Server initialized
1:M 10 Sep 2019 04:24:53.313 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 10 Sep 2019 04:24:53.313 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 10 Sep 2019 04:24:53.313 * Ready to accept connections
  • --rm 运行结束后自动删除容器
  • --volume-from redis_primary将redis_primary中的卷映射到新的redis日志容器中

创建Redis副本容器

1
docker run -d -h redis_replical --name redis_replial --net express flag0/redis_replica

查看Redis副本容器服务器日志

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
docker run -it --rm --volumes-from redis_replical ubuntu cat /var/log/redis/redis-replica.log

1:C 10 Sep 2019 14:43:15.177 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 10 Sep 2019 14:43:15.177 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 10 Sep 2019 14:43:15.177 # Configuration loaded
1:S 10 Sep 2019 14:43:15.179 * Running mode=standalone, port=6379.
1:S 10 Sep 2019 14:43:15.179 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:S 10 Sep 2019 14:43:15.179 # Server initialized
1:S 10 Sep 2019 14:43:15.179 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:S 10 Sep 2019 14:43:15.179 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:S 10 Sep 2019 14:43:15.180 * Ready to accept connections
1:S 10 Sep 2019 14:43:15.180 * Connecting to MASTER redis_primary:6379
1:S 10 Sep 2019 14:43:15.190 * MASTER <-> REPLICA sync started
1:S 10 Sep 2019 14:43:15.190 * Non blocking connect for SYNC fired the event.
1:S 10 Sep 2019 14:43:15.191 * Master replied to PING, replication can continue...
1:S 10 Sep 2019 14:43:15.191 * Partial resynchronization not possible (no cached master)
1:S 10 Sep 2019 14:43:15.193 * Full resync from master: cd1a91d73c03c48a040716bfa98941b9cf31a2cb:406
1:S 10 Sep 2019 14:43:15.272 * MASTER <-> REPLICA sync: receiving 176 bytes from master
1:S 10 Sep 2019 14:43:15.272 * MASTER <-> REPLICA sync: Flushing old data
1:S 10 Sep 2019 14:43:15.272 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 10 Sep 2019 14:43:15.273 * MASTER <-> REPLICA sync: Finished with success

运行第二个Redis副本容器

为了确保万无一失我们加入另一个副本容器redis_replica2

1
docker run -d -h redis_replica2 --name redis_replica2 --net express flag0/redis_replica

第二个Redis副本容器的日志

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
docker run -it --rm --volumes-from redis_replica2 ubuntu cat /var/log/redis/redis-replica.log

1:S 10 Sep 2019 02:21:42.516 * Connecting to MASTER redis_primary:6379
1:S 10 Sep 2019 02:21:42.517 * MASTER <-> REPLICA sync started
1:S 10 Sep 2019 02:21:42.517 * Non blocking connect for SYNC fired the event.
1:S 10 Sep 2019 02:21:42.517 # Error condition on socket for SYNC: Connection reset by peer
1:S 10 Sep 2019 02:21:43.519 * Connecting to MASTER redis_primary:6379
1:S 10 Sep 2019 02:21:43.520 * MASTER <-> REPLICA sync started
1:S 10 Sep 2019 02:21:43.520 * Non blocking connect for SYNC fired the event.
1:S 10 Sep 2019 02:21:43.520 # Error condition on socket for SYNC: Connection reset by peer
1:S 10 Sep 2019 02:21:44.522 * Connecting to MASTER redis_primary:6379
1:S 10 Sep 2019 02:21:44.522 * MASTER <-> REPLICA sync started
1:S 10 Sep 2019 02:21:44.523 * Non blocking connect for SYNC fired the event.
1:S 10 Sep 2019 02:21:44.523 # Error condition on socket for SYNC: Connection reset by peer
1:S 10 Sep 2019 02:21:45.524 * Connecting to MASTER redis_primary:6379
1:S 10 Sep 2019 02:21:45.525 * MASTER <-> REPLICA sync started
1:S 10 Sep 2019 02:21:45.525 * Non blocking connect for SYNC fired the event.
1:S 10 Sep 2019 02:21:45.525 # Error condition on socket for SYNC: Connection reset by peer
1:S 10 Sep 2019 02:21:46.528 * Connecting to MASTER redis_primary:6379
1:S 10 Sep 2019 02:21:46.529 * MASTER <-> REPLICA sync started
1:S 10 Sep 2019 02:21:46.529 * Non blocking connect for SYNC fired the event.
1:S 10 Sep 2019 02:21:46.529 # Error condition on socket for SYNC: Connection reset by peer
1:S 10 Sep 2019 02:21:47.531 * Connecting to MASTER redis_primary:6379
1:S 10 Sep 2019 02:21:47.533 * MASTER <-> REPLICA sync started
1:S 10 Sep 2019 02:21:47.533 * Non blocking connect for SYNC fired the event.
1:S 10 Sep 2019 02:21:47.533 # Error condition on socket for SYNC: Connection reset by peer
1:S 10 Sep 2019 02:21:48.555 * Connecting to MASTER redis_primary:6379
1:S 10 Sep 2019 02:21:48.563 * MASTER <-> REPLICA sync started
1:S 10 Sep 2019 02:21:48.563 * Non blocking connect for SYNC fired the event.
1:S 10 Sep 2019 02:21:48.564 # Error condition on socket for SYNC: Connection reset by peer

创建Node容器

运行Node.js容器

1
docker run -d --name nodeapp -p 3000:3000 --net express flag0/nodejs

查看Nodeapp容器的日志

1
2
3
docker logs nodeapp

Listening on port 3000

查看相应网页

1
2
3
curl http://127.0.0.1:3000

{"status":"ok"}

正常启动

捕获应用日志

在生产环境中需要确保可以捕获日志并将日志保存到日志服务器，使用Logstash来完成

创建Logstash的Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
FROM ubuntu:14.04
MAINTAINER flag0 <root@flag.com>
ENV FEFRESHED_AT 2019-09-10

RUN apt-get -y update
RUN apt-get -y install wget
RUN wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | apt-key add -
RUN echo 'deb http://packages.elasticsearch.org/logstash/1.4/debian stable main' | sudo tee /etc/apt/sources.list.d/logstash.list
RUN apt-get -y update
RUN apt-get -y install logstash

ADD logstash.conf /etc/
WORKDIR /opt/logstash
ENTRYPOINT ["bin/logstash"]
CMD ["--config=/etc/logstash.conf"]

Dockerfile创建了镜像并安装了Logstash，将logstash.conf文件使用ADD指令添加到etc目录

Logstash配置文件

1
2
3
4
5
6
7
8
9
10
11
input{
    file{
        type => "syslog"
        path =>["/var/log/nodeapp/nodeapp.log","/var/log/redis/redis-server.log"]
    }
}
output{
    stdout{
        codec => rubydebug
    }
}

该配置文件功能为

input其监控两个文件，即/var/log/nodeapp/nodeapp.log和/var/log/redis/redis-server.logLogstash会一直监控这两个文件，将更新的内容发送给Logstash

output部分，接收所有的Logstash输入的内容并将其输出到标准输出上。实际项目中，一般会将Logstash配置为输出到Elasticsearch集群或者其他目的地。

构建Logstash镜像

1
docker build -t flag0/logstash .

启动Logstash容器

1
docker run -d --name logstash --volumes-from redis_primary --volumes-from nodeapp flag0/logstash

查看logstash容器的日志

1
2
docker logs -f logstash
{:timestamp=>"2019-09-11T01:20:36.029000+0000", :message=>"Using milestone 2 input plugin 'file'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.4.5/plugin-milestones", :level=>:warn}

在浏览器中刷新Web应用，产生一个新的日志事件，能在logstash容器的输出中看到这个事件。

Logstash的中的Node事件

1
2
3
4
5
6
7
8
{
       "message" => "::ffff:112.54.84.218 - - [11/Sep/2019:01:22:30 +0000] \"GET / HTTP/1.1\" 200 15 \"-\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763\"",
      "@version" => "1",
    "@timestamp" => "2019-09-11T01:22:30.755Z",
          "type" => "syslog",
          "host" => "e805867dff5d",
          "path" => "/var/log/nodeapp/nodeapp.log"
}

构建成功

遇到的问题

node版本过低，运行node service.js报出语法错误

1568165743488

解决办法：

1
2
RUN npm install -g n
RUN n stable

使用npm安装n模块进行更新

n stable会安装最新版本的nodejs

Contents
  1. 1. 构建nodejs镜像
  2. 2. 构建Redis基础镜像
  3. 3. 构建Redis主镜像
  4. 4. 构建Redis副本镜像
  5. 5. 构建Redis后端集群
  6. 6. 创建Node容器
  7. 7. 捕获应用日志
  8. 8. 遇到的问题