Docker部署MySQL8主从模式

Docker部署MySQL8主从模式

文章目录


Mysql 8.1.0
Docker 24.0.5

关于主从模式,Mysql8.0一些版本开始,有许多变化,这里使用8.1.0

一、运行容器

新建两个MySQL文件夹,分别新建data文件夹和conf/my.cnf文件。
根据需要理解并更改以下脚本。

language-bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#!/bin/bash

containerName="MyWeb02-MySQL"
MySQLData="/root/MyWeb02/MySQL/data"
MySQLConf="/root/MyWeb02/MySQL/conf/my.cnf"

containerSlaveName="MyWeb02-MySQL-Slave"
MySQLSlaveData="/root/MyWeb02/MySQL-Slave/data"
MySQLSlaveConf="/root/MyWeb02/MySQL-Slave/conf/my.cnf"

docker run -d --name "$containerName" \
-p 3307:3306 \
-v "$MySQLData":/var/lib/mysql \
-v "$MySQLConf":/etc/mysql/my.cnf \
-e MYSQL_ROOT_PASSWORD=20028888 \
mysql:8

docker run -d --name "$containerSlaveName" \
-p 3308:3306 \
-v "$MySQLSlaveData":/var/lib/mysql \
-v "$MySQLSlaveConf":/etc/mysql/my.cnf \
-e MYSQL_ROOT_PASSWORD=20028888 \
mysql:8

主节点的my.cnf容器如下

language-bash
1
2
3
[mysqld]
server-id=1
log_bin=mysql-bin

从节点的my.cnf容器如下

language-bash
1
2
[mysqld]
server-id=2

运行脚本。

二、配置主从

到主节点命令行,运行以下命令

language-sql
1
2
CREATE USER 'replica'@'%' IDENTIFIED WITH mysql_native_password BY 'replica';
GRANT REPLICATION SLAVE ON *.* TO 'replica'@'%';

到从节点命令行,运行以下命令

language-sql
1
2
3
4
5
6
7
CHANGE REPLICATION SOURCE TO 
SOURCE_HOST='172.17.0.6',
SOURCE_PORT=3306,
SOURCE_USER='replica',
SOURCE_PASSWORD='replica';
START REPLICA; //开启备份
SHOW REPLICA STATUS\G //查看主从情况

其中SOURCE_HOST为主节点容器的ip
查看主从情况时,主要注意下面两个字段是否为Yes。不是的话,就有问题,读docker logs然后去解决它。

language-bash
1
2
Slave_IO_Running: Yes
Slave_SQL_Running: Yes

Navicat等第三方软件可能不支持\G,结果以行显示。

三、测试效果

在主节点新建一个数据库

language-bash
1
create database `test`;

随后可以在从节点也看到效果。

Docker部署xxl-job调度器并结合SpringBoot测试

Docker部署xxl-job调度器并结合SpringBoot测试

一、Docker部署

1. 创建数据库

去Github下载最新发布的源码,https://github.com/xuxueli/xxl-job/releases,找到/xxl-job/doc/db/tables_xxl_job.sql文件,对数据库进行执行即可,脚本里面包含数据库的创建。

2. 启动容器

参考官方中文文档,写出如下docker-compose示例。使用-e PARAMS: ""来指定一些变量,包括数据库信息,一般需要根据自身情况修改。

language-yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
version: "3.8"
networks:
docker_xuecheng:
ipam:
config:
- subnet: 172.20.0.0/16

services:
xxl-job:
container_name: xxl-job
image: xuxueli/xxl-job-admin:2.4.0
volumes:
- ./xxl_job/logs:/data/applogs
ports:
- "8088:8080"
environment:
PARAMS: '
--spring.datasource.url=jdbc:mysql://172.20.0.2:3306/xxl_job?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=Asia/Shanghai
--spring.datasource.username=root
--spring.datasource.password=1009'
networks:
docker_xuecheng:
ipv4_address: 172.20.3.1

3. 访问

访问http://192.168.101.65:8088/xxl-job-admin/即可。

4. 新建执行器

新增一个简单的testHandler执行器。

二、SpringBoot整合

1. 模块注册到执行器

在对应模块引入依赖

language-xml
1
2
3
4
<dependency>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job-core</artifactId>
</dependency>

并指定执行器的appname

language-yml
1
2
3
4
5
6
7
8
9
10
11
12
xxl:
job:
admin:
addresses: http://192.168.101.65:8088/xxl-job-admin
executor:
appname: testHandler
address:
ip:
port: 9999
logpath: /data/applogs/xxl-job/jobhandler
logretentiondays: 30
accessToken: default_token

2. 创建配置类

在源码中找到src/main/java/com/xxl/job/executor/core/config/XxlJobConfig.java,复制到模块代码中。如下

language-java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
package com.xuecheng.media.config;

import com.xxl.job.core.executor.impl.XxlJobSpringExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
* xxl-job config
*
* @author xuxueli 2017-04-28
*/
@Configuration
public class XxlJobConfig {

private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);

@Value("${xxl.job.admin.addresses}")
private String adminAddresses;

@Value("${xxl.job.accessToken}")
private String accessToken;

@Value("${xxl.job.executor.appname}")
private String appname;

@Value("${xxl.job.executor.address}")
private String address;

@Value("${xxl.job.executor.ip}")
private String ip;

@Value("${xxl.job.executor.port}")
private int port;

@Value("${xxl.job.executor.logpath}")
private String logPath;

@Value("${xxl.job.executor.logretentiondays}")
private int logRetentionDays;


@Bean
public XxlJobSpringExecutor xxlJobExecutor() {

logger.info(">>>>>>>>>>> xxl-job config init.");
XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
xxlJobSpringExecutor.setAppname(appname);
xxlJobSpringExecutor.setAddress(address);
xxlJobSpringExecutor.setIp(ip);
xxlJobSpringExecutor.setPort(port);
xxlJobSpringExecutor.setAccessToken(accessToken);
xxlJobSpringExecutor.setLogPath(logPath);
xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);

return xxlJobSpringExecutor;
}

/**
* 针对多网卡、容器内部署等情况,可借助 "spring-cloud-commons" 提供的 "InetUtils" 组件灵活定制注册IP;
*
* 1、引入依赖:
* <dependency>
* <groupId>org.springframework.cloud</groupId>
* <artifactId>spring-cloud-commons</artifactId>
* <version>${version}</version>
* </dependency>
*
* 2、配置文件,或者容器启动变量
* spring.cloud.inetutils.preferred-networks: 'xxx.xxx.xxx.'
*
* 3、获取IP
* String ip_ = inetUtils.findFirstNonLoopbackHostInfo().getIpAddress();
*/


}

3. 启动测试

重启模块,访问XXL-JOB网页端,查看情况。如果执行器的OnLine 机器地址有一个信息,表示模块绑定成功。

三、任务发布-普通任务

1. 编写任务代码

源代码中有任务代码示例,路径为src/main/java/com/xxl/job/executor/service/jobhandler/SampleXxlJob.java,仿照写一个简单的任务,如下。

language-java
1
2
3
4
5
6
7
8
9
10
11
12
@Component
public class SampleXxlJob {

/**
* 1、简单任务示例(Bean模式)
*/
@XxlJob("demoJobHandler")
public void demoJobHandler() throws Exception {

System.out.println("处理视频");
}
}

2. 创建任务

选择执行器,并指定JobHandler

3. 启动任务

启动刚才创建的任务

对应模块的日志可以看到每10秒打印一次输出。

XXL-JOB网页管理也可以看到相关任务执行记录。

四、任务发布-分片任务

1. 编写任务代码

language-java
1
2
3
4
5
6
7
8
9
10
@XxlJob("shardingJobHandler")
public void shardingJobHandler() throws Exception {

// 分片参数
int shardIndex = XxlJobHelper.getShardIndex();
int shardTotal = XxlJobHelper.getShardTotal();

System.out.println("分片参数:当前分片序号 = " + shardIndex + ", 总分片数 = " + shardTotal);

}

2. 启动多个实例

添加虚拟机参数-Dserver.port=63051 -Dxxl.job.executor.port=9998,前者区分程序端口,后者区分执行器端口。

3. 创建任务

创建任务之前,检查一下两个模块是否注册到指定执行器。

随后创建任务,指定执行器JobHandler,同时路由策略选择分片广播

4. 启动任务

启动任务后,观察两个模块的日志。

同时任务记录也在XXL-JOB管理网页中可以查询到。

五、动态扩容

当运行分片任务时,又添加一个新的模块示例,此时分片任务会自动扩容再分配。如图,我们再复制一个运行配置。

然后将其运行,等待一会,执行器可以看到有3个绑定的机器。

新增的运行实例日志如下,

同时,先前两个运行实例的日志发送了变化,如下

参考资料

Docker多节点部署Minio分布式文件系统并测试

Docker多节点部署Minio分布式文件系统并测试

一、前提准备

准备如下文件夹和文件

language-txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
./
├── docker-compose-minio.yml
├── .env
├── env
│ ├── minio.env
├── minio
│ ├── minio1
│ │ ├── data1
│ │ └── data2
│ ├── minio2
│ │ ├── data1
│ │ └── data2
│ ├── minio3
│ │ ├── data1
│ │ └── data2
│ └── minio4
│ ├── data1
│ └── data2

二、文件配置

1. .env

language-env
1
MINIO_VERSION=RELEASE.2024-01-29T03-56-32Z

2. env/minio.env

language-env
1
2
MINIO_ROOT_USER=minio
MINIO_ROOT_PASSWORD=minio123

3. docker-compose-minio.yml

language-yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
version: "3.8"
networks:
docker_xuecheng:
ipam:
config:
- subnet: 172.20.0.0/16

services:
minio1:
container_name: minio1
image: minio/minio:${
MINIO_VERSION}
volumes:
- ./minio/minio1/data1:/data1
- ./minio/minio1/data2:/data2
ports:
- "9001:9000"
- "9011:9001"
env_file:
- ./env/minio.env
command: server --address ":9000" --console-address ":9001" http://172.20.2.{
1...4}/data{
1...2}
networks:
docker_xuecheng:
ipv4_address: 172.20.2.1

minio2:
container_name: minio2
image: minio/minio:${
MINIO_VERSION}
volumes:
- ./minio/minio2/data1:/data1
- ./minio/minio2/data2:/data2
ports:
- "9002:9000"
- "9012:9001"
env_file:
- ./env/minio.env
command: server --address ":9000" --console-address ":9001" http://172.20.2.{
1...4}/data{
1...2}
networks:
docker_xuecheng:
ipv4_address: 172.20.2.2

minio3:
container_name: minio3
image: minio/minio:${
MINIO_VERSION}
volumes:
- ./minio/minio3/data1:/data1
- ./minio/minio3/data2:/data2
ports:
- "9003:9000"
- "9013:9001"
env_file:
- ./env/minio.env
command: server --address ":9000" --console-address ":9001" http://172.20.2.{
1...4}/data{
1...2}
networks:
docker_xuecheng:
ipv4_address: 172.20.2.3

minio4:
container_name: minio4
image: minio/minio:${
MINIO_VERSION}
volumes:
- ./minio/minio4/data1:/data1
- ./minio/minio4/data2:/data2
ports:
- "9004:9000"
- "9014:9001"
env_file:
- ./env/minio.env
command: server --address ":9000" --console-address ":9001" http://172.20.2.{
1...4}/data{
1...2}
networks:
docker_xuecheng:
ipv4_address: 172.20.2.4

三、测试

访问宿主机ip:9011,输入账号密码。

language-txt
1
2
MINIO_ROOT_USER=minio
MINIO_ROOT_PASSWORD=minio123

点到Monitoring -> Metrics

四、Java测试

1. 引入依赖

language-xml
1
2
3
4
5
6
7
8
9
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>

2. 增删改

在这之前先去网页端,创建一个Bucket

language-java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
package com.xuecheng.media;

import io.minio.*;
import io.minio.errors.*;
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.compress.utils.IOUtils;
import org.junit.jupiter.api.Test;

import java.io.*;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;

public class MinioTest {

private MinioClient minioClient = MinioClient.builder()
.endpoint("http://192.168.101.65:9001") //改成你的宿主机ip
.credentials("minio", "minio123")
.build();

@Test
public void testCreate() throws IOException, ServerException, InsufficientDataException, ErrorResponseException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {

ObjectWriteResponse file = minioClient.uploadObject(
UploadObjectArgs.builder()
.bucket("test")
.filename("C:\\Users\\mumu\\Desktop\\1C6091EF9671978A9F1B6C6F8A3666FD.png")
.object("1.png")
.build()
);
}

@Test
public void testDelete() throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {

minioClient.removeObject(
RemoveObjectArgs.builder()
.bucket("test")
.object("12.msi")
.build()
);
}

@Test
public void testGet() throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {

InputStream inputStream = minioClient.getObject(
GetObjectArgs.builder()
.bucket("test")
.object("1.png")
.build()
);
FileOutputStream outputStream = new FileOutputStream(new File("C:\\Users\\mumu\\Desktop\\2.png"));
IOUtils.copy(inputStream, outputStream);
}

}

Docker-Compose部署Redis(v7.2)分片集群(含主从)

Docker-Compose部署Redis(v7.2)分片集群(含主从)

环境

  • docker desktop for windows 4.23.0
  • redis 7.2

目标

搭建如下图分片+主从集群。

一、前提准备

1. 文件夹结构

因为Redis 7.2 docker镜像里面没有配置文件,所以需要去redis官网下载一个复制里面的redis.conf
博主这里用的是7.2.3版本的redis.conf,这个文件就在解压后第一层文件夹里。

然后构建如下文件夹结构。

language-txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
sharding/
├── docker-compose.yaml
├── master1
│ └── conf
│ └── redis.conf
├── master2
│ └── conf
│ └── redis.conf
├── master3
│ └── conf
│ └── redis.conf
├── replica1
│ └── conf
│ └── redis.conf
├── replica2
│ └── conf
│ └── redis.conf
└── replica3
└── conf
└── redis.conf

二、配置文件

1. redis.conf

对每个redis.conf都做以下修改。分片集群的redis主从的redis.conf目前都是一样的。

language-bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
port 6379
# 开启集群功能
cluster-enabled yes
# 集群的配置文件名称,不需要我们创建,由redis自己维护
cluster-config-file /data/nodes.conf
# 节点心跳失败的超时时间
cluster-node-timeout 5000
# 持久化文件存放目录
dir /data
# 绑定地址
bind 0.0.0.0
# 让redis后台运行
daemonize no
# 保护模式
protected-mode no
# 数据库数量
databases 1
# 日志
logfile /data/run.log

2. docker-compose文件

language-yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
version: '3.8'

networks:
redis-sharding:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.30.2.0/24

services:
master1:
container_name: master1
image: redis:7.2
volumes:
- ./master1/conf:/usr/local/etc/redis
ports:
- "7001:6379"
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
networks:
redis-sharding:
ipv4_address: 172.30.2.11

master2:
container_name: master2
image: redis:7.2
volumes:
- ./master2/conf:/usr/local/etc/redis
ports:
- "7002:6379"
command: [ "redis-server", "/usr/local/etc/redis/redis.conf" ]
networks:
redis-sharding:
ipv4_address: 172.30.2.12

master3:
container_name: master3
image: redis:7.2
volumes:
- ./master3/conf:/usr/local/etc/redis
ports:
- "7003:6379"
command: [ "redis-server", "/usr/local/etc/redis/redis.conf" ]
networks:
redis-sharding:
ipv4_address: 172.30.2.13

replica1:
container_name: replica1
image: redis:7.2
volumes:
- ./replica1/conf:/usr/local/etc/redis
ports:
- "8001:6379"
command: [ "redis-server", "/usr/local/etc/redis/redis.conf" ]
networks:
redis-sharding:
ipv4_address: 172.30.2.21

replica2:
container_name: replica2
image: redis:7.2
volumes:
- ./replica2/conf:/usr/local/etc/redis
ports:
- "8002:6379"
command: [ "redis-server", "/usr/local/etc/redis/redis.conf" ]
networks:
redis-sharding:
ipv4_address: 172.30.2.22

replica3:
container_name: replica3
image: redis:7.2
volumes:
- ./replica3/conf:/usr/local/etc/redis
ports:
- "8003:6379"
command: [ "redis-server", "/usr/local/etc/redis/redis.conf" ]
networks:
redis-sharding:
ipv4_address: 172.30.2.23


需要注意以下几点

  • 这里自定义了bridge子网并限定了范围,如果该范围已经被使用,请更换。
  • 这里没有对data进行-v挂载,如果要挂载,请注意宿主机对应文件夹权限问题。

随后运行

language-bash
1
docker-compose -p redis-sharding up -d

三、构建集群

接下来所有命令都在master1容器的命令行执行

1. 自动分配主从关系

这个命令会创建了一个集群,包括三个主节点和三个从节点,每个主节点分配一个从节点作为副本,前3个ip为主节点,后3个为从节点,主节点的从节点随机分配。

language-bash
1
redis-cli --cluster create 172.30.2.11:6379 172.30.2.12:6379 172.30.2.13:6379 172.30.2.21:6379 172.30.2.22:6379 172.30.2.23:6379 --cluster-replicas 1

如果希望手动指定主从关系,看下面,否则你可以跳过这一章节了。

2.1 构建3 master集群

language-bash
1
redis-cli --cluster create 172.30.2.11:6379 172.30.2.12:6379 172.30.2.13:6379 --cluster-replicas 0

2.2 手动配置从节点

查看3个主节点的ID

language-bash
1
redis-cli -h 172.30.2.11 -p 6379 cluster nodes

下面3个命令会将3个从节点加入集群中,其中172.30.2.11可以是三个主节点的任意一个。

language-bash
1
2
3
redis-cli -h 172.30.2.21 -p 6379 cluster meet 172.30.2.11 6379
redis-cli -h 172.30.2.22 -p 6379 cluster meet 172.30.2.11 6379
redis-cli -h 172.30.2.23 -p 6379 cluster meet 172.30.2.11 6379

然后为每个从节点指定主节点。

language-bash
1
2
3
redis-cli -h 172.30.2.21 -p 6379 cluster replicate <master-ID>
redis-cli -h 172.30.2.22 -p 6379 cluster replicate <master-ID>
redis-cli -h 172.30.2.23 -p 6379 cluster replicate <master-ID>

四、测试

1. 集群结构

可以通过以下命令查看集群中每个节点的id、角色、ip、port、插槽范围等信息

language-bash
1
redis-cli -h 172.30.2.11 -p 6379 cluster nodes

2. 分片测试

往集群存入4个键值

language-bash
1
2
3
4
redis-cli -c -h 172.30.2.11 -p 6379 set key1 value1
redis-cli -c -h 172.30.2.11 -p 6379 set key2 value2
redis-cli -c -h 172.30.2.11 -p 6379 set key3 value3
redis-cli -c -h 172.30.2.11 -p 6379 set key4 value4

查看每个主节点现有的键值,会发现每个节点只有一部分键值。

language-bash
1
2
3
redis-cli -h 172.30.2.11 -p 6379 --scan
redis-cli -h 172.30.2.12 -p 6379 --scan
redis-cli -h 172.30.2.13 -p 6379 --scan
Docker-Compose部署Redis(v7.2)主从模式

Docker-Compose部署Redis(v7.2)主从模式

环境

  • docker desktop for windows 4.23.0
  • redis 7.2

一、前提准备

1. redis配置文件

因为Redis 7.2 docker镜像里面没有配置文件,所以需要去redis官网下载一个复制里面的redis.conf
博主这里用的是7.2.3版本的redis.conf,这个文件就在解压后第一层文件夹里。

2. 下载redis镜像

language-bash
1
docker pull redis:7.2

3. 文件夹结构

如下建立cluster文件夹,并复制出三份conf文件到如图位置。

二、docker-compose

docker-compose文件具体内容如下。

language-yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
version: '3.8'

networks:
redis-network:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.30.1.0/24

services:
redis-master:
container_name: redis-master
image: redis:7.2
volumes:
- ./master/redis.conf:/usr/local/etc/redis/redis.conf
# - ./master/data:/data
ports:
- "7001:6379"
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
networks:
redis-network:
ipv4_address: 172.30.1.2

redis-replica1:
container_name: redis-replica1
image: redis:7.2
volumes:
- ./replica1/redis.conf:/usr/local/etc/redis/redis.conf
# - ./replica1/data:/data
ports:
- "7002:6379"
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
depends_on:
- redis-master
networks:
redis-network:
ipv4_address: 172.30.1.3

redis-replica2:
container_name: redis-replica2
image: redis:7.2
volumes:
- ./replica2/redis.conf:/usr/local/etc/redis/redis.conf
# - ./replica2/data:/data
ports:
- "7003:6379"
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
depends_on:
- redis-master
networks:
redis-network:
ipv4_address: 172.30.1.4

需要注意以下几点

  1. 这里自定义了bridge子网并限定了范围,如果该范围已经被使用,请更换。
  2. 这里没有对data进行-v挂载,如果要挂载,请注意宿主机对应文件夹权限问题。

三、主从配置

1.主节点配置文件

主节点对应的配置文件是master/redis.conf,需要做以下修改

  1. bind
    bind 127.0.0.1 -::1修改为bind 0.0.0.0,监听来自任意网络接口的连接。

  2. protected-mode
    protected-mode设置为no,关闭保护模式,接收远程连接。

  3. masterauth
    masterauth设置为1009,这是从节点连接到主节点的认证密码,你可以指定为其他的。

  4. requirepass
    requirepass设置为1009,这是客户端连接到本节点的认证密码,你可以指定为其他的。

2.从节点配置文件

把上面主节点的配置文件复制粘贴,然后继续做以下更改,就可以作为从节点配置文件了

  1. replicaof
    旧版本添加一行replicaof redis-master 6379,表示本节点为从节点,并且主节点ipredis-master,端口为6379。这里你也可以把ip填成172.30.1.2,因为在docker-compose中我们为各节点分配了固定的ip,以及端口是6379而不是映射的700x,这些都是docker的知识,这里不再赘述。

redis在5.0引入了replica的概念来替换slave,所以后续的新版本推荐使用replicaof,即便slaveof目前仍然支持。

四、运行

配置好三个节点的配置文件后,用以下命令运行整个服务

language-shell
1
docker-compose -p redis-cluster up -d

查看主节点日志,可以看到主节点向172.30.1.3172.30.1.4两个从节点同步数据,并且连接正常,以及一系列success。

language-bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
2024-01-05 15:12:59 1:M 05 Jan 2024 07:12:59.008 * Opening AOF incr file appendonly.aof.1.incr.aof on server start
2024-01-05 15:12:59 1:M 05 Jan 2024 07:12:59.008 * Ready to accept connections tcp
2024-01-05 15:13:00 1:M 05 Jan 2024 07:13:00.996 * Replica 172.30.1.4:6379 asks for synchronization
2024-01-05 15:13:00 1:M 05 Jan 2024 07:13:00.996 * Full resync requested by replica 172.30.1.4:6379
2024-01-05 15:13:00 1:M 05 Jan 2024 07:13:00.996 * Replication backlog created, my new replication IDs are '5bef8fa8e58042f1aee8eae528c6e10228a0c96b' and '0000000000000000000000000000000000000000'
2024-01-05 15:13:00 1:M 05 Jan 2024 07:13:00.996 * Delay next BGSAVE for diskless SYNC
2024-01-05 15:13:01 1:M 05 Jan 2024 07:13:01.167 * Replica 172.30.1.3:6379 asks for synchronization
2024-01-05 15:13:01 1:M 05 Jan 2024 07:13:01.167 * Full resync requested by replica 172.30.1.3:6379
2024-01-05 15:13:01 1:M 05 Jan 2024 07:13:01.167 * Delay next BGSAVE for diskless SYNC
2024-01-05 15:13:05 1:M 05 Jan 2024 07:13:05.033 * Starting BGSAVE for SYNC with target: replicas sockets
2024-01-05 15:13:05 1:M 05 Jan 2024 07:13:05.033 * Background RDB transfer started by pid 20
2024-01-05 15:13:05 20:C 05 Jan 2024 07:13:05.035 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
2024-01-05 15:13:05 1:M 05 Jan 2024 07:13:05.035 * Diskless rdb transfer, done reading from pipe, 2 replicas still up.
2024-01-05 15:13:05 1:M 05 Jan 2024 07:13:05.052 * Background RDB transfer terminated with success
2024-01-05 15:13:05 1:M 05 Jan 2024 07:13:05.052 * Streamed RDB transfer with replica 172.30.1.4:6379 succeeded (socket). Waiting for REPLCONF ACK from replica to enable streaming
2024-01-05 15:13:05 1:M 05 Jan 2024 07:13:05.052 * Synchronization with replica 172.30.1.4:6379 succeeded
2024-01-05 15:13:05 1:M 05 Jan 2024 07:13:05.052 * Streamed RDB transfer with replica 172.30.1.3:6379 succeeded (socket). Waiting for REPLCONF ACK from replica to enable streaming
2024-01-05 15:13:05 1:M 05 Jan 2024 07:13:05.052 * Synchronization with replica 172.30.1.3:6379 succeeded

接着看看从节点日志,可以看到Connecting to MASTER redis-master:6379,向主节点连接并申请同步数据,以及一系列success。

language-bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
2024-01-05 15:13:01 1:S 05 Jan 2024 07:13:01.166 * Connecting to MASTER redis-master:6379
2024-01-05 15:13:01 1:S 05 Jan 2024 07:13:01.166 * MASTER <-> REPLICA sync started
2024-01-05 15:13:01 1:S 05 Jan 2024 07:13:01.166 * Non blocking connect for SYNC fired the event.
2024-01-05 15:13:01 1:S 05 Jan 2024 07:13:01.167 * Master replied to PING, replication can continue...
2024-01-05 15:13:01 1:S 05 Jan 2024 07:13:01.167 * Partial resynchronization not possible (no cached master)
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.033 * Full resync from master: 5bef8fa8e58042f1aee8eae528c6e10228a0c96b:0
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.035 * MASTER <-> REPLICA sync: receiving streamed RDB from master with EOF to disk
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.038 * MASTER <-> REPLICA sync: Flushing old data
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.038 * MASTER <-> REPLICA sync: Loading DB in memory
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.056 * Loading RDB produced by version 7.2.3
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.056 * RDB age 0 seconds
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.056 * RDB memory usage when created 0.90 Mb
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.056 * Done loading RDB, keys loaded: 1, keys expired: 0.
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.057 * MASTER <-> REPLICA sync: Finished with success
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.057 * Creating AOF incr file temp-appendonly.aof.incr on background rewrite
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.057 * Background append only file rewriting started by pid 21
2024-01-05 15:13:05 21:C 05 Jan 2024 07:13:05.067 * Successfully created the temporary AOF base file temp-rewriteaof-bg-21.aof
2024-01-05 15:13:05 21:C 05 Jan 2024 07:13:05.068 * Fork CoW for AOF rewrite: current 0 MB, peak 0 MB, average 0 MB
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.084 * Background AOF rewrite terminated with success
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.084 * Successfully renamed the temporary AOF base file temp-rewriteaof-bg-21.aof into appendonly.aof.5.base.rdb
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.084 * Successfully renamed the temporary AOF incr file temp-appendonly.aof.incr into appendonly.aof.5.incr.aof
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.093 * Removing the history file appendonly.aof.4.incr.aof in the background
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.093 * Removing the history file appendonly.aof.4.base.rdb in the background
2024-01-05 15:13:05 1:S 05 Jan 2024 07:13:05.101 * Background AOF rewrite finished successfully

五、测试

用你喜欢的docker容器连接工具或者redis连接工具来连接主节点redis服务,只要能进入redis-cli就行。这里以docker容器连接为例。

  1. 主节点设置一个字段并查看从节点信息
language-txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
root@ac1ecfc4e3a5:/data# redis-cli 
127.0.0.1:6379> auth 1009
OK
127.0.0.1:6379> set num 67899
OK
127.0.0.1:6379> get num
"67899"
127.0.0.1:6379> INFO replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.30.1.4,port=6379,state=online,offset=3388,lag=1
slave1:ip=172.30.1.3,port=6379,state=online,offset=3388,lag=1
master_failover_state:no-failover
master_replid:5bef8fa8e58042f1aee8eae528c6e10228a0c96b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:3388
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:3388
  1. 从节点获取
language-txt
1
2
3
4
5
root@a3016db388e3:/data# redis-cli 
127.0.0.1:6379> auth 1009
OK
127.0.0.1:6379> get num
"67899"

测试成功。

Docker-Compose部署Redis(v7.2)哨兵模式

Docker-Compose部署Redis(v7.2)哨兵模式

环境

  • docker desktop for windows 4.23.0
  • redis 7.2

一、前提准备

1. 主从集群

首先需要有一个redis主从集群,才能接着做redis哨兵。具体可以参考下面这篇文章
Docker-Compose部署Redis(v7.2)主从模式(之后简称”主从模式博文“)

2. 文件夹结构

和主从模式不同的是,redis sentinel(哨兵)会更改你的conf文件,无论是redis server节点还是sentinel节点本身,都可能被修改,所以这里需要注意文件权限问题。不然会一直警告Sentinel was not able to save the new configuration on disk

有兴趣可以参考以下几个帖子,或者接着本文做就行了。

总的来说,需要对主从模式博文里提到的文件夹结构做一定改善和添加,具体如下:

language-txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
cluster/
├── docker-compose.yaml
├── master
│ └── conf
│ └── redis.conf
├── replica1
│ └── conf
│ └── redis.conf
├── replica2
│ └── conf
│ └── redis.conf
├── sentinel1
│ └── conf
│ └── sentinel.conf
├── sentinel2
│ └── conf
│ └── sentinel.conf
└── sentinel3
└── conf
└── sentinel.conf

其中redis.confdocker-compose.yaml主从模式博文内容暂时保持一致,其余的都是新增的,暂时保持空白即可。

二、配置文件

1. redis server配置文件

保持不变

2. redis sentinel配置文件

对于上述三个sentinel.conf内容都填入以下

language-sql
1
2
3
4
5
sentinel monitor mymaster 172.30.1.2 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel auth-pass mymaster 1009
dir "/data"

意思分别是

  • 监控的主节点:通过 sentinel monitor 指定要监控的主节点。这包括一个用户定义的名称(如 mymaster)、主节点的地址、端口号和一个”仲裁”阈值,后者表示要进行故障转移所需的最小 Sentinel 投票数量。
  • 故障检测:设置 Sentinel 判断主节点是否下线所需的时间
  • 故障转移设置:配置故障转移的行为,如故障转移的超时时间
  • 认证密码(如果主节点设置了密码):如果主节点设置了密码,Sentinel 需要这个密码来连接主节点和副本节点
  • 设置 Sentinel 的工作目录

3. docker compose文件

language-yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
version: '3.8'

networks:
redis-network:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.30.1.0/24

services:
redis-master:
container_name: redis-master
image: redis:7.2
volumes:
- ./master/conf:/usr/local/etc/redis
# - ./master/data:/data
ports:
- "7001:6379"
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
networks:
redis-network:
ipv4_address: 172.30.1.2

redis-replica1:
container_name: redis-replica1
image: redis:7.2
volumes:
- ./replica1/conf:/usr/local/etc/redis
# - ./replica1/data:/data
ports:
- "7002:6379"
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
depends_on:
- redis-master
networks:
redis-network:
ipv4_address: 172.30.1.3

redis-replica2:
container_name: redis-replica2
image: redis:7.2
volumes:
- ./replica2/conf:/usr/local/etc/redis
# - ./replica2/data:/data
ports:
- "7003:6379"
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
depends_on:
- redis-master
networks:
redis-network:
ipv4_address: 172.30.1.4

redis-sentinel1:
container_name: redis-sentinel1
image: redis:7.2
volumes:
- ./sentinel1/conf:/usr/local/etc/redis
ports:
- "27001:26379"
command: ["redis-sentinel", "/usr/local/etc/redis/sentinel.conf"]
depends_on:
- redis-master
networks:
redis-network:
ipv4_address: 172.30.1.11

redis-sentinel2:
container_name: redis-sentinel2
image: redis:7.2
volumes:
- ./sentinel2/conf:/usr/local/etc/redis
ports:
- "27002:26379"
command: [ "redis-sentinel", "/usr/local/etc/redis/sentinel.conf" ]
depends_on:
- redis-master
networks:
redis-network:
ipv4_address: 172.30.1.12

redis-sentinel3:
container_name: redis-sentinel3
image: redis:7.2
volumes:
- ./sentinel3/conf:/usr/local/etc/redis
ports:
- "27003:26379"
command: [ "redis-sentinel", "/usr/local/etc/redis/sentinel.conf" ]
depends_on:
- redis-master
networks:
redis-network:
ipv4_address: 172.30.1.13



需要注意以下几点

  • 主从模式博文不同,这里所有的配置文件挂载都采用文件夹挂载而非单文件挂载
  • 这里自定义了bridge子网并限定了范围,如果该范围已经被使用,请更换。
  • 这里没有对data进行-v挂载,如果要挂载,请注意宿主机对应文件夹权限问题。
  • 主节点地址为172.30.1.2,如果更改请注意sentinel.conf中也需要更改。

三、运行

在运行之前,记得备份一下所有的conf文件,因为sentinel会修改挂载到容器的conf。

language-bash
1
docker-compose -p redis-cluster up -d

查看其中一个sentinel节点的日志,可以看到监听端口是26379,同时监测主节点mymaster 172.30.1.2 6379,以及添加了172.30.1.4 6379172.30.1.3 6379两个从节点,并且感应到了位于172.30.1.13 26379172.30.1.12 26379两个同为sentinel节点的服务。

language-txt
1
2
3
4
5
6
7
8
9
10
11
12
2024-01-05 18:06:40 1:X 05 Jan 2024 10:06:40.758 * Running mode=sentinel, port=26379.
2024-01-05 18:06:40 1:X 05 Jan 2024 10:06:40.789 * Sentinel new configuration saved on disk
2024-01-05 18:06:40 1:X 05 Jan 2024 10:06:40.790 * Sentinel ID is 499007c98c0a165b13e026a4443ceb890695c191
2024-01-05 18:06:40 1:X 05 Jan 2024 10:06:40.790 # +monitor master mymaster 172.30.1.2 6379 quorum 2
2024-01-05 18:06:40 1:X 05 Jan 2024 10:06:40.791 * +slave slave 172.30.1.4:6379 172.30.1.4 6379 @ mymaster 172.30.1.2 6379
2024-01-05 18:06:40 1:X 05 Jan 2024 10:06:40.815 * Sentinel new configuration saved on disk
2024-01-05 18:06:42 1:X 05 Jan 2024 10:06:42.055 * +sentinel sentinel bcfaed15fb01e7ad03b013fe5e964479c1a1f138 172.30.1.13 26379 @ mymaster 172.30.1.2 6379
2024-01-05 18:06:42 1:X 05 Jan 2024 10:06:42.093 * Sentinel new configuration saved on disk
2024-01-05 18:06:42 1:X 05 Jan 2024 10:06:42.356 * +sentinel sentinel 92d9a1419be1256d1715df2aa17cea4bbacfdf60 172.30.1.12 26379 @ mymaster 172.30.1.2 6379
2024-01-05 18:06:42 1:X 05 Jan 2024 10:06:42.376 * Sentinel new configuration saved on disk
2024-01-05 18:06:50 1:X 05 Jan 2024 10:06:50.823 * +slave slave 172.30.1.3:6379 172.30.1.3 6379 @ mymaster 172.30.1.2 6379
2024-01-05 18:06:50 1:X 05 Jan 2024 10:06:50.837 * Sentinel new configuration saved on disk

四、测试

直接让redis-master容器停止运行,查看sentinel日志,可以看到sentinel监测到master节点挂掉后,选举了172.30.1.3为新的主节点,并将其余两个作为slave节点。

language-txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
2024-01-05 18:10:08 1:X 05 Jan 2024 10:10:08.896 # +sdown master mymaster 172.30.1.2 6379
2024-01-05 18:10:08 1:X 05 Jan 2024 10:10:08.968 # +odown master mymaster 172.30.1.2 6379 #quorum 2/2
2024-01-05 18:10:08 1:X 05 Jan 2024 10:10:08.968 # +new-epoch 1
2024-01-05 18:10:08 1:X 05 Jan 2024 10:10:08.968 # +try-failover master mymaster 172.30.1.2 6379
2024-01-05 18:10:08 1:X 05 Jan 2024 10:10:08.987 * Sentinel new configuration saved on disk
2024-01-05 18:10:08 1:X 05 Jan 2024 10:10:08.987 # +vote-for-leader 499007c98c0a165b13e026a4443ceb890695c191 1
2024-01-05 18:10:08 1:X 05 Jan 2024 10:10:08.990 * 92d9a1419be1256d1715df2aa17cea4bbacfdf60 voted for 92d9a1419be1256d1715df2aa17cea4bbacfdf60 1
2024-01-05 18:10:09 1:X 05 Jan 2024 10:10:09.021 * bcfaed15fb01e7ad03b013fe5e964479c1a1f138 voted for 499007c98c0a165b13e026a4443ceb890695c191 1
2024-01-05 18:10:09 1:X 05 Jan 2024 10:10:09.054 # +elected-leader master mymaster 172.30.1.2 6379
2024-01-05 18:10:09 1:X 05 Jan 2024 10:10:09.054 # +failover-state-select-slave master mymaster 172.30.1.2 6379
2024-01-05 18:10:09 1:X 05 Jan 2024 10:10:09.125 # +selected-slave slave 172.30.1.3:6379 172.30.1.3 6379 @ mymaster 172.30.1.2 6379
2024-01-05 18:10:09 1:X 05 Jan 2024 10:10:09.125 * +failover-state-send-slaveof-noone slave 172.30.1.3:6379 172.30.1.3 6379 @ mymaster 172.30.1.2 6379
2024-01-05 18:10:09 1:X 05 Jan 2024 10:10:09.209 * +failover-state-wait-promotion slave 172.30.1.3:6379 172.30.1.3 6379 @ mymaster 172.30.1.2 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.033 * Sentinel new configuration saved on disk
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.033 # +promoted-slave slave 172.30.1.3:6379 172.30.1.3 6379 @ mymaster 172.30.1.2 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.033 # +failover-state-reconf-slaves master mymaster 172.30.1.2 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.094 * +slave-reconf-sent slave 172.30.1.4:6379 172.30.1.4 6379 @ mymaster 172.30.1.2 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.262 * +slave-reconf-inprog slave 172.30.1.4:6379 172.30.1.4 6379 @ mymaster 172.30.1.2 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.262 * +slave-reconf-done slave 172.30.1.4:6379 172.30.1.4 6379 @ mymaster 172.30.1.2 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.338 # +failover-end master mymaster 172.30.1.2 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.338 # +switch-master mymaster 172.30.1.2 6379 172.30.1.3 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.338 * +slave slave 172.30.1.4:6379 172.30.1.4 6379 @ mymaster 172.30.1.3 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.338 * +slave slave 172.30.1.2:6379 172.30.1.2 6379 @ mymaster 172.30.1.3 6379
2024-01-05 18:10:10 1:X 05 Jan 2024 10:10:10.373 * Sentinel new configuration saved on disk

接着让我们看看172.30.1.3的日志,也就是redis-replica1的日志,可以看到与主节点连接失败后,它开启了主节点模式MASTER MODE enabled

language-txt
1
2
3
4
5
6
7
8
9
10
11
2024-01-05 18:10:03 1:S 05 Jan 2024 10:10:03.812 * Reconnecting to MASTER 172.30.1.2:6379
2024-01-05 18:10:03 1:S 05 Jan 2024 10:10:03.813 * MASTER <-> REPLICA sync started
2024-01-05 18:10:03 1:S 05 Jan 2024 10:10:03.813 # Error condition on socket for SYNC: Connection refused
2024-01-05 18:10:04 1:S 05 Jan 2024 10:10:04.582 * Connecting to MASTER 172.30.1.2:6379
2024-01-05 18:10:04 1:S 05 Jan 2024 10:10:04.582 * MASTER <-> REPLICA sync started
2024-01-05 18:10:09 1:M 05 Jan 2024 10:10:09.209 * Discarding previously cached master state.
2024-01-05 18:10:09 1:M 05 Jan 2024 10:10:09.209 * Setting secondary replication ID to 5032654a1279c56d758c93a4eb1c4b89c99975a9, valid up to offset: 40756. New replication ID is d3464601d550e1159d91234567a366fa1f1a0b5e
2024-01-05 18:10:09 1:M 05 Jan 2024 10:10:09.209 * MASTER MODE enabled (user request from 'id=8 addr=172.30.1.11:55710 laddr=172.30.1.3:6379 fd=13 name=sentinel-499007c9-cmd age=199 idle=0 flags=x db=0 sub=0 psub=0 ssub=0 multi=4 qbuf=188 qbuf-free=20286 argv-mem=4 multi-mem=169 rbs=2048 rbp=1024 obl=45 oll=0 omem=0 tot-mem=23717 events=r cmd=exec user=default redir=-1 resp=2 lib-name= lib-ver=')
2024-01-05 18:10:09 1:M 05 Jan 2024 10:10:09.229 * CONFIG REWRITE executed with success.
2024-01-05 18:10:10 1:M 05 Jan 2024 10:10:10.120 * Replica 172.30.1.4:6379 asks for synchronization
2024-01-05 18:10:10 1:M 05 Jan 2024 10:10:10.120 * Partial resynchronization request from 172.30.1.4:6379 accepted. Sending 567 bytes of backlog starting from offset 40756.

并且还有redis-replica2的日志,里面会显示将数据同步请求地址变成了172.30.1.3而不是先前的172.30.1.2

接着连接redis-replica1容器看看,发现这个节点以前作为从节点时是只读节点,现在可以写入数据了。

language-txt
1
2
3
4
5
6
7
root@1eefea35001f:/data# redis-cli 
127.0.0.1:6379> auth 1009
OK
127.0.0.1:6379> set num 8766
OK
127.0.0.1:6379> get num
"8766"

并且会发现另外两个节点变成只读了,同时,即使先前的主节点又恢复正常了,它不会去夺回master地位。

测试成功。

Docker单点部署Seata(2.0.0) + Nacos(v2.3.0) + Mysql(5.7)

Docker单点部署Seata(2.0.0) + Nacos(v2.3.0) + Mysql(5.7)

系统环境
docker desktop for windows v4.23.0
nacosmysqlseata三者都在bridge网络中

一、部署Nacos

language-bash
1
2
3
4
5
6
docker run -itd \
-e MODE=standalone
-e NACOS_SERVER_PORT=8848
-p 8848:8848
--name=nacos_standalone
nacos/nacos-server:v2.3.0

二、部署Mysql

language-bash
1
2
3
4
5
docker run -itd \
-e MYSQL_ROOT_PASSWORD=1009
-p 3306:3306
--name=mysql_itcast
mysql:5.7

三、Seata准备工作

1. 记住nacos、mysql、宿主机的ip

language-bash
1
$ docker network inspect bridge

假设这里nacos172.17.0.3mysql172.17.0.2

language-bash
1
ipconfig /all

这里假设宿主机ip为192.168.1.102

之后遇到上述三个ip,记得写成自己的

2. 建立数据库

mysql_itcast中新建seata数据库,然后导入以下脚本

language-sql
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
-- -------------------------------- The script used when storeMode is 'db' --------------------------------
-- the table to store GlobalSession data
CREATE TABLE IF NOT EXISTS `global_table`
(
`xid` VARCHAR(128) NOT NULL,
`transaction_id` BIGINT,
`status` TINYINT NOT NULL,
`application_id` VARCHAR(32),
`transaction_service_group` VARCHAR(32),
`transaction_name` VARCHAR(128),
`timeout` INT,
`begin_time` BIGINT,
`application_data` VARCHAR(2000),
`gmt_create` DATETIME,
`gmt_modified` DATETIME,
PRIMARY KEY (`xid`),
KEY `idx_status_gmt_modified` (`status` , `gmt_modified`),
KEY `idx_transaction_id` (`transaction_id`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8mb4;

-- the table to store BranchSession data
CREATE TABLE IF NOT EXISTS `branch_table`
(
`branch_id` BIGINT NOT NULL,
`xid` VARCHAR(128) NOT NULL,
`transaction_id` BIGINT,
`resource_group_id` VARCHAR(32),
`resource_id` VARCHAR(256),
`branch_type` VARCHAR(8),
`status` TINYINT,
`client_id` VARCHAR(64),
`application_data` VARCHAR(2000),
`gmt_create` DATETIME(6),
`gmt_modified` DATETIME(6),
PRIMARY KEY (`branch_id`),
KEY `idx_xid` (`xid`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8mb4;

-- the table to store lock data
CREATE TABLE IF NOT EXISTS `lock_table`
(
`row_key` VARCHAR(128) NOT NULL,
`xid` VARCHAR(128),
`transaction_id` BIGINT,
`branch_id` BIGINT NOT NULL,
`resource_id` VARCHAR(256),
`table_name` VARCHAR(32),
`pk` VARCHAR(36),
`status` TINYINT NOT NULL DEFAULT '0' COMMENT '0:locked ,1:rollbacking',
`gmt_create` DATETIME,
`gmt_modified` DATETIME,
PRIMARY KEY (`row_key`),
KEY `idx_status` (`status`),
KEY `idx_branch_id` (`branch_id`),
KEY `idx_xid` (`xid`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8mb4;

CREATE TABLE IF NOT EXISTS `distributed_lock`
(
`lock_key` CHAR(20) NOT NULL,
`lock_value` VARCHAR(20) NOT NULL,
`expire` BIGINT,
primary key (`lock_key`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8mb4;

INSERT INTO `distributed_lock` (lock_key, lock_value, expire) VALUES ('AsyncCommitting', ' ', 0);
INSERT INTO `distributed_lock` (lock_key, lock_value, expire) VALUES ('RetryCommitting', ' ', 0);
INSERT INTO `distributed_lock` (lock_key, lock_value, expire) VALUES ('RetryRollbacking', ' ', 0);
INSERT INTO `distributed_lock` (lock_key, lock_value, expire) VALUES ('TxTimeoutCheck', ' ', 0);

3. Nacos远程配置文件

访问Nacos网页,一般是http://localhost:8848/nacos/,新建一个配置seataServer.properties

具体内容如下:

language-puppet
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
store.mode=db
#-----db-----
store.db.datasource=druid
store.db.dbType=mysql
# 需要根据mysql的版本调整driverClassName
# mysql8及以上版本对应的driver:com.mysql.cj.jdbc.Driver
# mysql8以下版本的driver:com.mysql.jdbc.Driver
store.db.driverClassName=com.mysql.jdbc.Driver
store.db.url=jdbc:mysql://172.17.0.2:3306/seata?useUnicode=true&characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true&useSSL=false
store.db.user= root
store.db.password=1009
# 数据库初始连接数
store.db.minConn=1
# 数据库最大连接数
store.db.maxConn=20
# 获取连接时最大等待时间 默认5000,单位毫秒
store.db.maxWait=5000
# 全局事务表名 默认global_table
store.db.globalTable=global_table
# 分支事务表名 默认branch_table
store.db.branchTable=branch_table
# 全局锁表名 默认lock_table
store.db.lockTable=lock_table
store.db.distributedLockTable=distributed_lock
# 查询全局事务一次的最大条数 默认100
store.db.queryLimit=100


# undo保留天数 默认7天,log_status=1(附录3)和未正常清理的undo
server.undo.logSaveDays=7
# undo清理线程间隔时间 默认86400000,单位毫秒
server.undo.logDeletePeriod=86400000
# 二阶段提交重试超时时长 单位ms,s,m,h,d,对应毫秒,秒,分,小时,天,默认毫秒。默认值-1表示无限重试
# 公式: timeout>=now-globalTransactionBeginTime,true表示超时则不再重试
# 注: 达到超时时间后将不会做任何重试,有数据不一致风险,除非业务自行可校准数据,否者慎用
server.maxCommitRetryTimeout=-1
# 二阶段回滚重试超时时长
server.maxRollbackRetryTimeout=-1
# 二阶段提交未完成状态全局事务重试提交线程间隔时间 默认1000,单位毫秒
server.recovery.committingRetryPeriod=1000
# 二阶段异步提交状态重试提交线程间隔时间 默认1000,单位毫秒
server.recovery.asynCommittingRetryPeriod=1000
# 二阶段回滚状态重试回滚线程间隔时间 默认1000,单位毫秒
server.recovery.rollbackingRetryPeriod=1000
# 超时状态检测重试线程间隔时间 默认1000,单位毫秒,检测出超时将全局事务置入回滚会话管理器
server.recovery.timeoutRetryPeriod=1000

四、部署Seata

宿主机新建一个application.yml文件,内容如下

language-yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
server:
port: 7091

spring:
application:
name: seata-server

logging:
config: classpath:logback-spring.xml
file:
path: ${
user.home}/logs/seata
extend:
logstash-appender:
destination: 127.0.0.1:4560
kafka-appender:
bootstrap-servers: 127.0.0.1:9092
topic: logback_to_logstash

console:
user:
username: seata
password: seata

seata:
config:
# support: nacos, consul, apollo, zk, etcd3
type: nacos
nacos:
server-addr: 172.17.0.3:8848
namespace:
group: DEFAULT_GROUP
username: nacos
password: nacos
data-id: seataServer.properties

registry:
# support: nacos, eureka, redis, zk, consul, etcd3, sofa
type: nacos
nacos:
application: seata-tc-server
server-addr: 172.17.0.3:8848
group: DEFAULT_GROUP
namespace:
# tc集群名称
cluster: SH
username: nacos
password: nacos
# server:
# service-port: 8091 #If not configured, the default is '${server.port} + 1000'
security:
secretKey: SeataSecretKey0c382ef121d778043159209298fd40bf3850a017
tokenValidityInMilliseconds: 1800000
ignore:
urls: /,/**/*.css,/**/*.js,/**/*.html,/**/*.map,/**/*.svg,/**/*.png,/**/*.ico,/console-fe/public/**,/api/v1/auth/login

然后用以下命令运行seata容器

language-bash
1
2
3
4
5
6
7
8
9
docker run --name seata-server \
-itd \
-p 8091:8091 \
-p 7091:7091 \
-e STORE_MODE=db \
-e SEATA_IP="192.168.1.102" \
-e SEATA_PORT=8091 \
-v "path/to/application.yml:/seata-server/resources/application.yml" \
seataio/seata-server:2.0.0

五、初步检验Seata部署情况

访问Seata网页,这里是http://192.168.1.102:7091/,输入两个seata后进入系统。

Nacos网页上查看Seata服务详情,ip为宿主机ip,不要是docker容器内网ip就行。

六、微服务使用Seata

1.引入依赖

language-xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<!--seata-->
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-seata</artifactId>
<exclusions>
<!--版本较低,1.3.0,因此排除-->
<exclusion>
<artifactId>seata-spring-boot-starter</artifactId>
<groupId>io.seata</groupId>
</exclusion>
</exclusions>
</dependency>
<!--seata starter 采用1.4.2版本-->
<dependency>
<groupId>io.seata</groupId>
<artifactId>seata-spring-boot-starter</artifactId>
<version>1.4.2</version>
<!--2.0.0貌似有点问题,TCC模式BusinessActionContextParameter无效-->
</dependency>

2. application.yml配置

language-yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
seata:
registry:
type: nacos
nacos: # tc
server-addr: localhost:8848
namespace: ""
group: DEFAULT_GROUP
application: seata-tc-server # tc服务在nacos中的服务名称
cluster: SH
username: nacos
password: nacos
tx-service-group: seata-demo # 事务组,根据这个获取tc服务的cluster名称
service:
vgroup-mapping: # 事务组与TC服务cluster的映射关系
seata-demo: SH

启动微服务后,除了可以看微服务的日志外,还可以看Seata容器日志,出现类似以下日志即为正常

language-txt
1
2
3
4
5
6
7
2023-12-30 21:37:35 digest=seata-demo,192.168.222.1,1703943453643
2023-12-30 21:37:35 timestamp=1703943453643
2023-12-30 21:37:35 authVersion=V4
2023-12-30 21:37:35 vgroup=seata-demo
2023-12-30 21:37:35 ip=192.168.222.1
2023-12-30 21:37:35 '},channel:[id: 0x7f82356a, L:/172.17.0.4:8091 - R:/172.17.0.1:35092],client version:2.0.0
2023-12-30 21:37:36 21:37:36.389 INFO --- [rverHandlerThread_1_6_500] [rocessor.server.RegRmProcessor] [ onRegRmMessage] [] : RM register success,message:RegisterRMRequest{resourceIds='jdbc:mysql://localhost:3306/seata_demo', version='2.0.0', applicationId='order-service', transactionServiceGroup='seata-demo', extraData='null'},channel:[id: 0x3a9f4e29, L:/172.17.0.4:8091 - R:/172.17.0.1:35096],client version:2.0.0

七、遇到的坑

1. Nacos显示Seata服务的ip为容器内网ip导致微服务无法访问

网上看到以下各种方法均无效

  1. 使用host网络
  2. application.yml指定spring.cloud.nacos.discovery.ip

以下方法有效

  1. 容器创建时使用 -e SEATA_IP="宿主机ip"

2. 使用host宿主机网络

一开始为了图方便,给Nacos用过host网络,结果容器程序运行正常,打不开网页,玄学的一批。
也给Seata使用host网络,为了配置文件里面不用自己手动查询nacos和mysql的ip,结果然并卵。

3. seata The distribute lock table is not config, please create the target table and config it

这个是因为很多文档,都只有3张表,少了一张。
官方文档说store.db.distributedLockTable1.5.1版本新增的参数。
https://seata.io/zh-cn/docs/user/configurations
但是很多文档和博客,都只有3张表,第4张在哪里呢?
在这里
https://seata.io/zh-cn/docs/ops/deploy-by-docker-compose.html
里面写到nacos注册中心,db存储时会提供 [建表脚本]
以及最后最重要的是,要在Nacos配置中心配置seataServer.properties时,要多加一行

language-bash
1
store.db.distributedLockTable=distributed_lock

这点在官网文档都没有提及。

4. 高版本中BusinessActionContextParameter和TwoPhaseBusinessAction推荐都放在实现类中,接口上的做法后续将会废除

具体参考这个Issue
2.0.0 TCC模式@BusinessActionContextParameter修饰的参数失效,无法在BusinessActionContext获取

Docker Compose怎么保证其他服务在Nacos完全启动后再启动

Docker Compose怎么保证其他服务在Nacos完全启动后再启动

首先,docker compose有一个关键字depends_on是可以一定程度解决services之间的依赖问题,但是depends_on仅仅只是指定了services的启动顺序,并不能保证,前置service完全启动后,后置service才启动。

此时,需要另一个关键字叫healthcheck

样例

nacos版本为v2.3.0

language-yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
  mysql_nacos:
container_name: mysql_nacos
build:
context: .
dockerfile: ./image/mysql/5.7/Dockerfile
image: example/mysql:5.7
env_file:
- ../env/mysql.env
volumes:
- ./mysql:/var/lib/mysql
# ports:
# - "3307:3306"
healthcheck:
test: [ "CMD", "mysqladmin" ,"ping", "-h", "localhost", "-u", "root", "-p${MYSQL_ROOT_PASSWORD}" ]
interval: 10s
timeout: 10s
retries: 10

nacos1:
hostname: nacos1
container_name: nacos1
image: nacos/nacos-server:${
NACOS_VERSION}
volumes:
- ./cluster-logs/nacos1:/home/nacos/logs
# ports:
# - "7848:7848"
# - "8845:8848"
# - "9868:9848"
# - "9850:9849"
env_file:
- ../env/nacos-hostname.env
# restart: always
healthcheck:
test: [ "CMD", "curl", "-f", "http://localhost:8848/nacos/v1/console/health/readiness" ]
interval: 10s
timeout: 10s
retries: 10
depends_on:
mysql_nacos:
condition: service_healthy

nginx_nacos:
hostname: nginx_nacos
image: nginx:stable
container_name: nginx_nacos
volumes:
- ../nginx_nacos/nginx.conf:/etc/nginx/nginx.conf
- ../nginx_nacos/conf.d/default.conf:/etc/nginx/conf.d/default.conf
- ../nginx_nacos/logs:/var/log/nginx
- ../nginx_nacos/html:/usr/share/nginx/html
ports:
- "80:80"
depends_on:
nacos1:
condition: service_healthy

这是一个双层依赖,nginx_nacos依赖于nacos1,同时nacos1依赖于mysql_nacos

mysql_nacos基于 healthcheck:test: [ "CMD", "mysqladmin" ,"ping", "-h", "localhost", "-u", "root", "-p${MYSQL_ROOT_PASSWORD}" ]来确认自己的健康状态,只有当test内容执行成功并返回状态码0,才会认为此service已完全执行成功。

同理nacos服务是基于healthcheck:test: [ "CMD", "curl", "-f", "http://localhost:8848/nacos/v1/console/health/readiness" ]来判断健康状态的。

优雅地给Docker容器添加新端口

优雅地给Docker容器添加新端口

一共分为三步,停止容器和docker服务,修改配置文件,重启服务和容器。

这里只讲如何修改配置文件。

如果你是Linux环境

容器配置文件hostconfig.json 通常位于 /var/lib/docker/containers/[hash_of_the_container]/hostconfig.json 或者 /var/snap/docker/common/var-lib-docker/containers/[hash_of_the_container]/hostconfig.json

找到PortBindings字段,以下是一个端口的格式例子

1
2
3
4
5
6
7
8
9
10
11
12
13
14
"PortBindings": {
"8080/tcp": [
{
"HostIp": "",
"HostPort": "8080"
}
],
"8088/tcp": [
{
"HostIp": "",
"HostPort": "8088"
}
]
},

如果不起作用,建议同时修改下面提到的config.v2.json

如果你是windws+wsl2环境

那么你需要修改两个文件,hostconfig.jsonconfig.v2.json,它们都位于/mnt/wsl/docker-desktop-data/data/docker/<containerID>下。

hostconfig.json文件修改和linux的一样。

config.v2.json需要修改以下两个字段

1
2
3
"ExposedPorts":{"8080/tcp":{},"8088/tcp":{}}

"Ports":{"8080/tcp":[{"HostIp":"0.0.0.0","HostPort":"8080"}],"8088/tcp":[{"HostIp":"0.0.0.0","HostPort":"8088"}]}

参考资料

How do I assign a port mapping to an existing Docker container? - Stack Overflowhttps://stackoverflow.com/questions/19335444/how-do-i-assign-a-port-mapping-to-an-existing-docker-containerAdding new exposed ports to existing docker container (Windows/WSL 2) | by LifeOnKeyboardBlog | Mediumhttps://medium.com/@lifeonkeyboardblog/adding-new-exposed-ports-to-existing-docker-container-windows-wsl-2-3cfe58d551e

Docker容器中删除文件后,大小不减反增怎么办?

Docker容器中删除文件后,大小不减反增怎么办?

Docker的镜像是由多层存储结构组成的,每一层都是只读的,所以您无法删除之前层的操作。

但是可以通过以下步骤达到一样的效果。

假设你从原始镜像a中创建了容器b。

现在你在容器b中删除了一些东西。

您可以使用docker export命令将容器的文件系统导出为一个tar文件:

1
docker export -o my.tar containerID

然后,使用docker import命令从这个tar文件导入为一个新的镜像:

1
docker import my.tar new-image

此时这个new-image镜像就摆脱了旧镜像的历史层,大小也会相应大大减少。

但是你可能会失去一些东西:环境变量、端口映射、镜像的历史记录和层级结构、镜像的标签和版本号、镜像的创建时间和作者、容器的启动命令和参数等等。