mongodb 3.4 分片 一主 一副 一仲 鉴权集群部署.

时间:2023-03-10 20:40:13

Docker方式部署

为了避免过分冗余,并且在主节点挂了,还能顺利自动提升,所以加入仲裁节点

mongodb版本:mongodb 3.4 分片 一主 一副 一仲 鉴权集群部署.

环境:一台虚拟机

三个configsvr 副本: 端口为 27020,27021,27022

两个分片:

  shard1:-> 三个副本,端口为 27010,27011,27012

  shard2:-> 三个副本,端口为 27013,27014,27015

一个路由:mongos -> 端口为 27023

前置条件:

创建数据存储文件的目录

mkdir /usr/local/mongodb/data
mkdir /usr/local/mongodb/log
cd /usr/local/mongodb/data
mkdir c0 && mkdir c1 && mkdir c2 && mkdir s100 && mkdir s101 && mkdir s102 && mkdir s200 && mkdir s201 && mkdir s202

生成鉴权需要的keyfile,keyfile 内容不能太长,否则启动不了,权限不能太大,否则也是启动不了

openssl rand -base64 512 > /usr/local/mongodb/keyfile
chmod 600 /usr/local/mongodb/keyfile

副本模式启动configsvr

 mongod --dbpath /usr/local/mongodb/data/c0 --logpath /usr/local/mongodb/log/c0.log --fork --smallfiles --port  --replSet cs --configsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/c1 --logpath /usr/local/mongodb/log/c1.log --fork --smallfiles --port --replSet cs --configsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/c2 --logpath /usr/local/mongodb/log/c2.log --fork --smallfiles --port --replSet cs --configsvr --bind_ip=192.168.1.9

集群配置,登陆任意一个configsvr

mongo 192.168.1.9:
var css={_id:"cs","configsvr":true,members:[{_id:,host:"192.168.1.9:27020"},{_id:,host:"192.168.1.9:27021"},{_id:,host:"192.168.1.9:27022"}]}
rs.initiate(css)

副本模式启动分片1

 mongod --dbpath /usr/local/mongodb/data/s100 --logpath /usr/local/mongodb/log/s100.log --fork --smallfiles --port  --replSet shard1 --shardsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/s101 --logpath /usr/local/mongodb/log/s101.log --fork --smallfiles --port --replSet shard1 --shardsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/s102 --logpath /usr/local/mongodb/log/s102.log --fork --smallfiles --port --replSet shard1 --shardsvr --bind_ip=192.168.1.9

登陆任意一个分片1

use admin;
var cnf={_id:"shard1",members:[{_id:,host:"192.168.1.9:27017"},{_id:,host:"192.168.1.9:27018"}]}
rs.initiate(cnf)
rs.addArb("192.168.1.9:27019")#仲裁节点

副本模式启动分片2

 mongod --dbpath /usr/local/mongodb/data/s100 --logpath /usr/local/mongodb/log/s100.log --fork --smallfiles --port  --replSet shard2 --shardsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/s101 --logpath /usr/local/mongodb/log/s101.log --fork --smallfiles --port --replSet shard2 --shardsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/s102 --logpath /usr/local/mongodb/log/s102.log --fork --smallfiles --port --replSet shard2 --shardsvr --bind_ip=192.168.1.9

登陆任意一个分片2,操作同分片1

启动路由

mongos --logpath /usr/local/mongodb/log/m23.log --port  --fork --configdb cs/192.168.1.9:,192.168.1.9:,192.168.1.9: --bind_ip=192.168.1.9

登陆路由

mongo 192.168.1.9:

添加分片,这里写全了分片的 host, 实际只需要添加其中一个副本或者仲裁host即可 可选192.168.1.9:27010 192.168.1.9:27011 192.168.1.9:27012 其中一个

mongodb 3.4 分片 一主 一副 一仲 鉴权集群部署.

设置分片数据库,设置片键

mongos> sh.enableSharding("testdb")
mongos> sh.shardCollection("testdb.orderInfo",{"_id":"hashed"})
{ "collectionsharded" : "testdb.orderInfo", "ok" : }

趁还没有加上鉴权,赶紧添加用户

use admin
db.createUser(
{
user: "dba",
pwd: "dba",
roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
}
)
use testdb
mongos> db.createUser(
...   {
...     user: "testuser",
...     pwd: "testuser",
...     roles: [ { role: "readWrite", db: "testdb" } ]
...   }
... )
Successfully added user: {
"user" : "testuser",
"roles" : [
{
"role" : "readWrite",
"db" : "testdb"
}
]
}
mongos> db.auth("testuser","testuser") mongos> exit

然后依次关闭mongodb,等下添加鉴权再启动.

因为懒,我选择重启,自己的电脑,随便整,别太较真.......

mongodb 3.4 分片 一主 一副 一仲 鉴权集群部署.

依次启动mongod,这次加上鉴权参数   --keyFile /usr/local/mongodb/keyfile

mongod --dbpath /usr/local/mongodb/data/c0 --logpath /usr/local/mongodb/log/c0.log --keyFile /usr/local/mongodb/keyfile --fork --smallfiles --port  --replSet cs --configsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/c1 --logpath /usr/local/mongodb/log/c1.log --keyFile /usr/local/mongodb/keyfile --fork --smallfiles --port --replSet cs --configsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/c2 --logpath /usr/local/mongodb/log/c2.log --keyFile /usr/local/mongodb/keyfile --fork --smallfiles --port --replSet cs --configsvr --bind_ip=192.168.1.9 mongod --dbpath /usr/local/mongodb/data/s100 --logpath /usr/local/mongodb/log/s100.log --keyFile /usr/local/mongodb/keyfile --fork --smallfiles --port --replSet shard1 --shardsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/s101 --logpath /usr/local/mongodb/log/s101.log --keyFile /usr/local/mongodb/keyfile --fork --smallfiles --port --replSet shard1 --shardsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/s102 --logpath /usr/local/mongodb/log/s102.log --keyFile /usr/local/mongodb/keyfile --fork --smallfiles --port --replSet shard1 --shardsvr --bind_ip=192.168.1.9 mongod --dbpath /usr/local/mongodb/data/s100 --logpath /usr/local/mongodb/log/s100.log --keyFile /usr/local/mongodb/keyfile --fork --smallfiles --port --replSet shard2 --shardsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/s101 --logpath /usr/local/mongodb/log/s101.log --keyFile /usr/local/mongodb/keyfile --fork --smallfiles --port --replSet shard2 --shardsvr --bind_ip=192.168.1.9
mongod --dbpath /usr/local/mongodb/data/s102 --logpath /usr/local/mongodb/log/s102.log --keyFile /usr/local/mongodb/keyfile --fork --smallfiles --port --replSet shard2 --shardsvr --bind_ip=192.168.1.9
mongos --logpath /usr/local/mongodb/log/m23.log --port --fork --keyFile /usr/local/mongodb/keyfile --configdb cs/192.168.1.9:,192.168.1.9:,192.168.1.9: --bind_ip=192.168.1.9

测试:

[root@ conf]# mongo 192.168.1.9:
MongoDB shell version v3.4.10
connecting to: 192.168.1.9:
MongoDB server version: 3.4.
mongos> use testdb
switched to db testdb
mongos> db.auth("testuser","testuser") mongos> db.auth("testuser","testuser") mongos> exit
bye

或者

mongo 192.168.1.9:/testdb -u testuser -p

测试不使用账号密码

mongodb 3.4 分片 一主 一副 一仲 鉴权集群部署.

使用dba登陆,创建超级管理员用户,否则没有sh权限

mongos> use admin
switched to db admin
mongos> db.createUser(
... {
... user: "root",
... pwd: "root",
... roles: [ { role: "root", db: "admin" } ]
... }
... )
Successfully added user: {
"user" : "root",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
mongos> db.auth("root","root")
1
mongos> sh.status();
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5a0d5d371c121ebe9dcbdace")
}
shards:
{ "_id" : "shard1", "host" : "shard1/192.168.1.9:27010,192.168.1.9:27011,192.168.1.9:27012", "state" : 1 }
{ "_id" : "shard2", "host" : "shard2/192.168.1.9:27013,192.168.1.9:27014,192.168.1.9:27015", "state" : 1 }
active mongoses:
"3.4.10" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
NaN
Failed balancer rounds in last 5 attempts: 2
Last reported error: could not find host matching read preference { mode: "primary" } for set shard1
Time of Reported error: Fri Nov 17 2017 08:33:47 GMT+0800 (CST)
Migration Results for the last 24 hours:
28 : Success
1 : Failed with error 'aborted', from shard2 to shard1
databases:
{ "_id" : "testdb", "primary" : "shard2", "partitioned" : true }
testdb.orderInfo
shard key: { "taxNo" : 1, "lastModifyDate" : 1 }
unique: false
balancing: true
chunks:
shard1 28
shard2 29
too many chunks to print, use verbose if you want to force print

补充:

集群用户不能用来认证单个shard节点,必须要在shard节点单独建立用户

以下测试,片键为公司编码以及用户名,复合键片,实现多热点

插入500W 单文档数据,不包含内嵌文档,在NVME 固态硬盘下表现 速度非常快.只用了56秒

完成插入查看数据分布情况

mongodb 3.4 分片 一主 一副 一仲 鉴权集群部署.

分布不太均衡,等一会再看一次

mongodb 3.4 分片 一主 一副 一仲 鉴权集群部署.

非常均衡,因为我用了9个公司编码,14个随机用户名插入.但是只有两个分片,所以导致数据会分发倒第一次插入的分片中,导致数据需要频繁自动均衡.建议有条件的,初始化的时候,创建多个分片