只在这里做简单的演示
ceotos7 环境
3个mon节点
3个osd节点
环境搭建我这里不再叙述
我们查看一下分区情况：
[root@ceph_1 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 100G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 99G 0 part
├─cl-root 253:0 0 50G 0 lvm /
├─cl-swap 253:1 0 7.8G 0 lvm [SWAP]
└─cl-home 253:2 0 41.2G 0 lvm /home
sdb 8:16 0 100G 0 disk
├─sdb1 8:17 0 95G 0 part /var/lib/ceph/osd/ceph-0
└─sdb2 8:18 0 5G 0 part
sdc 8:32 0 100G 0 disk
sr0 11:0 1 680M 0 rom
我们要把journal的分区又sdb2上挪到sdc盘上，sdc盘最好是ssd的盘，读写效率高
这里做之前我们先看一下各节点上sdb的分区
ceph_1节点上：
[root@ceph_1 ~]# sgdisk -i 1 /dev/sdb
Partition GUID code: 4FBD7E29-9D25-41B8-AFD0-062C0CEFF05D (Unknown)
Partition unique GUID: B78B5DF4-FF30-4F43-89A9-CF6FD951F9FE
First sector: 10487808 (at 5.0 GiB)
Last sector: 209715166 (at 100.0 GiB)
Partition size: 199227359 sectors (95.0 GiB)
Attribute flags: 0000000000000000
Partition name: 'ceph data'
[root@ceph_1 ~]# sgdisk -i 2 /dev/sdb
Partition GUID code: 45B0969E-9B03-4F30-B4C6-B4B80CEFF106 (Unknown)
Partition unique GUID: 4106357A-CB56-4099-AA4A-028431E91C11
First sector: 2048 (at 1024.0 KiB)
Last sector: 10487807 (at 5.0 GiB)
Partition size: 10485760 sectors (5.0 GiB)
Attribute flags: 0000000000000000
Partition name: 'ceph journal'
ceph_2节点上：
[root@ceph_2 ~]# sgdisk -i 1 /dev/sdb
Partition GUID code: 4FBD7E29-9D25-41B8-AFD0-062C0CEFF05D (Unknown)
Partition unique GUID: FA672CE1-E5E4-4FD3-8E42-9F453546D63D
First sector: 10487808 (at 5.0 GiB)
Last sector: 209715166 (at 100.0 GiB)
Partition size: 199227359 sectors (95.0 GiB)
Attribute flags: 0000000000000000
Partition name: 'ceph data'
[root@ceph_2 ~]# sgdisk -i 2 /dev/sdb
Partition GUID code: 45B0969E-9B03-4F30-B4C6-B4B80CEFF106 (Unknown)
Partition unique GUID: 9C35A75C-E1AA-49E5-8DC3-BFA01304889D
First sector: 2048 (at 1024.0 KiB)
Last sector: 10487807 (at 5.0 GiB)
Partition size: 10485760 sectors (5.0 GiB)
Attribute flags: 0000000000000000
Partition name: 'ceph journal'
节点3的就不看了，也是一样的
对比可以得出一个结论：
ceph-osd 分区的都有一个固定的uuid：4FBD7E29-9D25-41B8-AFD0-062C0CEFF05D
ceph-journal分区也有一个固定的uuid：45B0969E-9B03-4F30-B4C6-B4B80CEFF106
接下来就开始换盘的操作
首先先对sdc进行分区
[root@ceph_1 ~]# sgdisk -n 1:0:0 -c 1:"ceph journal" -t 1:45B0969E-9B03-4F30-B4C6-B4B80CEFF106 -g /dev/sdc ////-c 指定name -t指定uuid -g 转换为gpt格式
Creating new GPT entries.
The operation has completed successfully.
[root@ceph_1 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 100G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 99G 0 part
├─cl-root 253:0 0 50G 0 lvm /
├─cl-swap 253:1 0 7.8G 0 lvm [SWAP]
└─cl-home 253:2 0 41.2G 0 lvm /home
sdb 8:16 0 100G 0 disk
├─sdb1 8:17 0 95G 0 part /var/lib/ceph/osd/ceph-0
└─sdb2 8:18 0 5G 0 part
sdc 8:32 0 100G 0 disk
└─sdc1 8:33 0 100G 0 part
sr0 11:0 1 680M 0 rom
验证一下sdc是否按我们上面的uuid设置成功
[root@ceph_1 ~]# sgdisk -i 1 /dev/sdc
Partition GUID code: 45B0969E-9B03-4F30-B4C6-B4B80CEFF106 (Unknown)
Partition unique GUID: 2E7E105B-2C0E-4809-9B5B-B030D6B52F8E
First sector: 2048 (at 1024.0 KiB)
Last sector: 209715166 (at 100.0 GiB)
Partition size: 209713119 sectors (100.0 GiB)
Attribute flags: 0000000000000000
Partition name: 'ceph journal'
看一下各个磁盘对应的uuid，后面也需要用到这个
[root@ceph_1 ~]# ll /dev/disk/by-partuuid/
total 0
lrwxrwxrwx 1 root root 10 Oct 27 11:36 2e7e105b-2c0e-4809-9b5b-b030d6b52f8e -> ../../sdc1
lrwxrwxrwx 1 root root 10 Oct 27 11:26 4106357a-cb56-4099-aa4a-028431e91c11 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Oct 27 11:26 b78b5df4-ff30-4f43-89a9-cf6fd951f9fe -> ../../sdb1
看一下之前的journal是链接到哪个磁盘上的
[root@ceph_1 ~]# ll /var/lib/ceph/osd/ceph-0/journal
lrwxrwxrwx 1 ceph ceph 58 Oct 27 10:11 /var/lib/ceph/osd/ceph-0/journal -> /dev/disk/by-partuuid/4106357a-cb56-4099-aa4a-028431e91c11
由此可以看出之前的链接是指向sdb2分区上的，
我们要把journal分区sdb2换成sdc1，
第一步：设置策略 ceph osd set noout
[root@ceph_1 ~]# ceph osd set noout
set noout
第二步：停osd服务
systemctl stop ceph-osd@0
查看服务是否停掉：
[root@ceph_1 ~]# systemctl status ceph-osd@0
● ceph-osd@0.service - Ceph object storage daemon
Loaded: loaded (/usr/lib/systemd/system/ceph-osd@.service; enabled; vendor preset: disabled)
Active: inactive (dead) since Fri 2017-10-27 15:14:59 CST; 7s ago
Main PID: 1495 (code=exited, status=0/SUCCESS)
Oct 27 11:21:25 ceph_1 systemd[1]: Starting Ceph object storage daemon...
Oct 27 11:21:28 ceph_1 ceph-osd-prestart.sh[1103]: create-or-move updated item name 'osd.0' weight 0.0488 at location {host=ceph_1,root=default} to crush map
Oct 27 11:21:28 ceph_1 systemd[1]: Started Ceph object storage daemon.
Oct 27 11:21:28 ceph_1 ceph-osd[1495]: starting osd.0 at :/0 osd_data /var/lib/ceph/osd/ceph-0 /var/lib/ceph/osd/ceph-0/journal
Oct 27 11:21:29 ceph_1 ceph-osd[1495]: 2017-10-27 11:21:29.719086 7f0bc5064800 -1 osd.0 32 log_to_monitors {default=true}
Oct 27 15:14:57 ceph_1 systemd[1]: Stopping Ceph object storage daemon...
Oct 27 15:14:57 ceph_1 ceph-osd[1495]: 2017-10-27 15:14:57.666313 7f0b9d2f1700 -1 osd.0 38 *** Got signal Terminated ***
Oct 27 15:14:57 ceph_1 ceph-osd[1495]: 2017-10-27 15:14:57.727750 7f0b9d2f1700 -1 osd.0 38 shutdown
Oct 27 15:14:59 ceph_1 systemd[1]: Stopped Ceph object storage daemon.
第三步：删除之前的软连接，或者移除掉之前的软连接
[root@ceph_1 ~]# rm -rf /var/lib/ceph/osd/ceph-0/journal
第四步：创建软连接并赋予权限：
[root@ceph_1 ~]# ln -fs /dev/disk/by-partuuid/2e7e105b-2c0e-4809-9b5b-b030d6b52f8e /var/lib/ceph/osd/ceph-0/journal ///////2e7e105b-2c0e-4809-9b5b-b030d6b52f8e为sdc2的partuuid
[root@ceph_1 ~]# chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/journal
第五步：初始化journal
[root@ceph_1 ~]# ceph-osd --mkjournal -i 0
SG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2017-10-27 15:24:56.752101 7feb86a36800 -1 journal check: ondisk fsid 00000000-0000-0000-0000-000000000000 doesn't match expected b78b5df4-ff30-4f43-89a9-cf6fd951f9fe,
invalid (someone else's?) journalSG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2017-10-27 15:24:56.758849 7feb86a36800 -1 created new journal /var/lib/ceph/osd/ceph-0/journal for object store /var/lib/ceph/osd/ceph-0
第六步：启动osd服务并查看状态
[root@ceph_1 ~]# systemctl start ceph-osd@0
[root@ceph_1 ~]# ceph osd tree
ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY
-1 0.27809 root default
-2 0.09270 host ceph_1
0 0.09270 osd.0 up 1.00000 1.00000
-3 0.09270 host ceph_2
1 0.09270 osd.1 up 1.00000 1.00000
-4 0.09270 host ceph_3
2 0.09270 osd.2 up 1.00000 1.00000
[root@ceph_1 ~]#
第七步：取消策略
[root@ceph_1 ~]# ceph osd unset noout
unset noout
[root@ceph_1 ~]# ceph -w
cluster bc9c1346-a31c-436d-ac52-f5c21cddeb91
health HEALTH_OK
monmap e2: 3 mons at {ceph_1=10.10.10.25:6789/0,ceph_2=10.10.10.26:6789/0,ceph_3=10.10.10.27:6789/0}
election epoch 20, quorum 0,1,2 ceph_1,ceph_2,ceph_3
osdmap e44: 3 osds: 3 up, 3 in
flags sortbitwise,require_jewel_osds
pgmap v118: 64 pgs, 1 pools, 0 bytes data, 0 objects
105 MB used, 284 GB / 284 GB avail
64 active+clean
2017-10-27 15:27:29.084832 mon.0 [INF] HEALTH_OK