goldengate初始化

时间:2023-03-08 20:09:44

对丢弃已久的goldengate环境重新配置,使其重新开始跑起来

环境是一个主机上的两个库,都是单机,所以也就没配pump进程了,trail file都是在一个文件夹下的,extract写trail file到这,然后replicate在这读取trail file。

先来看下大致的环境情况:VDBSL3--->VDBCB3

源库:-----------VDBSL3(dom schema )以下几张表要去用到ogg,

Table dom.YFS_ITEM;

Table dom.YFS_ITEM_ALIAS;

Table dom.YFS_INVENTORY_ITEM;

Table dom.YFS_INVENTORY_ALERTS;

Table dom.YFS_INVENTORY_SUPPLY;

Table dom.EXTN_STORE_REGION_CACHE;

目标库:----------VDBCB3(ce_hub schema):

MAP dom.YFS_ITEM, TARGET ce_hub.YFS_ITEM;

MAP dom.YFS_ITEM_ALIAS, TARGET ce_hub.YFS_ITEM_ALIAS;

MAP dom.YFS_INVENTORY_ITEM, TARGET ce_hub.YFS_INVENTORY_ITEM;

MAP dom.YFS_INVENTORY_ALERTS, TARGET ce_hub.YFS_INVENTORY_ALERTS;

MAP dom.YFS_INVENTORY_SUPPLY, TARGET ce_hub.YFS_INVENTORY_SUPPLY;

MAP dom.EXTN_STORE_REGION_CACHE, TARGET ce_hub.EXTN_STORE_REGION_CACHE;

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com) 1>

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING

EXTRACT     ABENDED     E_ICUB3     00:00:00      2122:38:24

REPLICAT    ABENDED     R_ICUB3     00:00:07      2155:36:01  可见废弃很久了

===================================================================================

现在开始重新配置了

把extract和replicate进程delete掉,清下相关的rpt file,dsc file,trail file

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com) 5> dblogin USERID ggadmin, PASSWORD xxx

Successfully logged into database.

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com as ggadmin@VDBSL3) 7> delete extract E_ICUB3

Deleted EXTRACT E_ICUB3.

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com as ggadmin@VDBCB3) 3> delete replicat R_ICUB3

Deleted REPLICAT R_ICUB3.

Delete掉之后,清除一些文件,主要是dirrpt下的rpt和dsc file和dirdat下的trail file,当然还可以去清下gglog dump file

做完这些基本上就可以了,然后replicate那边去对应的数据库里看看GGADMIN.GG_CHECKPOINT这张表是否还有这个replicate的记录,理论上是应该没有的,如果有的话,那么把记录delete掉

=================================================================================================

接下来就是做下exp和impdp的操作,在做这之前,因为这个是delphix database,定期会被刷新的,先来检查一些必要的参数

SELECT supplemental_log_data_min, force_logging FROM  v$database;

ALTER DATABASE FORCE LOGGING;

alter database add supplemental log data;

alter system switch logfile;

alter system set enable_goldengate_replication=TRUE scope=both;      (this parameter used in 11.2.0.4)

顺便建下directory

create directory DATA_PUMP_DIR1 as '/u01/home/oracle/datapump/vdbsl3';

然后的话准备添加extract进程并起起来,如下:

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com) 2> obey /u01/app/ggadmin/product/12.1.2.1.0/dirprm/e_icub3.oby

也可以不执行obey这个文件,手动执行这几条命令

more /u01/app/ggadmin/product/12.1.2.1.0/dirprm/e_icub3.oby

-- Create the change data capture extract, and local extract trail of 1024mb

dblogin userid ggadmin, password xxxxx

add extract e_icub3, tranlog, begin now

add rmttrail /u01/app/ggadmin/product/12.1.2.1.0/dirdat/extract/VDBCB3/e1, extract e_icub3 , megabytes 1024

然后的话就准备起起来,但是要注意的是数据库当前不要有long running transaction在跑,可以用如下命令确认当前的数据库事务情况:

select min(start_time) from gv$transaction;

select count(*) from gv$transaction;

select t.start_time, t.xidusn||'.'||t.xidslot||'.'||t.xidsqn xid, s.status, s.sid,s.serial#,s.username,s.status,s.schemaname, decode(s.sql_id,null,s.prev_sql_id) sqlid, decode(s.sql_child_number,null,s.prev_child_number) child from v$transaction t, v$session s where s.saddr = t.ses_addr order by t.start_time ;

要是确认好了,没有长事务在跑的话,就把extract起起来吧,然后记一下scn点,待会expdp根据这个scn点导出,start replicat也是根据这个时间点

源库执行scn查询:

select to_char(current_scn) from gv$database;

OR

       select
to_char(dbms_flashback.get_system_change_number) from dual;

主要是要保证在scn点抓取的时候没有长shiwu6在跑,那种典型的时间短的OLTP事务可以忽略,因为extract开始了就一直抓取数据库的DML变化,然后目标库的数据初始化时以这个scn开始的,所以只要在抓scn的时候没有长事务,那么两边是可以保持一致的。如果有长事务,这个可能就会在目标库里miss掉!

在原库vdbsl3上做expdp

expdp \'/ as sysdba\'  
parfile=gg_reint_1213.par

oracle@delphix-vdb-n-3:VDBSL3:/u01/home/oracle/datapump/vdbsl3
$ more gg_reint_1213.par

directory=DATA_PUMP_DIR1

dumpfile=gg_reint_1213_%U.dmp

logfile=ogg_1213.log

cluster=N

parallel=8

Compression=ALL

tables=dom.YFS_ITEM,dom.YFS_ITEM_ALIAS,dom.YFS_INVENTORY_ITEM,dom.YFS_INVENTORY_ALERTS,dom.YFS_INVENTORY_SUPPLY,dom.EXTN_STORE_REGION_CACHE

flashback_scn=14193095280953

(可以加上grants=n)

导出之后,便可以做导入了

impdp  \'/ as
sysdba\'   parfile=ogg_1213_imp.par
remap_schema=dom:ce_hub
remap_tablespace='(DOM_DATA_HIO:ICUB_DATA,DOM_DATA:ICUB_DATA,DOM_INDEX_HIO:ICUB_INDEX)'

oracle@delphix-vdb-n-3:VDBCB3:/u01/home/oracle/datapump/vdbsl3
$ more ogg_1213_imp.par

directory=DATA_PUMP_DIR1

dumpfile=gg_reint_1213_%U.dmp

logfile=gg_imp_1213.log

cluster=N

parallel=8

tables=dom.YFS_ITEM,dom.YFS_ITEM_ALIAS,dom.YFS_INVENTORY_ITEM,dom.YFS_INVENTORY_ALERTS,dom.YFS_INVENTORY_SUPPLY,dom.EXTN_STORE_REGION_CACHE

table_exists_action=replace

和以前的expdp/impdp一样,需要注意的是表对应的表空间是不是存在,需不需要remap或者新建表空间

在这里我是用了remap_tablespace重定向三个表空间

如下是查看表空间的方法:

源库上:

SQL> select owner,table_name,TABLESPACE_NAME from
dba_tables

where table_name in
('YFS_ITEM','YFS_ITEM_ALIAS','YFS_INVENTORY_ITEM','YFS_INVENTORY_ALERTS','YFS_INVENTORY_SUPPLY','EXTN_STORE_REGION_CACHE');

OWNER                          TABLE_NAME                             TABLESPACE_NAME

------------------------------
------------------------------------------------

DOM                           
YFS_INVENTORY_SUPPLY                 DOM_DATA_HIO

DOM                           
YFS_INVENTORY_ITEM                    DOM_DATA_HIO

DOM                            YFS_ITEM                                        DOM_DATA_HIO

DOM                            YFS_ITEM_ALIAS                               DOM_DATA

DOM                           
EXTN_STORE_REGION_CACHE           DOM_DATA

DOM                            YFS_INVENTORY_ALERTS                   DOM_DATA

6 rows selected.

select TABLESPACE_NAME,table_name,index_name,owner from
dba_indexes

where table_name in
('YFS_ITEM','YFS_ITEM_ALIAS','YFS_INVENTORY_ITEM','YFS_INVENTORY_ALERTS','YFS_INVENTORY_SUPPLY','EXTN_STORE_REGION_CACHE');

TABLESPACE_NAME                TABLE_NAME                     INDEX_NAME                     OWNER

------------------------------
------------------------------ ------------------------------
------------------------------

DOM_DATA                      
EXTN_STORE_REGION_CACHE       
ST_R_CACHE_PK                  DOM

DOM_DATA                       YFS_INVENTORY_ALERTS           SYS_IL0000228939C00018$$       DOM

DOM_INDEX                     
EXTN_STORE_REGION_CACHE       
EXTN_STORE_REGION_CACHE_I1     DOM

DOM_INDEX                      YFS_INVENTORY_SUPPLY           EXTN_YFS_INVENTORY_SUPPLY_I3   DOM

DOM_INDEX                      YFS_INVENTORY_SUPPLY           YFS_INVENTORY_SUPPLY_I3        DOM

DOM_INDEX                      YFS_ITEM_ALIAS                 EXTN_YFS_ITEM_ALIAS_I3         DOM

DOM_INDEX                      YFS_ITEM_ALIAS                 YFS_ITEM_ALIAS_PK              DOM

DOM_INDEX                      YFS_ITEM_ALIAS                 YFS_ITEM_ALIAS_I2              DOM

DOM_INDEX                      YFS_ITEM_ALIAS                 YFS_ITEM_ALIAS_I1              DOM

DOM_INDEX                      YFS_INVENTORY_ALERTS           YFS_INVENTORY_ALERTS_PK        DOM

DOM_INDEX                      YFS_INVENTORY_ALERTS           YFS_INVENTORY_ALERTS_I4        DOM

DOM_INDEX                 
    YFS_INVENTORY_ALERTS           YFS_INVENTORY_ALERTS_I3        DOM

DOM_INDEX                      YFS_INVENTORY_ALERTS           YFS_INVENTORY_ALERTS_I2        DOM

DOM_INDEX                      YFS_INVENTORY_ALERTS           YFS_INVENTORY_ALERTS_I1        DOM

DOM_INDEX_HIO                  YFS_INVENTORY_SUPPLY           YFS_INVENTORY_SUPPLY_PK        DOM

DOM_INDEX_HIO                  YFS_INVENTORY_SUPPLY           YFS_INVENTORY_SUPPLY_I2        DOM

DOM_INDEX_HIO                  YFS_INVENTORY_SUPPLY           YFS_INVENTORY_SUPPLY_I1        DOM

DOM_INDEX_HIO                  YFS_ITEM                       EXTN_YFS_ITEM_I12              DOM

DOM_INDEX_HIO                  YFS_ITEM                       EXTN_YFS_ITEM_I11              DOM

DOM_INDEX_HIO                  YFS_ITEM                       EXTN_YFS_ITEM_I10              DOM

DOM_INDEX_HIO                  YFS_ITEM                       YFS_ITEM_PK                    DOM

DOM_INDEX_HIO                  YFS_ITEM                       YFS_ITEM_I6                    DOM

DOM_INDEX_HIO                  YFS_ITEM                       YFS_ITEM_I5                    DOM

DOM_INDEX_HIO                  YFS_ITEM                       YFS_ITEM_I4                    DOM

DOM_INDEX_HIO                  YFS_ITEM                       YFS_ITEM_I3                    DOM

DOM_INDEX_HIO                  YFS_ITEM                       YFS_ITEM_I2                    DOM

DOM_INDEX_HIO                  YFS_ITEM                       YFS_ITEM_I1                    DOM

DOM_INDEX_HIO                  YFS_ITEM                       EXTN_YFS_ITEM_I9               DOM

DOM_INDEX_HIO                  YFS_INVENTORY_ITEM             YFS_INVENTORY_ITEM_PK          DOM

DOM_INDEX_HIO                  YFS_INVENTORY_ITEM             YFS_INVENTORY_ITEM_I1          DOM

30 rows selected.

目标库执行:

select distinct TABLESPACE_NAME from dba_tablespaces;

TABLESPACE_NAME

------------------------------

GGADMIN_DATA

DOM_INDEX

SYSAUX

UNDOTBS1

TEMP

USERS

SYSTEM

NIKE_AUDIT_TBS

ICUB_DATA

ICUB_INDEX

UNDOTBS2

11 rows selected.

可对比发现有几个表空间在目标库是不存在的,需要remap_tablespace

当然也可以用如下的方法来看表空间的问题:

impdp
\'/ as sysdba\'  directory=DATA_PUMP_DIR1
dumpfile=gg_reint_1213_%U.dmp sqlfile=myddl.sql

grep
TABLESPACE myddl.sql | sort | uniq | awk -F" " '{print $2}' | uniq

"DOM_DATA"

"DOM_DATA_HIO"

"DOM_DATA"

"DOM_INDEX_HIO"

"DOM_INDEX"

无论怎么看,总而让impdp成功即可

在impdp之后,则新建下replicate进程,并start
replicat aftercsn起起来,如下:

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com) 2> obey
/u01/app/ggadmin/product/12.1.2.1.0/dirprm/r_icub4.oby

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com) 3> dblogin userid
ggadmin, password eur3k2

Successfully logged into database.

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com as ggadmin@VDBCB3)
4> add replicat r_icub3, exttrail /u01/app/ggadmin/product/12.1.2.1.0/dirdat/extract/VDBCB3/e1

REPLICAT added.

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com as ggadmin@VDBCB3)
5> info all

Program     Status      Group     
 Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING

EXTRACT    
RUNNING     E_ICUB3     00:00:00      00:00:09

REPLICAT   
STOPPED     R_ICUB3     00:00:00      00:00:07

GGSCI (delphix-vdb-n-3.va2.b2c.nike.com as ggadmin@VDBCB3)
6> start R_ICUB3 aftercsn 14193095280953

Sending START request to MANAGER ...

REPLICAT R_ICUB3 starting

最后的话要是数据库事务量比较多的话,是可以及时看到replicate的RBA在增长的,或者stats可以看到有DML的操作数目

当然这里的库很闲,自己可以拿相关的表做做DML去测试一下整体功能,这里就省略了。

========================       ENDED                ===============================================