Showing posts with label Golden Gate. Show all posts
Showing posts with label Golden Gate. Show all posts

Sunday, July 28, 2013

Setting up High Availability for Golden Gate


The first step is to setup a virtual IP
The VIP needs to reside on the public network subnet. To find the subnet, you can do the below


crsctl stat res -p | grep -i .network

NETWORK = ora.net1.network

crsctl stat res -p | grep -i subnet
USR_ORA_SUBNET=10.2.170.128



Add the VIP to the /etc/hosts file


10.2.170.157 prdgg-vip.prd1.company.com prdgg-vip



Add the VIP into the cluster


GRID_HOME/bin/appvipcfg create -network=1 -ip=10.2.170.157 -vipname=prdgg-vip -user=root
usage: tail [+/-[n][lbc][f]] [file]
tail [+/-[n][l][r|f]] [file]
Production Copyright 2007, 2008, Oracle.All rights reserved
2013-05-29 13:51:03: Creating Resource Type
2013-05-29 13:51:03: Executing cmd: /u01/app/crs/11.2.0/grid/bin/crsctl add type app.appvip.type -basetype cluster_resource -file /u01/app/crs/11.2.0/grid/crs/template/appvip.type
2013-05-29 13:51:04: Create the Resource
2013-05-29 13:51:04: Executing cmd: /u01/app/crs/11.2.0/grid/bin/crsctl add resource prdgg-vip -type app.appvip.type -attr USR_ORA_VIP=10.2.170.157,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL='owner:root:rwx,pgrp:root:r-x,other::r--,user:root:r-x'



Check if the VIP has been added into the cluster


GRID_HOME/bin/crsctl stat res prdgg-vip
NAME=prdgg-vip
TYPE=app.appvip.type
TARGET=OFFLINE
STATE=OFFLINE



Give Oracle user the privilege.


GRID_HOME/bin/crsctl setperm resource prdgg-vip -u user:oracle:r-x



Start the VIP


GRID_HOME/bin/crsctl start res prdgg-vip
CRS-2672: Attempting to start 'prdgg-vip' on 'prdnode2'
CRS-2676: Start of 'prdgg-vip' on 'prdnode2' succeeded



Try switching the VIP between the nodes


GRID_HOME/bin/crsctl relocate res prdgg-vip
CRS-2673: Attempting to stop 'prdgg-vip' on 'prdnode2'
CRS-2677: Stop of 'prdgg-vip' on 'prdnode2' succeeded
CRS-2672: Attempting to start 'prdgg-vip' on 'prdnode1'
CRS-2676: Start of 'prdgg-vip' on 'prdnode1' succeeded




GRID_HOME/bin/crsctl stat res prdgg-vip
NAME=prdgg-vip
TYPE=app.appvip.type
TARGET=ONLINE
STATE=ONLINE on prdnode1



Create the action script


#!/bin/sh
set -x
#GG auto restart
OGG_HOME=/u01/app/ogg/GG11g
export OGG_HOME;
start_delay_secs=5
#Include the GoldenGate home in the library path to start GGSCI
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${OGG_HOME}
#set the oracle home to the database to ensure GoldenGate will get the
#right environment settings to be able to connect to the database
ORACLE_HOME=/u01/app/oracle/product/11.2.0/db1
JAVA_HOME=/usr/java/default
PATH=$PATH:/usr/java/default/bin
#check_process validates that a manager process is running at the PID
#that GoldenGate specifies.

check_process () {
if ( [ -f "${OGG_HOME}/dirpcs/MGR.pcm" ] )
then
pid=`cut -f8 "${OGG_HOME}/dirpcs/MGR.pcm"`
if [ ${pid} = `ps -e |grep ${pid} |grep mgr |cut -d " " -f2` ]
then
#manager process is running on the PID . exit success
exit 0
else
if [ ${pid} = `ps -e |grep ${pid} |grep mgr |cut -d " " -f1` ]
then
#manager process is running on the PID . exit success
exit 0
else
#manager process is not running on the PID
exit 1
fi
fi
else
#manager is not running because there is no PID file
exit 1
fi
}

#call_ggsci is a generic routine that executes a ggsci command
call_ggsci () {
ggsci_command=$1
ggsci_output=`${OGG_HOME}/ggsci << EOF
${ggsci_command}
exit
EOF`
}

case $1 in
'start')
#start manager
call_ggsci 'start manager'
#there is a small delay between issuing the start manager command
#and the process being spawned on the OS . wait before checking
sleep ${start_delay_secs}
#Start the Extract and Replicat processes
call_ggsci 'start *'
#check whether manager is running and exit accordingly
check_process
;;

'stop')
#attempt a clean stop for all non-manager processes
call_ggsci 'stop er *'
#ensure everything is stopped
call_ggsci 'stop er *!'
#stop manager without (y/n) confirmation
call_ggsci 'stop manager!'
#exit success
exit 0
;;

'check')
check_process
;;

'clean')
#attempt a clean stop for all non-manager processes
call_ggsci 'stop er *'
#ensure everything is stopped
call_ggsci 'stop er *!'
#in case there are lingering processes
call_ggsci 'kill er *'
#stop manager without (y/n) confirmation
call_ggsci 'stop manager!'
#exit success
exit 0
;;

'abort')
#ensure everything is stopped
call_ggsci 'stop er *!'
#in case there are lingering processes
call_ggsci 'kill er *'
#stop manager without (y/n) confirmation
call_ggsci 'stop manager!'
#exit success
exit 0
;;
esac



[prdnode1.prd1.company: /u01/oraogg/prdnode/scripts] ls -ltr 11gr2_gg_action.scr
-rwxr-xr-x 1 oracle oinstall 2353 May 29 13:59 11gr2_gg_action.scr



Add the GoldenGate Application to the cluster


GRID_HOME/bin/crsctl add resource ggateapp -type cluster_resource -attr "ACTION_SCRIPT=/u01/oraogg/prdnode/scripts/11gr2_gg_action.scr,CHECK_INTERVAL=30, START_DEPENDENCIES='hard(prdgg-vip)pullup(prdgg-vip)', STOP_DEPENDENCIES='hard(prdgg-vip)'"



Check the GoldenGate Application in the cluster


GRID_HOME/bin/crsctl stat res ggateapp -p
NAME=ggateapp
TYPE=cluster_resource
ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=/u01/oraogg/prdnode/scripts/11gr2_gg_action.scr
ACTIVE_PLACEMENT=0
AGENT_FILENAME=%CRS_HOME%/bin/scriptagent
AUTO_START=restore
CARDINALITY=1
CHECK_INTERVAL=30
DEFAULT_TEMPLATE=
DEGREE=1
DESCRIPTION=
ENABLED=1
FAILOVER_DELAY=0
FAILURE_INTERVAL=0
FAILURE_THRESHOLD=0
HOSTING_MEMBERS=
LOAD=1
LOGGING_LEVEL=1
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
PLACEMENT=balanced
PROFILE_CHANGE_TEMPLATE=
RESTART_ATTEMPTS=1
SCRIPT_TIMEOUT=60
SERVER_POOLS=
START_DEPENDENCIES=hard(prdgg-vip)pullup(prdgg-vip)
START_TIMEOUT=0
STATE_CHANGE_TEMPLATE=
STOP_DEPENDENCIES=hard(prdgg-vip)
STOP_TIMEOUT=0
UPTIME_THRESHOLD=1h



Start the cluster resource on the node where GoldenGate is running.


GRID_HOME/bin/crsctl start resource ggateapp -n prdnode1
CRS-2672: Attempting to start 'ggateapp' on 'prdnode1'
CRS-2676: Start of 'ggateapp' on 'prdnode1' succeeded



Check if it running


GRID_HOME/bin/crsctl stat res ggateapp
NAME=ggateapp
TYPE=cluster_resource
TARGET=ONLINE
STATE=ONLINE on prdnode1


Tuesday, June 4, 2013

OGG-01031 Pump Abending

A fairly common occurance in the Golden Gate world. The pump abends as the target destination becomes full.
In this scenario, simply clearing space on the drive will not restart the pump.
The pump complains as below


2013-05-14 10:41:42 INFO OGG-00993 Oracle GoldenGate Capture for Oracle, p1ppppd1.prm: EXTRACT P1PPPPD1 started. 2013-05-14 10:41:47 INFO OGG-01226 Oracle GoldenGate Capture for Oracle, p1ppppd1.prm: Socket buffer size set to 27985 (flush size 27985). 2013-05-14 10:41:58 ERROR OGG-01031 Oracle GoldenGate Capture for Oracle, p1ppppd1.prm: There is a problem in network communication, a remote file problem, encryption keys for target and source do not match (if using ENCRYPT) or an unknown error. (Reply received is Unable to open file "/u01/oraogg/dirdat/rp/rp000014" (error 13, Permission denied)). 2013-05-14 10:41:58 ERROR OGG-01668 Oracle GoldenGate Capture for Oracle, p1ppppd1.prm: PROCESS ABENDING.

We tell the pump to Rollover onto a new trace file.


alter extract P1PPPPD1 etrollover
2013-05-14 10:57:19 INFO OGG-01520 Oracle GoldenGate Command Interpreter for Oracle: Rollover performed. For each affected output trail of Version 10 or higher format, after starting the source extract, issue ALTER EXTSEQNO for that trail's reader (either pump EXTRACT or REPLICAT) to move the reader's scan to the new trail file; it will not happen automatically.


Run info pumpname to get the new trail file name.

At the target end, the replicat should have parsed all the data received until the drive became full and data stopped coming through.
To let the target know to switch to the new file, use below command.


alter replicat R1PPPPD2 ,EXTSEQNO 15 , EXTRBA 0 --- replicat was reading the seqno 14. moved to 15.