Monday 19 October 2015

Enabling Fast Start Failover and the Observer

Enabling Fast Start Failover and the Observer
Ensure standby redo logs are configured on both primary and standby databases

Fast-start failover enables Data Guard to rapidly and automatically fail over to a previouslychosen standby database without requiring manual intervention. This feature increases the availability of your database in the event of a disaster by reducing the need for you to performa failover operation manually.
The observer is an Oracle Call Interface (OCI) client-side component that typically runs on aseparate computer and monitors the availability of the primary database

When Does Fast-Start Failover Occur?
Fast-start failover occurs when any of the following conditions are met:
• Loss of connectivity between both the primary database and the observer and between the primary database and the fast-start failover target standby database exceeds the fast-start failover threshold.
• The database health-check mechanism determines any of the following (as optionally configured):
- A data file is offline because of a write error.
- Dictionary corruption of a critical database object occurs.
- Control file is permanently damaged because of a disk failure.
- LGWR is unable to write to any member of the log group because of an I/O error.
- Archiver is unable to archive a redo log because the device is full or unavailable.
• An instance crash occurs for a single-instance database.
• All instances of a Real Application Clusters (RAC) primary database crash.
• Shutdown abort of the primary database occurs.
• An application initiates a fast-start failover by calling the DBMS_DG.INITIATE_FS_FAILOVER function. 

SQL> SELECT GROUP#, BYTES/1024, STATUS FROM V$STANDBY_LOG;
  
Ensure the LogXptMode Property is set to SYNC

DGMGRL> EDIT DATABASE ORCL SET PROPERTY 'LogXptMode'='SYNC';
Property "LogXptMode" updated
DGMGRL>

DGMGRL> EDIT DATABASE STDY SET PROPERTY 'LogXptMode'='SYNC';
Property "LogXptMode" updated
DGMGRL>

Alert log

ALTER SYSTEM SET log_archive_dest_2='service="stdy"','LGWR SYNC AFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300 db_unique_name="stdy" net_timeout=30','valid_for=(all_logfiles,primary_role)' SCOPE=BOTH;
ALTER SYSTEM SWITCH ALL LOGFILE start (orcl)
Thread 1 cannot allocate new log, sequence 72
Checkpoint not complete
  Current log# 2 seq# 71 mem# 0: +DATA/orcl/onlinelog/group_2.264.719142951
  Current log# 2 seq# 71 mem# 1: +DATA/orcl/onlinelog/group_2.265.719142957
2012-08-28 17:34:32.652000 +05:30
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
******************************************************************
LGWR: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2
******************************************************************
NSS2 started with pid=43, OS id=29520
2012-08-28 17:34:36.838000 +05:30
LGWR: Standby redo logfile selected for thread 1 sequence 72 for destination LOG_ARCHIVE_DEST_2
ALTER SYSTEM SWITCH ALL LOGFILE complete (orcl)
Thread 1 advanced to log sequence 72 (LGWR switch)
  Current log# 3 seq# 72 mem# 0: +DATA/orcl/onlinelog/group_3.266.719142961
  Current log# 3 seq# 72 mem# 1: +DATA/orcl/onlinelog/group_3.267.719142965
Archived Log entry 105 added for thread 1 sequence 71 ID 0x4a53118e dest 1:
2012-08-28 17:34:39.158000 +05:30

Specify the FastStartFailoverTarget property

DGMGRL> EDIT DATABASE ORCL SET PROPERTY FastStartFailoverTarget='STDY';

Property "faststartfailovertarget" updated

DGMGRL> EDIT DATABASE STDY SET PROPERTY FastStartFailoverTarget='ORCL';

Property "faststartfailovertarget" updated
DGMGRL>

Upgrade the protection mode to MAXAVAILABILITY, if necessary.

DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MAXAVAILABILITY;
Succeeded.
DGMGRL>


ENABLE FAST START FAILOVER

DGMGRL> show configuration;
Configuration
Name: orcl
Enabled: YES
Protection Mode: MaxAvailability
Fast-Start Failover: DISABLED
Databases:
orcl - Primary database
stdy - Physical standby database
Current status for "orcl":
SUCCESS

DGMGRL> SHOW DATABASE STDY;

Database
  Name:            stdy
  Role:            PHYSICAL STANDBY
  Enabled:         YES
  Intended State:  APPLY-ON
  Instance(s):
    stdy
 Current status for "stdy":
SUCCESS

DGMGRL> ENABLE FAST_START FAILOVER;
Enabled.
DGMGRL>

Alert log

2012-03-04 17:25:35.344000 +05:30
FSFP started with pid=45, OS id=3550

DGMGRL> SHOW CONFIGURATION;
Configuration - orcl_stdy

  Protection Mode: MaxAvailability
  Databases:
    orcl - Primary database
      Warning: ORA-16819: fast-start failover observer not started
    stdy - (*) Physical standby database
      Warning: ORA-16819: fast-start failover observer not started

Fast-Start Failover: ENABLED
Configuration Status:
WARNING

Start the observer

From Another Terminal Start Observer and let it run connect. May need to bounce Listener once.

$ dgmgrl
DGMGRL> connect sys/oracle
Connected.
DGMGRL>
DGMGRL> start observer
Observer started

OR

To start Observer in background

$ dgmgrl -logfile ./observer_log

DGMGRL> connect sys/oracle
Connected.
DGMGRL> start observer

From another session

$ tail -100f observer_log

$ ps -ef|grep observer
oracle   24282 23873  0 09:11 pts/6    00:00:00 rlwrap dgmgrl sys/oracle@orcl start observer
oracle   24283 24282  0 09:11 pts/7    00:00:00 dgmgrl                 start observer
oracle   24320 23873  0 09:12 pts/6    00:00:00 tail -100f observer_log
oracle   24363 24338  0 09:12 pts/8    00:00:00 grep observer
[oracle@server ~]$

Verify the fast-start failover configuration

DGMGRL> SHOW CONFIGURATION VERBOSE;

Configuration - orcl_stdy

  Protection Mode: MaxAvailability
  Databases:
    orcl - Primary database
    stdy - (*) Physical standby database
   (*) Fast-Start Failover target
 Fast-Start Failover: ENABLED
   Threshold:        30 seconds
  Target:           stdy
  Observer:         11gr2.rhel5.com
  Lag Limit:        30 seconds (not in use)
  Shutdown Primary: TRUE
  Auto-reinstate:   TRUE

Configuration Status:
SUCCESS

DGMGRL> SHOW DATABASE STDY;

Database - stdy
   Role:            PHYSICAL STANDBY
  Intended State:  APPLY-ON
  Transport Lag:   0 seconds
  Apply Lag:       0 seconds
  Real Time Query: OFF
  Instance(s):
    stdy

Database Status:
SUCCESS
 
Perform Fast_Start_Failover

DGMGRL> show fast_start failover;

Fast-Start Failover: ENABLED
  Threshold:        30 seconds
  Target:           stdy
  Observer:         11gr2.rhel5.com
  Lag Limit:        30 seconds (not in use)
  Shutdown Primary: TRUE
  Auto-reinstate:   TRUE
Configurable Failover Conditions
  Health Conditions:
    Corrupted Controlfile          YES
    Corrupted Dictionary           YES
    Inaccessible Logfile            NO
    Stuck Archiver                  NO
    Datafile Offline               YES
  Oracle Error Conditions:
    (none)

DGMGRL> EDIT CONFIGURATION SET PROPERTY FASTSTARTFAILOVERLAGLIMIT=45;
Property "faststartfailoverlaglimit" updated

DGMGRL> show fast_start failover;

Fast-Start Failover: ENABLED
  Threshold:        30 seconds
  Target:           stdy
  Observer:         11gr2.rhel5.com
  Lag Limit:        45 seconds (not in use)
  Shutdown Primary: TRUE
  Auto-reinstate:   TRUE
Configurable Failover Conditions
  Health Conditions:
    Corrupted Controlfile          YES
    Corrupted Dictionary           YES
    Inaccessible Logfile            NO
    Stuck Archiver                  NO
    Datafile Offline               YES
  Oracle Error Conditions:
    (none)

SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PRIMARY

On Primary

SQL> shutdown abort

On Standby
SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PHYSICAL STANDBY

See the Observer Terminal
DGMGRL> start observer
Observer started
10:28:52.96  Sunday, June 21, 2009
Initiating fast-start failover to database "stdy"...
Performing failover NOW, please wait...
Failover succeeded, new primary is "stdy"
12:46:39.08  Sunday, June 21, 2009
Hangup

 On Standby

SQL> select database_role from v$database;
DATABASE_ROLE
----------------
PRIMARY

Before Failover  on Standby
SQL> SELECT fs_failover_status, fs_failover_current_target, fs_failover_threshold, fs_failover_observer_present, fs_failover_observer_host FROM v$database;

FS_FAILOVER_STATUS    FS_FAILOVER_CURRENT_TARGET     FS_FAILOVER_THRESHOLD FS_FAIL FS_FAILOVER_OBSERVER_HOST
--------------------- ------------------------------ --------------------- ------- ---------------------------------------------------------
SYNCHRONIZED          stdy                                              30 YES   server.rhel5.com

Before Reinstating on New Primary but After Failover

SQL> SELECT fs_failover_status, fs_failover_current_target, fs_failover_threshold, fs_failover_observer_present, fs_failover_observer_host FROM v$database;

FS_FAILOVER_STATUS     FS_FAILOVER_CURRENT_TARGET     FS_FAILOVER_THRESHOLD FS_FAIL FS_FAILOVER_OBSERVER_HOST
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
REINSTATE REQUIRED     orcl                                              30 YES     server.rhel5.com

On Old Primary

SQL> startup
ORACLE instance started.
Total System Global Area  213909504 bytes
Fixed Size                  1266656 bytes
Variable Size              83889184 bytes
Database Buffers          125829120 bytes
Redo Buffers                2924544 bytes
Database mounted.
ORA-16649: database will open after Data Guard broker has evaluated Fast-Start Failover status

SQL> SQL> SELECT fs_failover_status, fs_failover_current_target, fs_failover_threshold, fs_failover_observer_present, fs_failover_observer_host FROM v$database;

FS_FAILOVER_STATUS     FS_FAILOVER_CURRENT_TARGET     FS_FAILOVER_THRESHOLD FS_FAIL FS_FAILOVER_OBSERVER_HOST
---------------------- ------------------------------ --------------------- ------- --------------------------------------------------------------------------------
BYSTANDER                                                                30 YES

SQL> show parameter unique

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_unique_name                       string      ORCL

SQL> select database_role from v$database;

DATABASE_ROLE
----------------
PHYSICAL STANDBY

SQL> select open_mode from v$database;

OPEN_MODE
----------
READ ONLY

SQL>

On Observer Host

DGMGRL> connect sys/oracle@ORCL
Connected.
DGMGRL> START OBSERVER
Error: ORA-16795: database resource guard detects that database re-creation is required
Observer started
11:55:51.63  Sunday, June 21, 2009
Initiating reinstatement for database "orcl"...
Reinstating database "orcl", please wait...
Operation requires shutdown of instance "ORCL" on database "orcl"
Shutting down instance "ORCL"...
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
Operation requires startup of instance "ORCL" on database "orcl"
Starting instance "ORCL"...
ORACLE instance started.
Database mounted.
Continuing to reinstate database "orcl" ...
Reinstatement of database "orcl" succeeded
11:57:15.24  Sunday, June 21, 2009
Hangup

After Reinstating on New Primary

SQL> SELECT fs_failover_status, fs_failover_current_target, fs_failover_threshold, fs_failover_observer_present, fs_failover_observer_host FROM v$database;

FS_FAILOVER_STATUS    FS_FAILOVER_CURRENT_TARGET     FS_FAILOVER_THRESHOLD FS_FAIL FS_FAILOVER_OBSERVER_HOST
--------------------- ------------------------------ --------------------- ------- ---------------------------------------------------------
SYNCHRONIZED          orcl                                              30 NO   linux.com

On Old Primary

SQL> select database_role from v$database;
DATABASE_ROLE
----------------
PHYSICAL STANDBY

SQL> show parameter db_unique
NAME                                 TYPE        VALUE
------------------------------------ -----------
db_unique_name                       string      ORCL

SQL>

Dataguard Broker Configuration after reinstating
DGMGRL> connect sys/oracle@ORCL
Connected.
DGMGRL> SHOW CONFIGURATION;
Configuration
  Name:                orcl
  Enabled:             YES
  Protection Mode:     MaxAvailability
  Fast-Start Failover: ENABLED
  Databases:
    orcl - Physical standby database
         - Fast-Start Failover target
    stdy - Primary database
Current status for "orcl":
SUCCESS
DGMGRL>

Disable Fast Start Failover

DISABLE FAST-START FAILOVER;


Thanks (C0mmands Plz)

No comments:

Post a Comment