Configuration RACNODES
----------*********----------
vi /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
# Public Network - (eth0)
192.168.1.35 rac1.dba.com rac1
192.168.1.36 rac2.dba.com rac2
# Private Interconnect - (eth1)
192.168.0.135 rac1-priv.dba.com rac1-priv
192.168.0.136 rac2-priv.dba.com rac2-priv
# Public Virtual IP (VIP) addresses for - (eth0)
192.168.1.202 rac1-vip.dba.com rac1-vip
192.168.1.203 rac2-vip.dba.com rac2-vip
# Scan IP
192.168.1.188 cluster-scan
#Openfile Ip
192.168.1.100 san.dba.com san
5.1.2. Edit the -sysctl.conf- of orcl1 with the following values.
vi /etc/sysctl.conf
-------------------
kernel.core_uses_pid = 1
kernel.shmmax=4294967295
kernel.sem=256 32000 100 142
fs.file-max = 6815744
kernel.msgmnb = 65535
kernel.msgmni = 2878
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.tcp_wmem = 262144 262144 262144
net.ipv4.tcp_rmem = 262144 262144 262144
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
kernel.shmmni = 4096
5.1.3. Edit the -limits.conf- file of orcl1 with the following values.
vi /etc/security/limits.conf
----------------------------
hard nofile 65535
soft nofile 4096
hard nproc 16384
soft nproc 2047
grid hard nofile 65536
oracle hard nofile 65536
Run the following command to change the current kernel parameters.
/sbin/sysctl -p
5.1.4. Create groups and users
------------------------------
groupadd -g 501 oinstall
groupadd -g 502 dba
useradd -u 503 -g oinstall -G dba oracle
passwd oracle
5.1.5. Install missing packages
-------------------------------
Caution: If you are getting warning
cd /etc/pki/rpm-gpg/
rpm -import RPM-GPG-KEY
If already rpm installed with warning use --force after -Uvh
example rpm -Uvh --force binutils-2.*
# cd /stage/54
# more run54.txt
Copy the output and paste it.
Ensure all rpms installed with out any warning messages.
-------------------------------------------------------------
Create the directories in which the Oracle software will be installed.
mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/oracle/product/11.2.0/db_1
chown -R oracle:oinstall /u01
chmod -R 775 /u01/
Disable NTP
-----------
Either configure NTP, or make sure it is not configured so the Oracle Cluster Time Synchronization Service (ctssd) can synchronize the times of the RAC nodes. If you want to deconfigure NTP do the following.
# service ntpd stop
Shutting down ntpd: [ OK ]
# chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.orig
# rm /var/run/ntpd.pid
Login as oracle and set the following in .bash_profile
# Oracle Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_HOSTNAME=rac2.dba.com; export ORACLE_HOSTNAME
ORACLE_UNQNAME=sales; export ORACLE_UNQNAME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
GRID_HOME=/u01/app/11.2.0/grid; export GRID_HOME
DB_HOME=$ORACLE_BASE/product/11.2.0/db_1; export DB_HOME
ORACLE_HOME=$DB_HOME; export ORACLE_HOME
ORACLE_TERM=xterm; export ORACLE_TERM
BASE_PATH=/usr/sbin:$PATH; export BASE_PATH
PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
alias grid_env='. /home/oracle/grid_env'
alias db_env='. /home/oracle/db_env'
-------------------------------------------------------------------
Create a file called "/home/oracle/grid_env" with the following contents. Add the following contents to set grid environment.
ORACLE_SID=+ASM1; export ORACLE_SID
ORACLE_HOME=$GRID_HOME; export ORACLE_HOME
PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
----------------------------------------------------------------------
Create a file called "/home/oracle/db_env" with the following contents. Add the following contents to set database environment.
PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH
export ORACLE_UNQNAME=sales
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
-----------------------------------------------------------------------
Verify bash profiles:
Once the "/home/oracle/grid_env" has been run, you will be able to switch between environments as follows.
$ . grid_env
$ echo $ORACLE_HOME
/u01/app/11.2.0/grid
$ . db_env
$ echo $ORACLE_HOME
/u01/app/oracle/product/11.2.0/db_1
Enableing the ISCSI-TARGET for Both Nodes.
This is status of LUN after mapping. From systems tab iscsi-target.
Client [iscsi initiator] Configuration:
Here we are using a redhat enterprise linux5 as client. In order to use openfiler target as disk, it has to be set as iscsi initiator. For that we need to install the following package.
#rpm -ivh iscsi-initiator-utils-6.2.0.865-0.8.el5.i386.rpm
# service iscsid start
Turning off network shutdown. Starting iSCSI daemon: [ OK ]
[ OK ]
# chkconfig iscsid on
# chkconfig iscsi on
Searching the iscsi target:
# iscsiadm -m discovery -t sendtargets -p 192.168.0.29
You can also use port number if you want. As follows
# iscsiadm -m discovery -t sendtargets -p 192.168.0.29:3260
192.168.0.27:3260,1 iqn.2006-01.com.openfiler:storage.lap
You will get the scanned result as the line above.
Loging in to the iscsi target:
# iscsiadm -m node -T 192.168.0.27:3260,1 iqn.2006-01.com.openfiler:storage.lap -p 192.168.0.29 -l
-l or --login
#fdisk -l
will list the iscsi targets now.
Example:
[root@server ~]# iscsiadm -m discovery -t sendtargets -p 192.168.0.29
192.168.0.29:3260,1 iqn.2006-01.com.openfiler:storage.lap
[root@server ~]# iscsiadm -m node -T iqn.2006-01.com.openfiler:storage.lap -p 192.168.0.29 -l
fdisk -l -> formating newly added asm disks on RAC1
fdisk /dev/sdb -> n,p,1,w
fdisk /dev/sdc -> n,p,1,w
fdisk /dev/sdd -> n,p,1,w
fdisk /dev/sde -> n,p,1,w
# /etc/init.d/oracleasm configure -i
-> oracle
-> dba
-> y,y
# oracleasm init -> No messages also fine
# oraclasm status -> check the output it is mounted, that is fine.
/usr/sbin/oracleasm createdisk DISK1 /dev/sdb1
/usr/sbin/oracleasm createdisk DISK2 /dev/sdc1
/usr/sbin/oracleasm createdisk DISK3 /dev/sdd1
/usr/sbin/oracleasm createdisk DISK4 /dev/sde1
# oracleasm listdisks
/**************************************************************************
/* The following is not required and should not be run - run by advice by seniors only */
# ./oracleasm stop
# ./oracleasm start
# ./oracleasm exit
# oracleasm init -> should show output as it is mounted
# oracleasm scandisks -> should show up all the DISK* Instantiating...
**************************************************************************
start RAC1
and
start RAC2
Set network adapters
set hostname in neat
change grid_env,db_env if required
ping both priv and host ips
start grid installation
if you not using DNS SERVER then remove this file.
#mv /etc/resolv.conf /etc/resolv.conf_bkp
----------*********----------
vi /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
# Public Network - (eth0)
192.168.1.35 rac1.dba.com rac1
192.168.1.36 rac2.dba.com rac2
# Private Interconnect - (eth1)
192.168.0.135 rac1-priv.dba.com rac1-priv
192.168.0.136 rac2-priv.dba.com rac2-priv
# Public Virtual IP (VIP) addresses for - (eth0)
192.168.1.202 rac1-vip.dba.com rac1-vip
192.168.1.203 rac2-vip.dba.com rac2-vip
# Scan IP
192.168.1.188 cluster-scan
#Openfile Ip
192.168.1.100 san.dba.com san
5.1.2. Edit the -sysctl.conf- of orcl1 with the following values.
vi /etc/sysctl.conf
-------------------
kernel.core_uses_pid = 1
kernel.shmmax=4294967295
kernel.sem=256 32000 100 142
fs.file-max = 6815744
kernel.msgmnb = 65535
kernel.msgmni = 2878
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.tcp_wmem = 262144 262144 262144
net.ipv4.tcp_rmem = 262144 262144 262144
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
kernel.shmmni = 4096
5.1.3. Edit the -limits.conf- file of orcl1 with the following values.
vi /etc/security/limits.conf
----------------------------
hard nofile 65535
soft nofile 4096
hard nproc 16384
soft nproc 2047
grid hard nofile 65536
oracle hard nofile 65536
Run the following command to change the current kernel parameters.
/sbin/sysctl -p
5.1.4. Create groups and users
------------------------------
groupadd -g 501 oinstall
groupadd -g 502 dba
useradd -u 503 -g oinstall -G dba oracle
passwd oracle
5.1.5. Install missing packages
-------------------------------
Caution: If you are getting warning
cd /etc/pki/rpm-gpg/
rpm -import RPM-GPG-KEY
If already rpm installed with warning use --force after -Uvh
example rpm -Uvh --force binutils-2.*
# cd /stage/54
# more run54.txt
Copy the output and paste it.
Ensure all rpms installed with out any warning messages.
-------------------------------------------------------------
Create the directories in which the Oracle software will be installed.
mkdir -p /u01/app/11.2.0/grid
mkdir -p /u01/app/oracle/product/11.2.0/db_1
chown -R oracle:oinstall /u01
chmod -R 775 /u01/
Disable NTP
-----------
Either configure NTP, or make sure it is not configured so the Oracle Cluster Time Synchronization Service (ctssd) can synchronize the times of the RAC nodes. If you want to deconfigure NTP do the following.
# service ntpd stop
Shutting down ntpd: [ OK ]
# chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.orig
# rm /var/run/ntpd.pid
Login as oracle and set the following in .bash_profile
# Oracle Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_HOSTNAME=rac2.dba.com; export ORACLE_HOSTNAME
ORACLE_UNQNAME=sales; export ORACLE_UNQNAME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
GRID_HOME=/u01/app/11.2.0/grid; export GRID_HOME
DB_HOME=$ORACLE_BASE/product/11.2.0/db_1; export DB_HOME
ORACLE_HOME=$DB_HOME; export ORACLE_HOME
ORACLE_TERM=xterm; export ORACLE_TERM
BASE_PATH=/usr/sbin:$PATH; export BASE_PATH
PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
alias grid_env='. /home/oracle/grid_env'
alias db_env='. /home/oracle/db_env'
-------------------------------------------------------------------
Create a file called "/home/oracle/grid_env" with the following contents. Add the following contents to set grid environment.
ORACLE_SID=+ASM1; export ORACLE_SID
ORACLE_HOME=$GRID_HOME; export ORACLE_HOME
PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
----------------------------------------------------------------------
Create a file called "/home/oracle/db_env" with the following contents. Add the following contents to set database environment.
PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH
export ORACLE_UNQNAME=sales
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
-----------------------------------------------------------------------
Verify bash profiles:
Once the "/home/oracle/grid_env" has been run, you will be able to switch between environments as follows.
$ . grid_env
$ echo $ORACLE_HOME
/u01/app/11.2.0/grid
$ . db_env
$ echo $ORACLE_HOME
/u01/app/oracle/product/11.2.0/db_1
Enableing the ISCSI-TARGET for Both Nodes.
This is status of LUN after mapping. From systems tab iscsi-target.
Client [iscsi initiator] Configuration:
Here we are using a redhat enterprise linux5 as client. In order to use openfiler target as disk, it has to be set as iscsi initiator. For that we need to install the following package.
#rpm -ivh iscsi-initiator-utils-6.2.0.865-0.8.el5.i386.rpm
# service iscsid start
Turning off network shutdown. Starting iSCSI daemon: [ OK ]
[ OK ]
# chkconfig iscsid on
# chkconfig iscsi on
Searching the iscsi target:
# iscsiadm -m discovery -t sendtargets -p 192.168.0.29
You can also use port number if you want. As follows
# iscsiadm -m discovery -t sendtargets -p 192.168.0.29:3260
192.168.0.27:3260,1 iqn.2006-01.com.openfiler:storage.lap
You will get the scanned result as the line above.
Loging in to the iscsi target:
# iscsiadm -m node -T 192.168.0.27:3260,1 iqn.2006-01.com.openfiler:storage.lap -p 192.168.0.29 -l
-l or --login
#fdisk -l
will list the iscsi targets now.
Example:
[root@server ~]# iscsiadm -m discovery -t sendtargets -p 192.168.0.29
192.168.0.29:3260,1 iqn.2006-01.com.openfiler:storage.lap
[root@server ~]# iscsiadm -m node -T iqn.2006-01.com.openfiler:storage.lap -p 192.168.0.29 -l
fdisk -l -> formating newly added asm disks on RAC1
fdisk /dev/sdb -> n,p,1,w
fdisk /dev/sdc -> n,p,1,w
fdisk /dev/sdd -> n,p,1,w
fdisk /dev/sde -> n,p,1,w
# /etc/init.d/oracleasm configure -i
-> oracle
-> dba
-> y,y
# oracleasm init -> No messages also fine
# oraclasm status -> check the output it is mounted, that is fine.
/usr/sbin/oracleasm createdisk DISK1 /dev/sdb1
/usr/sbin/oracleasm createdisk DISK2 /dev/sdc1
/usr/sbin/oracleasm createdisk DISK3 /dev/sdd1
/usr/sbin/oracleasm createdisk DISK4 /dev/sde1
# oracleasm listdisks
/**************************************************************************
/* The following is not required and should not be run - run by advice by seniors only */
# ./oracleasm stop
# ./oracleasm start
# ./oracleasm exit
# oracleasm init -> should show output as it is mounted
# oracleasm scandisks -> should show up all the DISK* Instantiating...
**************************************************************************
start RAC1
and
start RAC2
Set network adapters
set hostname in neat
change grid_env,db_env if required
ping both priv and host ips
start grid installation
if you not using DNS SERVER then remove this file.
#mv /etc/resolv.conf /etc/resolv.conf_bkp
Thanks Comments Please.
