Step by step Oracle 12c R2 RAC installation on Oracle Solaris

Step 1: Update /etc/hosts file on both RAC nodes

---------IP Information Node1--------

--Public

10.11.1.18 badc-cdbpr01.said-bd.com badc-cdbpr01

10.11.1.19 badc-cdbpr02.said-bd.com badc-cdbpr02

--VIP

10.11.1.118 badc-cdbpr02-vip.said-bd.com badc-cdbpr01-vip

10.11.1.119 badc-cdbpr02-vip.said-bd.com badc-cdbpr02-vip

--Private

10.64.131.48 badc-cdbpr01-pvt.said-bd.com badc-cdbpr01-pvt

10.64.131.49 badc-cdbpr01-pvt.said-bd.com badc-cdbpr01-pvt

--SCAN

10.11.1.105 cdbpr-scan.said-bd.com cdbpr-scan

---------IP Information Node2--------

--Public

10.11.1.18 badc-cdbpr01.said-bd.com badc-cdbpr01

10.11.1.19 badc-cdbpr02.said-bd.com badc-cdbpr02

--VIP

10.11.1.118 badc-cdbpr02-vip.said-bd.com badc-cdbpr01-vip

10.11.1.119 badc-cdbpr02-vip.said-bd.com badc-cdbpr02-vip

--Private

10.64.131.48 badc-cdbpr01-pvt.said-bd.com badc-cdbpr01-pvt

10.64.131.49 badc-cdbpr01-pvt.said-bd.com badc-cdbpr01-pvt

--SCAN

10.11.1.105 cdbpr-scan.said-bd.com cdbpr-scan

svccfg -s system/identity:node listprop config

svccfg -s system/identity:node setprop config/nodename="badc-cdbpr01"

svccfg -s system/identity:node setprop config/loopback="badc-cdbpr01"

svccfg -s system/identity:node setprop config/nodename="badc-cdbpr02"

svccfg -s system/identity:node setprop config/loopback="badc-cdbpr02"

svcadm refresh system/identity:node

svcadm restart system/identity:node

svccfg -s system/identity:node listprop config

Step 2: Install packages

The following packages must be installed:

■ pkg://solaris/system/dtrace

■ pkg://solaris/developer/assembler

■ pkg://solaris/developer/build/make

■ pkg://solaris/system/xopen/xcu4 (if not already installed as part of standard Oracle Solaris 11 installation)

pkg://solaris/system/kernel/oracka (Only for Oracle Real Application Clusters installations)

■ pkg://solaris/x11/diagnostic/x11-info-clients

■ pkg://solaris/compress/unzip

pkg install system/kernel/oracka

pkg install system/dtrace

pkg install developer/assembler

pkg install developer/build/make

pkg install system/xopen/xcu4

pkg install system/kernel/oracka

pkg install x11/diagnostic/x11-info-clients

pkg install compress/unzip

Step 3: Increase Swap size

Step 4: create group

Below group needs to create with GUID on both rac nodes. You can create group without GUID, but sometimes its leading problem.

groupadd -g 1000 oinstall

groupadd -g 1020 asmadmin

groupadd -g 1021 asmdba

groupadd -g 1022 asmoper

groupadd -g 1031 dba

groupadd -g 1032 oper

groupadd -g 1033 racdba

Step 5: create user

Below users needs to create with group membership on both rac nodes.

useradd -u 1100 -g oinstall -G dba,asmadmin,asmdba,asmoper,racdba,oper -m -d /export/home/grid grid

useradd -u 1101 -g oinstall -G dba,asmadmin,asmdba,racdba,oper -m -d /export/home/oracle oracle

Step 6: password set for user

set oracle and grid user password on both rac nodes.

passwd oracle

passwd grid

Step 7: Create directories

To create the Oracle Inventory directory, enter the following commands as the root user:

mkdir -p /d01/app/oraInventory

chown -R grid:oinstall /d01/app/oraInventory

chmod -R 775 /d01/app/oraInventory

To create the Grid Infrastructure home directory, enter the following commands as the root user:

mkdir -p /d01/app/12.2.0.1/grid

chown -R grid:oinstall /d01/app/12.2.0.1/grid

chmod -R 775 /d01/app/12.2.0.1/grid

mkdir -p /d01/app/grid

chown -R grid:oinstall /d01/app/grid

chmod -R 775 /d01/app/grid

To create the Oracle Base directory, enter the following commands as the root user:

mkdir -p /d01/app/oracle

mkdir -p /d01/app/oracle/cfgtoollogs

chown -R oracle:oinstall /d01/app/oracle

chmod -R 775 /d01/app/oracle

To create the Oracle RDBMS Home directory, enter the following commands as the root user:

mkdir -p /d01/app/oracle/product/12.2.0.1/db_1

chown -R oracle:oinstall /d01/app/oracle/product/12.2.0.1/db_1

chmod -R 775 /d01/app/oracle/product/12.2.0.1/db_1

Step 9: Disable Firewall

#svcadm disable ipfilter

Step 10: Setting Network Time Protocol for Cluster Time Synchronization

• On Oracle Solaris 11 and later versions

Run the following command as root user:

# /usr/sbin/svccfg -s svc:/network/ntp:default setprop config/slew_always = true

Then, restart the NTP service:

# /usr/sbin/svcadm refresh svc:/network/ntp:default

To enable NTP after it has been disabled, enter the following command:

# /usr/sbin/svcadm enable ntp

cd /etc/inet

cp ntp.client ntp.conf

chmod +x ntp.conf

vi /etc/inet/ntp.conf

server 10.88.3.200

server 10.88.1.254

/usr/sbin/svcadm enable ntp

/usr/sbin/svcadm restart ntp

ntpq -p

Step 11: Verifying & set UDP and TCP Kernel Parameters

root@badc-cdbpr01 # ipadm show-prop -p smallest_anon_port,largest_anon_port tcp

PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE

tcp   smallest_anon_port    rw   32768        --           32768        1024-65535

tcp   largest_anon_port     rw   65535        --           65535        32768-65535

root@badc-cdbpr01 #

root@badc-cdbpr02 # ipadm show-prop -p smallest_anon_port,largest_anon_port tcp

PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE

tcp   smallest_anon_port    rw   32768        --           32768        1024-65535

tcp   largest_anon_port     rw   65535        --           65535        32768-65535

root@badc-cdbpr02 #

/usr/sbin/ndd /dev/tcp tcp_smallest_anon_port tcp_largest_anon_port

/usr/sbin/ndd /dev/udp udp_smallest_anon_port udp_largest_anon_port

ipadm set-prop -p smallest_anon_port=9000 tcp

ipadm set-prop -p largest_anon_port=65500 tcp

ipadm set-prop -p smallest_anon_port=9000 udp

ipadm set-prop -p largest_anon_port=65500 udp

ipadm set-prop -p send_buf=65536 udp

ipadm set-prop -p recv_buf=65536 udp

applied on both nodes:

ipadm set-prop -p smallest_anon_port=9000 tcp

ipadm set-prop -p smallest_anon_port=9000 udp

ipadm set-prop -p send_buf=65536 udp

ipadm set-prop -p recv_buf=65536 udp

Step 12: Set Grid profile

On both RAC nodes of Grid user, set below profile:

Node 1:

PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:$HOME/bin:/usr/bin/X11:/sbin:.

export PATH

if [ -s "$MAIL" ] # This is at Shell startup. In normal

then echo "$MAILMSG" # operation, the Shell checks

fi # periodically.

umask 022

TMP=/tmp; export TMP

TMPDIR=/tmp; export TMPDIR

TEMPDIR=/tmp; export TEMPDIR

TEMP=/tmp; export TEMP

ORACLE_HOSTNAME=badc-cdbpr01; export ORACLE_HOSTNAME

ORACLE_HOME=/d01/app/12.2.0.1/grid;export ORACLE_HOME

ORACLE_BASE=/d01/app/grid;export ORACLE_BASE

ORACLE_SID=+ASM1; export ORACLE_SID

ORACLE_TERM=xterm; export ORACLE_TERM

# NLS_LANG=AMERICAN_AMERICA.AL32UTF8; export NLS_LANG

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

PATH=/usr/sbin:$PATH; export PATH

PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib; export LD_LIBRARY_PATH

LIBPATH=$LD_LIBRARY_PATH; export LD_LIBRARY_PATH

set -o vi

if [ -t 0 ]; then

stty intr ^C

fi

CLASSPATH=$ORACLE_HOME/jdk:$ORACLE_HOME/jdk/jre/bin:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/jdbc/lib; export CLASSPATH

ulimit -t unlimited

ulimit -f unlimited

ulimit -d unlimited

ulimit -s unlimited

ulimit -n 65536

ulimit -v unlimited

Node 2:

PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:$HOME/bin:/usr/bin/X11:/sbin:.

export PATH

if [ -s "$MAIL" ] # This is at Shell startup. In normal

then echo "$MAILMSG" # operation, the Shell checks

fi # periodically.

umask 022

TMP=/tmp; export TMP

TMPDIR=/tmp; export TMPDIR

TEMPDIR=/tmp; export TEMPDIR

TEMP=/tmp; export TEMP

ORACLE_HOSTNAME=badc-cdbpr02; export ORACLE_HOSTNAME

ORACLE_HOME=/d01/app/12.2.0.1/grid;export ORACLE_HOME

ORACLE_BASE=/d01/app/grid;export ORACLE_BASE

ORACLE_SID=+ASM2; export ORACLE_SID

ORACLE_TERM=xterm; export ORACLE_TERM

# NLS_LANG=AMERICAN_AMERICA.AL32UTF8; export NLS_LANG

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

PATH=/usr/sbin:$PATH; export PATH

PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib; export LD_LIBRARY_PATH

LIBPATH=$LD_LIBRARY_PATH; export LD_LIBRARY_PATH

set -o vi

if [ -t 0 ]; then

stty intr ^C

fi

CLASSPATH=$ORACLE_HOME/jdk:$ORACLE_HOME/jdk/jre/bin:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/jdbc/lib; export CLASSPATH

ulimit -t unlimited

ulimit -f unlimited

ulimit -d unlimited

ulimit -s unlimited

ulimit -n 65536

ulimit -v unlimited

Step 8: Set Oracle profile

On both RAC nodes of oracle user, set below profile:

Node 1:

umask 022

TMP=/tmp; export TMP

TMPDIR=/tmp; export TMPDIR

TEMPDIR=/tmp; export TEMPDIR

TEMP=/tmp; export TEMP

ORACLE_HOSTNAME=badc-cdbpr01; export ORACLE_HOSTNAME

ORACLE_UNQNAME=cdbpr; export ORACLE_UNQNAME

ORACLE_BASE=/d01/app/oracle;export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/12.2.0.1/db_1;export ORACLE_HOME

ORACLE_SID=cdbpr01; export ORACLE_SID

ORACLE_TERM=xterm; export ORACLE_TERM

# NLS_LANG=AMERICAN_AMERICA.AL32UTF8; export NLS_LANG

# TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

PATH=/usr/sbin:$PATH; export PATH

PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib; export LD_LIBRARY_PATH

LIBPATH=$LD_LIBRARY_PATH; export LD_LIBRARY_PATH

if [ -t 0 ]; then

stty intr ^C

fi

CLASSPATH=$ORACLE_HOME/jdk:$ORACLE_HOME/jdk/jre/bin:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/jdbc/lib; export CLASSPATH

ulimit -t unlimited

ulimit -f unlimited

ulimit -d unlimited

ulimit -s unlimited

ulimit -n 65536

ulimit -v unlimited

Node 2:

umask 022

TMP=/tmp; export TMP

TMPDIR=/tmp; export TMPDIR

TEMPDIR=/tmp; export TEMPDIR

TEMP=/tmp; export TEMP

ORACLE_HOSTNAME=badc-cdbpr02; export ORACLE_HOSTNAME

ORACLE_UNQNAME=cdbpr; export ORACLE_UNQNAME

ORACLE_BASE=/d01/app/oracle;export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/12.2.0.1/db_1;export ORACLE_HOME

ORACLE_SID=cdbpr02; export ORACLE_SID

ORACLE_TERM=xterm; export ORACLE_TERM

# NLS_LANG=AMERICAN_AMERICA.AL32UTF8; export NLS_LANG

# TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

PATH=/usr/sbin:$PATH; export PATH

PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib; export LD_LIBRARY_PATH

LIBPATH=$LD_LIBRARY_PATH; export LD_LIBRARY_PATH

if [ -t 0 ]; then

stty intr ^C

fi

CLASSPATH=$ORACLE_HOME/jdk:$ORACLE_HOME/jdk/jre/bin:$ORACLE_HOME/jlib$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/jdbc/lib; export CLASSPATH:

ulimit -t unlimited

ulimit -f unlimited

ulimit -d unlimited

ulimit -s unlimited

ulimit -n 65536

ulimit -v unlimited

Step 13: Set user attribute

Set below user attribute on both RAC nodes

root@badc-cdbpr02:~# vi /etc/user_attr

grid::::lock_after_retries=no;profiles=System Administrator;roles=root

oracle::::lock_after_retries=no;profiles=System Administrator;roles=root

---Configuring Kernel Parameters on Oracle Solaris ---

Step 14: Minimum Parameter Settings for Installation

The value for the maximum shared memory depends on the SGA requirements and should be set to a value greater than the SGA size.

The project.max-shm-memory resource control value assumes that no other application is using the shared memory segment from

this project other than the Oracle instances. If applications, other than the Oracle instances are using the shared memory segment,

then you must add that shared memory usage to the project.max-shm-memory resource control value

Ensure that memory_target (or max_sga_size) does not exceed process.max-address-space and project.max-shm-memory

projadd -c "oracle" 'user.oracle'

projmod -s -K "project.max-shm-memory=(privileged,800G,deny)" 'user.oracle'

projmod -s -K "process.max-sem-nsems=(privileged,256,deny)" 'user.oracle'

projmod -s -K "project.max-shm-ids=(privileged,256,deny)" 'user.oracle'

projmod -s -K "project.max-sem-ids=(privileged,256,deny)" 'user.oracle'

projmod -s -K "process.max-file-descriptor=(priv,65536,deny)" 'user.oracle'

projadd -c "grid" 'user.grid'

projmod -s -K "project.max-shm-memory=(privileged,200G,deny)" 'user.grid'

projmod -s -K "process.max-sem-nsems=(privileged,256,deny)" 'user.grid'

projmod -s -K "project.max-shm-ids=(privileged,256,deny)" 'user.grid'

projmod -s -K "project.max-sem-ids=(privileged,256,deny)" 'user.grid'

projmod -s -K "process.max-file-descriptor=(priv,65536,deny)" 'user.grid'

projmod -U oracle user.oracle

projmod -U grid user.grid

To display the current values of the resource control:

su – oracle

$ id -p

uid=100(oracle) gid=100(dba) projid=1 (group.dba)

projmod -sK "project.max-shm-memory=(privileged,50G,deny)" group.oinstall

projmod -s -K "project.max-shm-memory=(privileged,800G,deny)" 'user.grid'

user.oracle

        projid : 100

        comment: "oracle"

        users  : oracle

        groups : (none)

        attribs: process.max-file-descriptor=(priv,65536,deny)

                 process.max-sem-nsems=(privileged,256,deny)

                 project.max-sem-ids=(privileged,256,deny)

                 project.max-shm-ids=(privileged,256,deny)

                 project.max-shm-memory=(privileged,858993459200,deny)

user.grid

        projid : 101

        comment: "grid"

        users  : grid

        groups : (none)

        attribs: process.max-file-descriptor=(priv,65536,deny)

                 process.max-sem-nsems=(privileged,256,deny)

                 project.max-sem-ids=(privileged,256,deny)

                 project.max-shm-ids=(privileged,256,deny)

                 project.max-shm-memory=(privileged,214748364800,deny)

-bash-4.4$

Unzip Grid Software to GIRD_HOME

-bash-4.4$ cd $ORACLE_HOME

-bash-4.4$ pwd

/d01/app/12.2.0.1/grid

-bash-4.4$ cd /d01/soft/

-bash-4.4$ ls -lrt

total 11240318

-rw-r--r--   1 grid     oinstall 2444447279 Mar 11 14:19 V840013-01.zip

-rw-r--r--   1 grid     oinstall 3306204620 Mar 11 14:27 V839961-01.zip

-bash-4.4$ chmod 755 *

-bash-4.4$ cd $ORACLE_HOME

-bash-4.4$ unzip -q /d01/soft/V840013-01.zip

root@badc-cdbpr01 # ./orainstRoot.sh

Changing permissions of /d01/app/oraInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /d01/app/oraInventory to oinstall.

The execution of the script is complete.

root@badc-cdbpr01 # cd /d01/app/12.2.0.1/grid/

root@badc-cdbpr01 # ./root.sh

Performing root user operation.

The following environment variables are set as:

    ORACLE_OWNER= grid

    ORACLE_HOME=  /d01/app/12.2.0.1/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Creating /var/opt/oracle/oratab file...

Entries will be added to the /var/opt/oracle/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Relinking oracle with rac_on option

Using configuration parameter file: /d01/app/12.2.0.1/grid/crs/install/crsconfig_params

The log of current session can be found at:

  /d01/app/grid/crsdata/badc-cdbpr01/crsconfig/rootcrs_badc-cdbpr01_2020-04-02_07-21-53PM.log

2020/04/02 19:21:56 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.

2020/04/02 19:21:56 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.

Apr  2 19:22:33 badc-cdbpr01 root: [ID 702911 user.alert] autorun file for ohasd is missing

2020/04/02 19:23:44 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.

2020/04/02 19:23:44 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.

2020/04/02 19:23:50 CLSRSC-363: User ignored prerequisites during installation

2020/04/02 19:23:50 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.

2020/04/02 19:23:53 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.

2020/04/02 19:23:54 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.

2020/04/02 19:24:02 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.

2020/04/02 19:24:02 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.

2020/04/02 19:24:03 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.

2020/04/02 19:24:29 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.

2020/04/02 19:24:39 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.

2020/04/02 19:24:39 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.

2020/04/02 19:24:46 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.

2020/04/02 19:25:01 CLSRSC-330: Adding Clusterware entries to file '/etc/inittab'

2020/04/02 19:25:36 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.

2020/04/02 19:25:42 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'badc-cdbpr01'

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'badc-cdbpr01' has completed

CRS-4133: Oracle High Availability Services has been stopped.

CRS-4123: Oracle High Availability Services has been started.

2020/04/02 19:26:45 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.

2020/04/02 19:26:52 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'badc-cdbpr01'

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'badc-cdbpr01' has completed

CRS-4133: Oracle High Availability Services has been stopped.

CRS-4123: Oracle High Availability Services has been started.

CRS-2672: Attempting to start 'ora.evmd' on 'badc-cdbpr01'

CRS-2672: Attempting to start 'ora.mdnsd' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.mdnsd' on 'badc-cdbpr01' succeeded

CRS-2676: Start of 'ora.evmd' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.gpnpd' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'badc-cdbpr01'

CRS-2672: Attempting to start 'ora.gipcd' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.cssdmonitor' on 'badc-cdbpr01' succeeded

CRS-2676: Start of 'ora.gipcd' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'badc-cdbpr01'

CRS-2672: Attempting to start 'ora.diskmon' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.diskmon' on 'badc-cdbpr01' succeeded

CRS-2676: Start of 'ora.cssd' on 'badc-cdbpr01' succeeded

Disk groups created successfully. Check /d01/app/grid/cfgtoollogs/asmca/asmca-200402PM072731.log for details.

2020/04/02 19:28:27 CLSRSC-482: Running command: '/d01/app/12.2.0.1/grid/bin/ocrconfig -upgrade grid oinstall'

CRS-2672: Attempting to start 'ora.crf' on 'badc-cdbpr01'

CRS-2672: Attempting to start 'ora.storage' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.storage' on 'badc-cdbpr01' succeeded

CRS-2676: Start of 'ora.crf' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.crsd' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.crsd' on 'badc-cdbpr01' succeeded

CRS-4256: Updating the profile

Successful addition of voting disk a9c85495be934fabbfa7e1bd511fc0b6.

Successfully replaced voting disk group with +OCRVOTE.

CRS-4256: Updating the profile

CRS-4266: Voting file(s) successfully replaced

##  STATE    File Universal Id                File Name Disk group

--  -----    -----------------                --------- ---------

 1. ONLINE   a9c85495be934fabbfa7e1bd511fc0b6 (/dev/rdsk/c0t60060160ED864D00B23C5F5EA8213DDDd0s0) [OCRVOTE]

Located 1 voting disk(s).

Apr  2 19:24:54 badc-cdbpr01 last message repeated 14 times

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'badc-cdbpr01'

CRS-2673: Attempting to stop 'ora.crsd' on 'badc-cdbpr01'

CRS-2677: Stop of 'ora.crsd' on 'badc-cdbpr01' succeeded

CRS-2673: Attempting to stop 'ora.storage' on 'badc-cdbpr01'

CRS-2673: Attempting to stop 'ora.crf' on 'badc-cdbpr01'

CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'badc-cdbpr01'

CRS-2673: Attempting to stop 'ora.gpnpd' on 'badc-cdbpr01'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'badc-cdbpr01'

CRS-2677: Stop of 'ora.drivers.acfs' on 'badc-cdbpr01' succeeded

CRS-2677: Stop of 'ora.crf' on 'badc-cdbpr01' succeeded

CRS-2677: Stop of 'ora.gpnpd' on 'badc-cdbpr01' succeeded

CRS-2677: Stop of 'ora.storage' on 'badc-cdbpr01' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'badc-cdbpr01'

CRS-2677: Stop of 'ora.mdnsd' on 'badc-cdbpr01' succeeded

CRS-2677: Stop of 'ora.asm' on 'badc-cdbpr01' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'badc-cdbpr01'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'badc-cdbpr01' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on 'badc-cdbpr01'

CRS-2673: Attempting to stop 'ora.evmd' on 'badc-cdbpr01'

CRS-2677: Stop of 'ora.ctssd' on 'badc-cdbpr01' succeeded

CRS-2677: Stop of 'ora.evmd' on 'badc-cdbpr01' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'badc-cdbpr01'

CRS-2677: Stop of 'ora.cssd' on 'badc-cdbpr01' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'badc-cdbpr01'

CRS-2677: Stop of 'ora.gipcd' on 'badc-cdbpr01' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'badc-cdbpr01' has completed

CRS-4133: Oracle High Availability Services has been stopped.

2020/04/02 19:29:32 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.

CRS-4123: Starting Oracle High Availability Services-managed resources

CRS-2672: Attempting to start 'ora.mdnsd' on 'badc-cdbpr01'

CRS-2672: Attempting to start 'ora.evmd' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.mdnsd' on 'badc-cdbpr01' succeeded

CRS-2676: Start of 'ora.evmd' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.gpnpd' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.gipcd' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.gipcd' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.cssdmonitor' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'badc-cdbpr01'

CRS-2672: Attempting to start 'ora.diskmon' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.diskmon' on 'badc-cdbpr01' succeeded

CRS-2676: Start of 'ora.cssd' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'badc-cdbpr01'

CRS-2672: Attempting to start 'ora.ctssd' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.ctssd' on 'badc-cdbpr01' succeeded

CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.asm' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.storage' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.storage' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.crf' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.crf' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.crsd' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.crsd' on 'badc-cdbpr01' succeeded

CRS-6023: Starting Oracle Cluster Ready Services-managed resources

CRS-6017: Processing resource auto-start for servers: badc-cdbpr01

CRS-6016: Resource auto-start has completed for server badc-cdbpr01

CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources

CRS-4123: Oracle High Availability Services has been started.

2020/04/02 19:31:28 CLSRSC-343: Successfully started Oracle Clusterware stack

2020/04/02 19:31:28 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.

CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.asm' on 'badc-cdbpr01' succeeded

CRS-2672: Attempting to start 'ora.OCRVOTE.dg' on 'badc-cdbpr01'

CRS-2676: Start of 'ora.OCRVOTE.dg' on 'badc-cdbpr01' succeeded

2020/04/02 19:33:01 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.

2020/04/02 19:33:44 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

root@badc-cdbpr01 #

-bash-4.4$ crsctl stat res -t

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.ASMNET1LSNR_ASM.lsnr

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.LISTENER.lsnr

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.MGMT.dg

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.OCRVOTE.dg

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.net1.network

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.ons

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.proxy_advm

               OFFLINE OFFLINE      badc-cdbpr01             STABLE

               OFFLINE OFFLINE      badc-cdbpr02             STABLE

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

ora.MGMTLSNR

      1        ONLINE  ONLINE       badc-cdbpr01             169.254.139.191 10.6

                                                             4.131.48,STABLE

ora.asm

      1        ONLINE  ONLINE       badc-cdbpr01             Started,STABLE

      2        ONLINE  ONLINE       badc-cdbpr02             Started,STABLE

      3        OFFLINE OFFLINE                               STABLE

ora.badc-cdbpr01.vip

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

ora.badc-cdbpr02.vip

      1        ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.cvu

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

ora.mgmtdb

      1        ONLINE  ONLINE       badc-cdbpr01             Open,STABLE

ora.qosmserver

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

ora.scan1.vip

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

--------------------------------------------------------------------------------

-bash-4.4$

From Node 2

-bash-4.4$ crsctl stat res -t

--------------------------------------------------------------------------------

Name           Target  State        Server                   State details

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.ASMNET1LSNR_ASM.lsnr

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.LISTENER.lsnr

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.MGMT.dg

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.OCRVOTE.dg

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.net1.network

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.ons

               ONLINE  ONLINE       badc-cdbpr01             STABLE

               ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.proxy_advm

               OFFLINE OFFLINE      badc-cdbpr01             STABLE

               OFFLINE OFFLINE      badc-cdbpr02             STABLE

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

ora.MGMTLSNR

      1        ONLINE  ONLINE       badc-cdbpr01             169.254.139.191 10.6

                                                             4.131.48,STABLE

ora.asm

      1        ONLINE  ONLINE       badc-cdbpr01             Started,STABLE

      2        ONLINE  ONLINE       badc-cdbpr02             Started,STABLE

      3        OFFLINE OFFLINE                               STABLE

ora.badc-cdbpr01.vip

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

ora.badc-cdbpr02.vip

      1        ONLINE  ONLINE       badc-cdbpr02             STABLE

ora.cvu

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

ora.mgmtdb

      1        ONLINE  ONLINE       badc-cdbpr01             Open,STABLE

ora.qosmserver

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

ora.scan1.vip

      1        ONLINE  ONLINE       badc-cdbpr01             STABLE

--------------------------------------------------------------------------------

-bash-4.4$

Now create ASM disk group using ASMCA command

Oracle Database Installation