[Oracle] Clusterware(CRS) not coming up

Clusterware(CRS) not coming up:-

1. From root user check crs :-

# crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4535: Cannot communicate with Cluster Ready Services CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online

 

​

2. Try to stop and start it:- 

# crsctl stop crs CRS-2796: The command may not proceed when Cluster Ready Services is not running CRS-4687: Shutdown command has completed with errors. CRS-4000: Command Stop failed, or completed with errors. # crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4535: Cannot communicate with Cluster Ready Services CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online # crsctl start crs CRS-4640: Oracle High Availability Services is already active CRS-4000: Command Start failed, or completed with errors.

 

 

3. Stop High availabilty service and crs resources:- 

crsctl stop res -init -all root@dbwr1 ~]# crsctl stop res -init -all CRS-2500: Cannot stop resource 'ora.diskmon' as it is not running CRS-2673: Attempting to stop 'ora.asm' on 'dbwr1' CRS-2673: Attempting to stop 'ora.storage' on 'dbwr1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'dbwr1' CRS-2673: Attempting to stop 'ora.gpnpd' on 'dbwr1' CRS-2673: Attempting to stop 'ora.gipcd' on 'dbwr1' CRS-2673: Attempting to stop 'ora.crf' on 'dbwr1' CRS-2673: Attempting to stop 'ora.evmd' on 'dbwr1' CRS-2677: Stop of 'ora.asm' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.storage' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'dbwr1' CRS-2673: Attempting to stop 'ora.crsd' on 'dbwr1' CRS-2673: Attempting to stop 'ora.ctssd' on 'dbwr1' CRS-2677: Stop of 'ora.evmd' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.gpnpd' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.gipcd' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.crsd' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.crf' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'dbwr1' CRS-2677: Stop of 'ora.cssd' on 'dbwr1' succeeded CRS-4000: Command Stop failed, or completed with errors. crsctl stop has [root@dbwr1 ~]# crsctl stop has CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'dbwr1' CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'dbwr1' has completed CRS-4133: Oracle High Availability Services has been stopped. Verify no crs processes should be running :- ps -ef|grep d.bin [root@dbwr1 ~]# ps -ef|grep d.bin root      6302  4587  0 19:56 pts/0    00:00:00 grep --color=auto d.bin

Note: Kill if any process is still running .


4. Now start crs :-

crsctl start crs

ASM instances and databases will also come up in current node.


5. If any ASM Disk group is dismounted then :-

ALTER DISKGROUP <Disk_Group_Name> MOUNT;

​

 

==============================================================

[root@opendb01 opendb01]# crsctl stop crs [root@dbwr1 ~]# crsctl stop crs CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'dbwr1' CRS-2673: Attempting to stop 'ora.crsd' on 'dbwr1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'dbwr1' CRS-2673: Attempting to stop 'ora.chad' on 'dbwr1' CRS-2677: Stop of 'ora.chad' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'dbwr1' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'dbwr1' CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server 'dbwr1' CRS-2673: Attempting to stop 'ora.asm' on 'dbwr1' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'dbwr1' CRS-2677: Stop of 'ora.scan1.vip' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.dbwr1.vip' on 'dbwr1' CRS-2677: Stop of 'ora.dbwr1.vip' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.asm' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'dbwr1' CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on 'dbwr1' CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on 'dbwr1' succeeded CRS-33677: Stop of resource group 'ora.asmgroup' on server 'dbwr1' succeeded. CRS-2672: Attempting to start 'ora.scan1.vip' on 'dbwr2' CRS-2672: Attempting to start 'ora.dbwr1.vip' on 'dbwr2' CRS-2676: Start of 'ora.dbwr1.vip' on 'dbwr2' succeeded CRS-2676: Start of 'ora.scan1.vip' on 'dbwr2' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'dbwr2' CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'dbwr2' succeeded CRS-2799: Failed to shut down resource 'ora.OCR_DISK.dg' on 'dbwr1' CRS-2794: Shutdown of Cluster Ready Services-managed resources on 'dbwr1' has failed CRS-2675: Stop of 'ora.crsd' on 'dbwr1' failed CRS-2799: Failed to shut down resource 'ora.crsd' on 'dbwr1' CRS-2795: Shutdown of Oracle High Availability Services-managed resources on 'dbwr1' has failed CRS-4687: Shutdown command has completed with errors. CRS-4000: Command Stop failed, or completed with errors. [root@opendb01 opendb01]# crsctl stop crs -f [root@dbwr1 ~]# crsctl stop crs -f CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'dbwr1' CRS-2673: Attempting to stop 'ora.crsd' on 'dbwr1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'dbwr1' CRS-2673: Attempting to stop 'ora.OCR_DISK.dg' on 'dbwr1' CRS-2677: Stop of 'ora.OCR_DISK.dg' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'dbwr1' CRS-2677: Stop of 'ora.ons' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'dbwr1' CRS-2677: Stop of 'ora.net1.network' on 'dbwr1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'dbwr1' has completed CRS-2677: Stop of 'ora.crsd' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'dbwr1' CRS-2673: Attempting to stop 'ora.crf' on 'dbwr1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'dbwr1' CRS-2677: Stop of 'ora.crf' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.asm' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'dbwr1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'dbwr1' CRS-2673: Attempting to stop 'ora.evmd' on 'dbwr1' CRS-2677: Stop of 'ora.evmd' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'dbwr1' CRS-2677: Stop of 'ora.cssd' on 'dbwr1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'dbwr1' CRS-2673: Attempting to stop 'ora.gpnpd' on 'dbwr1' CRS-2677: Stop of 'ora.gpnpd' on 'dbwr1' succeeded CRS-2677: Stop of 'ora.gipcd' on 'dbwr1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'dbwr1' has completed CRS-4133: Oracle High Availability Services has been stopped.

​

​

​==============================================================

[root@RAC01 ~]# crsctl stop crs CRS-2796: The command may not proceed when Cluster Ready Services is not running CRS-4687: Shutdown command has completed with errors. CRS-4000: Command Stop failed, or completed with errors. [root@RAC01 ~]# crsctl start cluster   [root@RAC01 ~]# crsctl check crs [root@dbwr1 ~]# crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online