Wednesday 26 October 2011

Apply Linux Kernel Update with ASMLib configured

Before your actual database server which has Linux running on is patched, you need to ensure that you have downloaded the right asmlib rpm for the new kernel version.

The asmlib rpm can be downloaded from:
http://www.oracle.com/technetwork/server-storage/linux/downloads/rhel5-084877.html

After the kernel update is applied on the server, you need to install the new asmlib rpm as root user:
For example, for the below kernel version, you need to install the following rpm
# uname -r
2.6.18-274.7.1.el5
# rpm -Uvh oracleasm-2.6.18-274.7.1.el5-2.0.5-1.el5.x86_64.rpm


Once the new rpm for the patched kernel is installed, you will need to restart the asm so that the disks can be discovered. To do this, issue the following as root:

# service oracleasm restart
Dropping Oracle ASMLib disks:                              [  OK  ]
Shutting down the Oracle ASMLib driver:                    [  OK  ]
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]

# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3
VOL4
VOL5
VOL6

You should now be able to start your asm instance successfully with all the required diskgroups mounted and ready to go.

Wednesday 12 October 2011

OUI-67200:Make failed to invoke "/usr/bin/make -f ins_net_client.mk client_sharedlib ..." 'genclntsh: genclntsh: Could not locate /u01/db/tech_st/11.1.0/rdbms/admin/shrept.lst

We had a situation where applying CPU patch to a database generated the following warnings:

OUI-67200:Make failed to invoke "/usr/bin/make -f ins_net_client.mk client_sharedlib ORACLE_HOME=/u01/db/tech_st/11.1.0"....'genclntsh: genclntsh: Could not locate  /u01/db/tech_st/11.1.0/rdbms/admin/shrept.lst
make: *** [client_sharedlib] Error 1
'
OUI-67124:Re-link fails on target "client_sharedlib".
Re-link fails on target "client_sharedlib".

Inorder to fix it, we simply copied the shrept.lst file from another Oracle Home which was at same patchset level to the current $ORACLE_HOME/rdbms/admin location. We re-ran the opatch utility to apply the CPU patch and this time it did not display the above warnings at all.

We were also able to start SQLPLUS successfully as well.

Wednesday 5 October 2011

OPatch failed to locate Central Inventory; The Central Inventory is corrupted

If you ever get this error while running opatch, don't think that it is end of the world and you have to reinstall oracle.
There is a fix for it and it works.
We got the following error for a client environment while running opatch:

$ $ORACLE_HOME/OPatch/opatch lsinventory
Invoking OPatch 11.1.0.6.2

Oracle Interim Patch Installer version 11.1.0.6.2
Copyright (c) 2007, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/product/11.1.0/db_1
Central Inventory : /u01/app/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.1.0.6.2
OUI version       : 11.1.0.7.0
OUI location      : /u01/app/product/11.1.0/db_1/oui
Log file location : /u01/app/product/11.1.0/db_1/cfgtoollogs/opatch/opatch2011-10-06_09-11-34AM.log

OPatch failed to locate Central Inventory.
Possible causes are:
    The Central Inventory is corrupted
    The oraInst.loc file specified is not valid.
LsInventorySession failed: OPatch failed to locate Central Inventory.
Possible causes are:
    The Central Inventory is corrupted
    The oraInst.loc file specified is not valid.

OPatch failed with error code 73
And here is how it was fixed:

1. Ensure /etc/oraInst.loc has the correct entry for inventory location. This was the output for ours:
$ cat /etc/oraInst.loc
inventory_loc=/u01/app/oraInventory
inst_group=oinstall

2. The following command was run to attach the current home to the inventory
$ cd $ORACLEW_HOME/oui/bin
$ ./runInstaller -silent -invPtrLoc "/etc/oraInst.loc" -attachHome ORACLE_HOME="/u01/app/product/11.1.0/db_1" ORACLE_HOME_NAME="Ora11gHome"
3. After the above command was run successfully, the opatch was running file:
$ opatch lsinventory
Invoking OPatch 11.1.0.6.2

Oracle Interim Patch Installer version 11.1.0.6.2
Copyright (c) 2007, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/product/11.1.0/db_1
Central Inventory : /u01/app/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.1.0.6.2
OUI version       : 11.1.0.7.0
OUI location      : /u01/app/product/11.1.0/db_1/oui
Log file location : /u01/app/product/11.1.0/db_1/cfgtoollogs/opatch/opatch2011-10-06_09-32-11AM.log

Lsinventory Output file location : /u01/app/product/11.1.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory2011-10-06_09-32-11AM.txt
--------------------------------------------------------------------------------
Installed Top-level Products (2):

Oracle Database 11g                                                  11.1.0.6.0
Oracle Database 11g Patch Set 1                                      11.1.0.7.0
There are 2 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.

--------------------------------------------------------------------------------
OPatch succeeded.