RMAN hangs with Direct NFS: please check that oradism is setuid

Issue: RMAN Restore command hang… Not progressing

[oracle]$ rman target /

Recovery Manager: Release 11.2.0.4.0 - Production on Fri Sep 28 04:42:19 2018

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

connected to target database: testdb (not mounted)

RMAN> restore controlfile from '/u01/testdb/bkp/testdb_controlfile_20180923_987582616_s293297_p1';

Starting restore at 28-SEP-18

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=2281 device type=DISK

Killed

Verified alert log and found below Direct NFS messages

Killed rman job which is suspend status

Alert log:

Fri Sep 28 04:27:05 2018

Direct NFS: please check that oradism is setuid

Fri Sep 28 04:28:06 2018

Direct NFS: please check that oradism is setuid

Fri Sep 28 04:29:06 2018

Direct NFS: please check that oradism is setuid

Fri Sep 28 04:30:06 2018

Direct NFS: please check that oradism is setuid

Fri Sep 28 04:31:07 2018

Direct NFS: please check that oradism is setuid

Fri Sep 28 04:32:07 2018

Direct NFS: please check that oradism is setuid

Fri Sep 28 04:33:08 2018

Direct NFS: please check that oradism is setuid

Fri Sep 28 04:34:08 2018

Direct NFS: please check that oradism is setuid

Fri Sep 28 04:35:09 2018

Solution: Check oradism permissions and change as per below

[oracle]$ ls -la $ORACLE_HOME/bin/oradism

-rwsr-x--- 1 oracle oinstall 71790 Aug 24  2013 /u01/app/oracle/product/11.2.0_64/bin/oradism

cd $ORACLE_HOME/bin/oradism

[root]# chown root:root oradism

[root]# chmod 4755 oradism

[oracle]$ ls -la $ORACLE_HOME/bin/oradism

-rwsr-xr-x 1 root root 71790 Aug 24  2013 /u01/app/oracle/product/11.2.0_64/bin/oradism

[oracle]$

 

RMAN Restore is successful after changing permission for oradism  

RMAN> restore controlfile from '/u01/testdb/bkp/testdb_controlfile_20180923_987582616_s293297_p1';

Starting restore at 28-SEP-18

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=2281 device type=DISK

channel ORA_DISK_1: restoring control file

channel ORA_DISK_1: restore complete, elapsed time: 00:00:01

output file name=/u01/testdb/control01.ctl

output file name=/u02/testdb/control02.ctl

Finished restore at 28-SEP-18

ORA-29783: GPnP attribute SET failed with error in RAC

Error while setting ASM_DISKSTRING parameter

SQL> alter system set ASM_DISKSTRING='ORCL:ORA_DATA*','ORCL:ORA_INDEX*','ORCL:ORA_FRA*' scope=both sid='*';

alter system set ASM_DISKSTRING='ORCL:ORA_DATA*','ORCL:ORA_INDEX*','ORCL:ORA_FRA*' scope=both sid='*'

*

ERROR at line 1:

ORA-29783: GPnP attribute SET failed with error [CLSGPNP_NOT_FOUND]

Issue: GI gpnpd is unhealthy

We can find below error in log

[ GPNP]clsgpnp_crs_ensuredSignAndPush: [at clsgpnp_crs.c:663] Result: (5) CLSGPNP_NOT_FOUND. discovery failed for 8 nodes

[ GPNP]clsgpnp_crs_ensuredSignAndPush: [at clsgpnp_crs.c:757] Result: (5) CLSGPNP_NOT_FOUND. Error(s) occurred while trying to push GPnP profile op=(remote gpnpd discovery). State may be inconsistent.

Solution:Kill unhealthy gpnpd.bin from all nodes, it’ll auto restarted by GI                                                         

Node 1:

root:# ps -ef | grep gpnpd

grid 19446 1 0 Aug08 ? 00:12:40 /u01/app/grid/11.2.0.4/bin/gpnpd.bin

grid 64810 34831  0 00:15 pts/1 00:00:00 grep --color=auto gpnpd

root:# kill -9 19446

root:# ps -ef | grep gpnpd

grid 2462 1 0 00:15 ? 00:00:00 /u01/app/grid/11.2.0.4/bin/gpnpd.bin

root 4295 31796  0 00:15 pts/1 00:00:00 grep --color=auto gpnpd

Node 2:

root:# ps -ef | grep gpnpd

grid 56456 1 0 Aug08 ? 00:11:59 /u01/app/grid/11.2.0.4/bin/gpnpd.bin

root 57877 56781  0 00:16 pts/0 00:00:00 grep --color=auto gpnpd

root:# kill -9 56456

root:# ps -ef | grep gpnpd

grid 58454 1 0 00:16 ? 00:00:00 /u01/app/grid/11.2.0.4/bin/gpnpd.bin

root 58738 56781  0 00:16 pts/0  00:00:00 grep --color=auto gpnpd

                                                           

SQL> alter system set ASM_DISKSTRING='ORCL:ORA_DATA*','ORCL:ORA_INDEX*','ORCL:ORA_FRA*' scope=both sid='*';

System altered.