19c upgrade Prechecks failed with error Clusterware patch level mismatch

19c upgrade Prechecks failed:

./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /u01/app/18.3.0.0/grid -dest_crshome /u01/app/19.3.0.0/grid -dest_version 19.7.0.0

 

Errors:

Verifying Clusterware Version Consistency ...FAILED

Verifying cluster upgrade state ...FAILED

racnode02: PRVG-13411 : Oracle Clusterware active patch level

"1844241963" does not match software patch level "2292884171"

on node "racnode02".

 

racnode01: PRVG-13411 : Oracle Clusterware active patch level

"1844241963" does not match software patch level "2292884171"

on node "racnode01".

 

Activeversion and releasepatch are different in cluster nodes: 

[grid@racnode01 ~]$ crsctl query crs activeversion -f

Oracle Clusterware active version on the cluster is [18.0.0.0.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [1844241963].

 

[grid@racnode01 ~]$ crsctl query crs releasepatch

Oracle Clusterware release patch level is [2292884171] and the complete list of patches [27494830 27908644 27923415 28090523 28090557 28090564 28256701 28553832 28790643 ] have been applied on the local node.

 

Patch level are different.. Looks patches not applied properly

Try by running pre and post patch steps

Execute Below Script from All cluster nodes :

1) crsctl stop crs

2) $GI_HOME/crs/install/rootcrs.sh -prepatch

3) $GI_HOME/crs/install/rootcrs.sh -postpatch

 

Verify Patch levels:

Node1 :

[grid@racnode01 grid]$ crsctl query crs softwarepatch

Oracle Clusterware patch level on node racnode01 is [2292884171].

[grid@racnode01 grid]$ crsctl query crs activeversion -f

Oracle Clusterware active version on the cluster is [18.0.0.0.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [2292884171].

[grid@racnode01 grid]$ crsctl query crs releasepatch

Oracle Clusterware release patch level is [2292884171] and the complete list of patches [27494830 27908644 27923415 28090523 28090557 28090564 28256701 28553832 28790643 ] have been applied on the local node. The release patch string is [18.3.2.0.0].

 

Node2 :

[grid@racnode02 ~]$ crsctl query crs softwarepatch

Oracle Clusterware patch level on node racnode02 is [2292884171].

[grid@racnode02 ~]$ crsctl query crs activeversion -f

Oracle Clusterware active version on the cluster is [18.0.0.0.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [2292884171].

[grid@racnode02 ~]$ crsctl query crs releasepatch

Oracle Clusterware release patch level is [2292884171] and the complete list of patches [27494830 27908644 27923415 28090523 28090557 28090564 28256701 28553832 28790643 ] have been applied on the local node. The release patch string is [18.3.2.0.0].