Can’t add a service already registered with the listener

Thursday, August 27, 2009
By Steve

This morning, while testing on a two node 10.2.0.3 EE SLES 10 cluster, I found that using srvctl to remove a service from the clusterware does not necessarily stop the service and de-register it from the listener. I’m not sure if this is officially documented, but it will be now…at least in my book :)

I do not show it below, but I had a service with the same name (reqman) as the one to be added running, but deleted it shortly before the steps shown below.

We first show that we add the reqman service and show it is registered, but not running…

09:44:47 oracle@racperf01 ~ >srvctl add service -d perfrac -s reqman -u -r perfrac1
 
09:44:55 oracle@racperf01 ~ >./crsstat.ksh
HA Resource                                   Target     State
-----------                                   ------     -----
ora.perfrac.db                                ONLINE     ONLINE on racperf01
ora.perfrac.perfcnx.cs                        ONLINE     ONLINE on racperf01
ora.perfrac.perfcnx.perfrac1.srv              ONLINE     ONLINE on racperf01
ora.perfrac.perfrac1.inst                     ONLINE     ONLINE on racperf01
ora.perfrac.perfrac2.inst                     ONLINE     ONLINE on racperf02
ora.perfrac.reqman.cs                         ONLINE     ONLINE on racperf02
ora.perfrac.reqman.perfrac1.srv               OFFLINE    OFFLINE
ora.perfrac.reqman.perfrac2.srv               ONLINE     ONLINE on racperf02
ora.racperf01.ASM1.asm                        ONLINE     ONLINE on racperf01
ora.racperf01.LISTENER_ASM_RACPERF01.lsnr     ONLINE     ONLINE on racperf01
ora.racperf01.LISTENER_RACPERF01.lsnr         ONLINE     ONLINE on racperf01
ora.racperf01.gsd                             ONLINE     ONLINE on racperf01
ora.racperf01.ons                             ONLINE     ONLINE on racperf01
ora.racperf01.vip                             ONLINE     ONLINE on racperf01
ora.racperf02.ASM2.asm                        ONLINE     ONLINE on racperf02
ora.racperf02.LISTENER_ASM_RACPERF02.lsnr     ONLINE     ONLINE on racperf02
ora.racperf02.LISTENER_RACPERF02.lsnr         ONLINE     ONLINE on racperf02
ora.racperf02.gsd                             ONLINE     ONLINE on racperf02
ora.racperf02.ons                             ONLINE     ONLINE on racperf02
ora.racperf02.vip                             ONLINE     ONLINE on racperf02

…and we then attempt to start our new service…

09:44:57 oracle@racperf01 ~ >srvctl start service -d perfrac -s reqman -i perfrac1
PRKP-1030 : Failed to start the service reqman.
CRS-0215: Could not start resource 'ora.perfrac.reqman.perfrac1.srv'.

…and when we see we cannot, we find our service is still registered with the listener…

09:45:29 oracle@racperf01 ~ >lsnrctl status LISTENER_RACPERF01
 
LSNRCTL for Linux: Version 10.2.0.3.0 - Production on 27-AUG-2009 09:45:31
 
Copyright (c) 1991, 2006, Oracle.  All rights reserved.
 
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=racperf01-vip.fake.domain)(PORT=2484)(IP=FIRST)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_RACPERF01
Version                   TNSLSNR for Linux: Version 10.2.0.3.0 - Production
Start Date                24-AUG-2009 13:05:46
Uptime                    2 days 20 hr. 39 min. 45 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/oracle/product/10.2.0/network/admin/listener.ora
Listener Log File         /u01/app/oracle/product/10.2.0/network/log/listener_racperf01.log
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.113.187)(PORT=2484)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.113.185)(PORT=2484)))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
  Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...<b>
Service "ReqMan.fake.domain" has 1 instance(s).
  Instance "perfrac1", status READY, has 1 handler(s) for this service...</b>
Service "perfcnx.fake.domain" has 1 instance(s).
  Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "perfrac.fake.domain" has 1 instance(s).
  Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "perfracXDB.fake.domain" has 1 instance(s).
  Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "pfan.fake.domain" has 1 instance(s).
  Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "prdrac.fake.domain" has 1 instance(s).
  Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "prdrac_XPT.fake.domain" has 1 instance(s).
  Instance "perfrac1", status READY, has 1 handler(s) for this service...
The command completed successfully

…so we log into the database instance on which we would like our service to be reflected as running in the clusterware and stop the service…

10:04:33 oracle@racperf01 ~ >sqlplus / AS sysdba
 
SQL*Plus: Release 10.2.0.3.0 - Production ON Thu Aug 27 10:04:36 2009
 
Copyright (c) 1982, 2006, Oracle.  ALL Rights Reserved.
 
 
Connected TO:
Oracle DATABASE 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
WITH the Partitioning, Real Application Clusters, OLAP AND DATA Mining options
 
SQL> exec dbms_service.stop_service('ReqMan')
 
PL/SQL procedure successfully completed.
 
Elapsed: 00:00:00.01
SQL> exit

…and find we can now start our service using srvctl…

10:04:49 oracle@racperf01 ~ >srvctl start service -d perfrac -s reqman -i perfrac1

Leave a Reply