This morning, while testing on a two node 10.2.0.3 EE SLES 10 cluster, I found that using srvctl to remove a service from the clusterware does not necessarily stop the service and de-register it from the listener. I’m not sure if this is officially documented, but it will be now…at least in my book 🙂
I do not show it below, but I had a service with the same name (reqman) as the one to be added running, but deleted it shortly before the steps shown below.
We first show that we add the reqman service and show it is registered, but not running…
09:44:47 oracle@racperf01 ~ >srvctl add service -d perfrac -s reqman -u -r perfrac1
09:44:55 oracle@racperf01 ~ >./crsstat.ksh
HA Resource Target State
----------- ------ -----
ora.perfrac.db ONLINE ONLINE on racperf01
ora.perfrac.perfcnx.cs ONLINE ONLINE on racperf01
ora.perfrac.perfcnx.perfrac1.srv ONLINE ONLINE on racperf01
ora.perfrac.perfrac1.inst ONLINE ONLINE on racperf01
ora.perfrac.perfrac2.inst ONLINE ONLINE on racperf02
ora.perfrac.reqman.cs ONLINE ONLINE on racperf02
ora.perfrac.reqman.perfrac1.srv OFFLINE OFFLINE
ora.perfrac.reqman.perfrac2.srv ONLINE ONLINE on racperf02
ora.racperf01.ASM1.asm ONLINE ONLINE on racperf01
ora.racperf01.LISTENER_ASM_RACPERF01.lsnr ONLINE ONLINE on racperf01
ora.racperf01.LISTENER_RACPERF01.lsnr ONLINE ONLINE on racperf01
ora.racperf01.gsd ONLINE ONLINE on racperf01
ora.racperf01.ons ONLINE ONLINE on racperf01
ora.racperf01.vip ONLINE ONLINE on racperf01
ora.racperf02.ASM2.asm ONLINE ONLINE on racperf02
ora.racperf02.LISTENER_ASM_RACPERF02.lsnr ONLINE ONLINE on racperf02
ora.racperf02.LISTENER_RACPERF02.lsnr ONLINE ONLINE on racperf02
ora.racperf02.gsd ONLINE ONLINE on racperf02
ora.racperf02.ons ONLINE ONLINE on racperf02
ora.racperf02.vip ONLINE ONLINE on racperf02
…and we then attempt to start our new service…
09:44:57 oracle@racperf01 ~ >srvctl start service -d perfrac -s reqman -i perfrac1
PRKP-1030 : Failed to start the service reqman.
CRS-0215: Could not start resource 'ora.perfrac.reqman.perfrac1.srv'.
…and when we see we cannot, we find our service is still registered with the listener…
09:45:29 oracle@racperf01 ~ >lsnrctl status LISTENER_RACPERF01
LSNRCTL for Linux: Version 10.2.0.3.0 - Production on 27-AUG-2009 09:45:31
Copyright (c) 1991, 2006, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=racperf01-vip.fake.domain)(PORT=2484)(IP=FIRST)))
STATUS of the LISTENER
------------------------
Alias LISTENER_RACPERF01
Version TNSLSNR for Linux: Version 10.2.0.3.0 - Production
Start Date 24-AUG-2009 13:05:46
Uptime 2 days 20 hr. 39 min. 45 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/oracle/product/10.2.0/network/admin/listener.ora
Listener Log File /u01/app/oracle/product/10.2.0/network/log/listener_racperf01.log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.113.187)(PORT=2484)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.113.185)(PORT=2484)))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "ReqMan.fake.domain" has 1 instance(s).
Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "perfcnx.fake.domain" has 1 instance(s).
Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "perfrac.fake.domain" has 1 instance(s).
Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "perfracXDB.fake.domain" has 1 instance(s).
Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "pfan.fake.domain" has 1 instance(s).
Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "prdrac.fake.domain" has 1 instance(s).
Instance "perfrac1", status READY, has 1 handler(s) for this service...
Service "prdrac_XPT.fake.domain" has 1 instance(s).
Instance "perfrac1", status READY, has 1 handler(s) for this service...
The command completed successfully
…so we log into the database instance on which we would like our service to be reflected as running in the clusterware and stop the service…
10:04:33 oracle@racperf01 ~ >sqlplus / as sysdba
SQL*Plus: Release 10.2.0.3.0 - Production on Thu Aug 27 10:04:36 2009
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
SQL> exec dbms_service.stop_service('ReqMan')
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.01
SQL> exit
…and find we can now start our service using srvctl…
10:04:49 oracle@racperf01 ~ >srvctl start service -d perfrac -s reqman -i perfrac1