{"id":3389,"date":"2013-12-31T08:12:40","date_gmt":"2013-12-31T13:12:40","guid":{"rendered":"http:\/\/appcrawler.com\/wordpress\/?p=3389"},"modified":"2013-12-31T10:02:33","modified_gmt":"2013-12-31T15:02:33","slug":"recreating-diskgroup-in-an-existing-rac","status":"publish","type":"post","link":"http:\/\/appcrawler.com\/wordpress\/2013\/12\/31\/recreating-diskgroup-in-an-existing-rac\/","title":{"rendered":"Recreating diskgroup in an existing RAC"},"content":{"rendered":"<p>We created a diskgroup with normal redundancy, and were wasting half the space on the mirror as a result.  We are completely striped and mirrored in EMC, so a mirror buys us no additional protection.<\/p>\n<p>We wanted to recreate the diskgroup with external redundancy rather than creating a new diskgroup and copying all the data to it.  This has the additional complexity that the cluster registry and voting disk are also stored in the diskgroup being recreated.<\/p>\n<p>Below is what we did to accomplish this.<\/p>\n<p>STEPS<\/p>\n<p>Ensure you have a good backup of all databases, including the spfile if stored in ASM.<\/p>\n<p>Shutdown applications using and databases in the cluster<\/p>\n<p>Verify cluster status<\/p>\n<pre lang=\"text\">\r\n[root@expressdb1 etc]# \/home\/oracle\/crsstat.sh\r\nNAME                                     TARGET     STATE           SERVER            STATE_DETAILS\r\n-------------------------                ---------- ----------      ------------      ----------------------------\r\nora.DATA.dg                              ONLINE     ONLINE          expressdb1\r\nora.asm                                  ONLINE     ONLINE          expressdb1        Started\r\nora.net1.network                         ONLINE     ONLINE          expressdb1\r\nora.ons                                  ONLINE     ONLINE          expressdb1\r\nora.LISTENER_SCAN1.lsnr                  ONLINE     ONLINE          expressdb1\r\nora.LISTENER_SCAN2.lsnr                  ONLINE     ONLINE          expressdb1\r\nora.LISTENER_SCAN3.lsnr                  ONLINE     ONLINE          expressdb1\r\nora.cvu                                  ONLINE     ONLINE          expressdb1\r\nora.expressdb1.vip                       ONLINE     ONLINE          expressdb1\r\nora.oc4j                                 ONLINE     ONLINE          expressdb1\r\nora.scan1.vip                            ONLINE     ONLINE          expressdb1\r\nora.scan2.vip                            ONLINE     ONLINE          expressdb1\r\nora.scan3.vip                            ONLINE     ONLINE          expressdb1\r\n<\/pre>\n<p>Backup OCR and voting disk<\/p>\n<pre lang=\"text\">\r\n[root@expressdb1 etc]# \/u01\/app\/11.2.0\/grid\/bin\/ocrconfig -manualbackup\r\n\r\nexpressdb1     2013\/12\/30 16:22:53     \/u01\/app\/11.2.0\/grid\/cdata\/expresscrs\/backup_20131230_162253.ocr\r\n<\/pre>\n<p>Backup ASM spfile<\/p>\n<pre lang=\"text\">\r\n[root@expressdb1 etc]# su - grid\r\n[grid@expressdb1 ~]$ profile\r\n1) +ASM1\r\n2) express\r\nPlease select an ORACLE environment from the list above: 1\r\n*************************************************\r\nUSER...............grid\r\nORACLE_SID.........+ASM1\r\nORACLE_HOME........\/u01\/app\/11.2.0\/grid\r\nORACLE_BASE........\/u01\/app\/oracle\r\nHOST...............expressdb1 192.168.3.50\r\n*************************************************\r\n\r\nexpressdb1:grid:+ASM1:\/home\/grid# asmcmd\r\nASMCMD> spget\r\n+DATA\/expresscrs\/asmparameterfile\/registry.253.835546595\r\nASMCMD> spcopy +DATA\/expresscrs\/asmparameterfile\/registry.253.835546595 \/tmp\/asm.ora\r\nASMCMD> exit\r\n<\/pre>\n<p>Stop cluster<\/p>\n<pre lang=\"text\">\r\nexpressdb1:grid:+ASM1:\/home\/grid# exit\r\nlogout\r\n[root@expressdb1 etc]# \/u01\/app\/11.2.0\/grid\/bin\/crsctl stop crs -f\r\nCRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'expressdb1'\r\nCRS-2673: Attempting to stop 'ora.crsd' on 'expressdb1'\r\nCRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'expressdb1'\r\n\r\nCRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'expressdb1' has completed\r\nCRS-4133: Oracle High Availability Services has been stopped.\r\n<\/pre>\n<p>Zero out ASM diskgroup disks<\/p>\n<pre lang=\"text\">\r\n[root@expressdb1 etc]# dd if=\/dev\/zero bs=10M count=10 of=\/dev\/mapper\/oracle1p1\r\n10+0 records in\r\n10+0 records out\r\n104857600 bytes (105 MB) copied, 9.35223 s, 11.2 MB\/s\r\n[root@expressdb1 etc]# dd if=\/dev\/zero bs=10M count=10 of=\/dev\/mapper\/oracle2p1\r\n10+0 records in\r\n10+0 records out\r\n104857600 bytes (105 MB) copied, 9.50856 s, 11.0 MB\/s\r\n[root@expressdb1 etc]# dd if=\/dev\/zero bs=10M count=10 of=\/dev\/mapper\/oracle3p1\r\n10+0 records in\r\n10+0 records out\r\n104857600 bytes (105 MB) copied, 9.50625 s, 11.0 MB\/s\r\n<\/pre>\n<p>Restart cluster in exclusive mode<\/p>\n<pre lang=\"text\">\r\n[root@expressdb1 etc]# \/u01\/app\/11.2.0\/grid\/bin\/crsctl start crs -excl -nocrs\r\nCRS-4123: Oracle High Availability Services has been started.\r\nCRS-2672: Attempting to start 'ora.mdnsd' on 'expressdb1'\r\n\r\nCRS-2676: Start of 'ora.asm' on 'expressdb1' succeeded\r\n<\/pre>\n<p>Recreate diskgroup with external redundancy<\/p>\n<pre lang=\"text\">\r\n[root@expressdb1 etc]# su - grid\r\n[grid@expressdb1 ~]$ profile\r\n1) +ASM1\r\n2) express\r\nPlease select an ORACLE environment from the list above: 1\r\n*************************************************\r\nUSER...............grid\r\nORACLE_SID.........+ASM1\r\nORACLE_HOME........\/u01\/app\/11.2.0\/grid\r\nORACLE_BASE........\/u01\/app\/oracle\r\nHOST...............expressdb1 192.168.3.50\r\n*************************************************\r\n\r\nexpressdb1:grid:+ASM1:\/home\/grid# sqlplus \/ as sysasm\r\n\r\nSQL*Plus: Release 11.2.0.3.0 Production on Mon Dec 30 16:48:57 2013\r\n\r\nCopyright (c) 1982, 2011, Oracle.  All rights reserved.\r\n\r\n\r\nConnected to:\r\nOracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production\r\nWith the Real Application Clusters and Automatic Storage Management options\r\n\r\nSQL> create diskgroup data external redundancy disk '\/dev\/mapper\/oracle1p1','\/dev\/mapper\/oracle2p1','\/dev\/mapper\/oracle3p1';\r\n\r\nDiskgroup created.\r\n\r\nSQL> alter diskgroup data set attribute 'compatible.asm' = '11.2.0.0.0';\r\n\r\nDiskgroup altered.\r\n\r\nSQL> exit\r\nDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production\r\nWith the Real Application Clusters and Automatic Storage Management options\r\n<\/pre>\n<p>Copy backup spfile into ASM<\/p>\n<pre lang=\"text\">\r\nexpressdb1:grid:+ASM1:\/home\/grid# asmcmd\r\nASMCMD> spcopy \/tmp\/asm.ora +DATA\/expresscrs\/asmparameterfile\/asm.ora\r\nASMCMD> spset +DATA\/expresscrs\/asmparameterfile\/asm.ora\r\nASMCMD> exit\r\n<\/pre>\n<p>Restart cluster<\/p>\n<pre lang=\"text\">\r\nexpressdb1:grid:+ASM1:\/home\/grid# exit\r\nlogout\r\n[root@expressdb1 etc]# \/u01\/app\/11.2.0\/grid\/bin\/crsctl stop crs -f\r\nCRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'expressdb1'\r\n\r\nCRS-4133: Oracle High Availability Services has been stopped.\r\n[root@expressdb1 etc]# \/u01\/app\/11.2.0\/grid\/bin\/crsctl start crs -excl -nocrs\r\nCRS-4123: Oracle High Availability Services has been started.\r\n\r\nCRS-2676: Start of 'ora.asm' on 'expressdb1' succeeded\r\n<\/pre>\n<p>Restore the OCR and voting disk<\/p>\n<pre lang=\"text\">\r\n[root@expressdb1 etc]# \/u01\/app\/11.2.0\/grid\/bin\/ocrconfig -restore \/u01\/app\/11.2.0\/grid\/cdata\/expresscrs\/backup_20131230_162253.ocr\r\n[root@expressdb1 etc]# \/u01\/app\/11.2.0\/grid\/bin\/crsctl replace votedisk +DATA\r\nSuccessful addition of voting disk 4ee46a25dc614ff9bfc94362cc62cf82.\r\nSuccessfully replaced voting disk group with +DATA.\r\nCRS-4266: Voting file(s) successfully replaced\r\n<\/pre>\n<p>Verify restored files<\/p>\n<pre lang=\"text\">\r\n[root@expressdb1 etc]# su - grid\r\n[grid@expressdb1 ~]$ profile\r\n1) +ASM1\r\n2) express\r\nPlease select an ORACLE environment from the list above: 1\r\n*************************************************\r\nUSER...............grid\r\nORACLE_SID.........+ASM1\r\nORACLE_HOME........\/u01\/app\/11.2.0\/grid\r\nORACLE_BASE........\/u01\/app\/oracle\r\nHOST...............expressdb1 192.168.3.50\r\n*************************************************\r\n\r\nexpressdb1:grid:+ASM1:\/home\/grid# \/u01\/app\/11.2.0\/grid\/bin\/crsctl query css votedisk\r\n##  STATE    File Universal Id                File Name Disk group\r\n--  -----    -----------------                --------- ---------\r\n 1. ONLINE   4ee46a25dc614ff9bfc94362cc62cf82 (\/dev\/mapper\/oracle1p1) [DATA]\r\nLocated 1 voting disk(s).\r\nexpressdb1:grid:+ASM1:\/home\/grid#\r\n<\/pre>\n<p>Restart clusterware in normal mode, restore and recover database<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We created a diskgroup with normal redundancy, and were wasting half the space on the mirror as a result. We are completely striped and mirrored in EMC, so a mirror buys us no additional protection. We wanted to recreate the&hellip;<\/p>\n<p class=\"more-link-p\"><a class=\"more-link\" href=\"http:\/\/appcrawler.com\/wordpress\/2013\/12\/31\/recreating-diskgroup-in-an-existing-rac\/\">Read more &rarr;<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"footnotes":""},"categories":[19,22,32],"tags":[],"_links":{"self":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/3389"}],"collection":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/comments?post=3389"}],"version-history":[{"count":7,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/3389\/revisions"}],"predecessor-version":[{"id":3399,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/3389\/revisions\/3399"}],"wp:attachment":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/media?parent=3389"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/categories?post=3389"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/tags?post=3389"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}