<rt id="bn8ez"></rt>
<label id="bn8ez"></label>

  • <span id="bn8ez"></span>

    <label id="bn8ez"><meter id="bn8ez"></meter></label>

    qileilove

    blog已經(jīng)轉(zhuǎn)移至github,大家請訪問 http://qaseven.github.io/

    Oracle RAC Database 11.1.0.6監(jiān)聽故障案例

     晚上接到電話,客戶的一套核心Oracle RAC數(shù)據(jù)庫連接不上,連接時報無監(jiān)聽程序,客戶的Oracle RAC版本為11.1.0.6,平臺為AIX 6.1.05,使用了IBM HACMP 5.5.0.8。
      當(dāng)我遠程過去的時候,發(fā)現(xiàn)節(jié)點2已經(jīng)沒有任何oracle用戶的進程,且concurrent的vg沒有激活,HACMP的服務(wù)也offline。
      另一個節(jié)點Oracle的實例是正常的,且有部分服務(wù)器進程依然在工作,但是本地監(jiān)聽器出現(xiàn)了故障,導(dǎo)致新的連接無法連接到實例,通過crs_stat -t看到兩個實例的監(jiān)聽也都是OFFLINE狀態(tài)。
      在節(jié)點上并沒有發(fā)現(xiàn)有LISTENER進程,且手動殺掉了所有的服務(wù)器進程,在oracle用戶下啟動監(jiān)聽時收到以下的報錯:
    $ lsnrctl start listener_cdfy740a
    LSNRCTL for IBM/AIX RISC System/6000: Version 11.1.0.6.0 - Production on 20-NOV-2014 20:09:09
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    Starting /oracle/app/oracle/product/11.1.0/db_1/bin/tnslsnr: please wait...
    TNSLSNR for IBM/AIX RISC System/6000: Version 11.1.0.6.0 - Production
    System parameter file is /oracle/app/oracle/product/11.1.0/db_1/network/admin/listener.ora
    Log messages written to /oracle/app/oracle/diag/tnslsnr/cdfy740a/listener_cdfy740a/alert/log.xml
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.107.64.1)(PORT=1521)))
    Error listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.107.64.1)(PORT=1521)(IP=FIRST)))
    TNS-12542: TNS:address already in use
    TNS-12560: TNS:protocol adapter error
    TNS-00512: Address already in use
    IBM/AIX RISC System/6000 Error: 67: Address already in use
    Listener failed to start. See the error message(s) above...
      10.107.64.1是該節(jié)點的vip地址,下面是RAC環(huán)境的hosts配置:
    10.107.64.1     vip1
    10.107.64.2     vip2
    10.107.64.3     cdfy740a
    10.107.64.4     cdfy740b
    172.201.201.1   prv1
    172.201.201.2   prv2
      手動停掉該節(jié)點的nodeapps服務(wù):
      cdfy740a@root[/oracle/app/11.1.0/crs/bin]./srvctl stop nodeapps -n cdfy740a
      成功停止后,VIP在主機層面已經(jīng)消失:
    cdfy740a@root[/oracle/app/11.1.0/crs/bin]ifconfig -a | more
    en0: flags=1e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>
    inet 172.200.200.1 netmask 0xffffff00 broadcast 172.200.200.255
    tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
    en1: flags=1e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>
    inet 172.201.201.1 netmask 0xffffff00 broadcast 172.201.201.255
    tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
    en4: flags=5e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),PSEG,LARGESEND,CHAIN>
    inet 10.107.64.3 netmask 0xffffff00 broadcast 10.107.64.255
    tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
    lo0: flags=e08084b,c0<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,LARGESEND,CHAIN>
    inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255
    inet6 ::1%1/0
    tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1
     再次啟動節(jié)點nodeapps服務(wù):
    cdfy740a@root[/oracle/app/11.1.0/crs/bin]./srvctl start nodeapps -n cdfy740a
    CRS-1006: No more members to consider
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:LSNRCTL for IBM/AIX RISC System/6000: Version 11.1.0.6.0 - Production on 20-NOV-2014 20:13:07
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Starting /oracle/app/oracle/product/11.1.0/db_1/bin/tnslsnr: please wait...
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:TNSLSNR for IBM/AIX RISC System/6000: Version 11.1.0.6.0 - Production
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:System parameter file is /oracle/app/oracle/product/11.1.0/db_1/network/admin/listener.ora
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Log messages written to /oracle/app/oracle/diag/tnslsnr/cdfy740a/listener_cdfy740a/alert/log.xml
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.107.64.1)(PORT=1521)))
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Error listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.107.64.1)(PORT=1521)(IP=FIRST)))
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:TNS-12542: TNS:address already in use
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr: TNS-12560: TNS:protocol adapter error
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:  TNS-00512: Address already in use
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:   IBM/AIX RISC System/6000 Error: 67: Address already in use
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Listener failed to start. See the error message(s) above...
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:LSNRCTL for IBM/AIX RISC System/6000: Version 11.1.0.6.0 - Production on 20-NOV-2014 20:13:08
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=vip1)(PORT=1521)))
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:TNS-12541: TNS:no listener
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr: TNS-12560: TNS:protocol adapter error
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:  TNS-00511: No listener
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:   IBM/AIX RISC System/6000 Error: 79: Connection refused
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.107.64.1)(PORT=1521)(IP=FIRST)))
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:TNS-12541: TNS:no listener
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr: TNS-12560: TNS:protocol adapter error
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:  TNS-00511: No listener
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:   IBM/AIX RISC System/6000 Error: 79: Connection refused
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.107.64.3)(PORT=1521)(IP=FIRST)))
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:TNS-12541: TNS:no listener
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr: TNS-12560: TNS:protocol adapter error
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:  TNS-00511: No listener
    cdfy740a:ora.cdfy740a.LISTENER_CDFY740A.lsnr:   IBM/AIX RISC System/6000 Error: 79: Connection refused
    CRS-0215: Could not start resource 'ora.cdfy740a.LISTENER_CDFY740A.lsnr'.
      之前使用lsnrctl status listener_cdfy740a查看監(jiān)聽器狀態(tài)時也收到Connection refused的錯誤。
      查看主機層面已經(jīng)成功綁定了VIP地址:
    cdfy740a@root[/oracle/app/11.1.0/crs/bin]ifconfig -a | more
    en0: flags=1e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>
    inet 172.200.200.1 netmask 0xffffff00 broadcast 172.200.200.255
    tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
    en1: flags=1e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>
    inet 172.201.201.1 netmask 0xffffff00 broadcast 172.201.201.255
    tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
    en4: flags=5e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),PSEG,LARGESEND,CHAIN>
    inet 10.107.64.3 netmask 0xffffff00 broadcast 10.107.64.255
    inet 10.107.64.1 netmask 0xffffff00 broadcast 10.107.64.255
    tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
    lo0: flags=e08084b,c0<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,LARGESEND,CHAIN>
    inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255
    inet6 ::1%1/0
    tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1
      再次嘗試手動啟動本地監(jiān)聽器:
    cdfy740a@root[/]su - oracle
    $ lsnrctl start listener_cdfy740a
    LSNRCTL for IBM/AIX RISC System/6000: Version 11.1.0.6.0 - Production on 20-NOV-2014 20:18:37
    Copyright (c) 1991, 2007, Oracle.  All rights reserved.
    Starting /oracle/app/oracle/product/11.1.0/db_1/bin/tnslsnr: please wait...
    TNSLSNR for IBM/AIX RISC System/6000: Version 11.1.0.6.0 - Production
    System parameter file is /oracle/app/oracle/product/11.1.0/db_1/network/admin/listener.ora
    Log messages written to /oracle/app/oracle/diag/tnslsnr/cdfy740a/listener_cdfy740a/alert/log.xml
    Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.107.64.1)(PORT=1521)))
    Error listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.107.64.1)(PORT=1521)(IP=FIRST)))
    TNS-12542: TNS:address already in use
    TNS-12560: TNS:protocol adapter error
    TNS-00512: Address already in use
    IBM/AIX RISC System/6000 Error: 67: Address already in use
    Listener failed to start. See the error message(s) above...
      啟動依然失敗。
      檢查監(jiān)聽器配置文件:
    $ cat listener.ora
    # listener.ora.cdfy740a Network Configuration File: /oracle/app/oracle/product/11.1.0/db_1/network/admin/listener.ora.cdfy740a
    # Generated by Oracle configuration tools.
    LISTENER_CDFY740A =
    (DESCRIPTION_LIST =
    (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = vip1)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = 10.107.64.1)(PORT = 1521)(IP = FIRST))
    (ADDRESS = (PROTOCOL = TCP)(HOST = 10.107.64.3)(PORT = 1521)(IP = FIRST))
    )
    )
      在監(jiān)聽配置文件中,vip1和10.107.64.1是兩個重復(fù)的地址,手動將10.107.64.1所在行去掉之后,監(jiān)聽即可正常的啟動。
      之后恢復(fù)節(jié)點2的HACMP服務(wù),Oracle RAC隨即恢復(fù)正常。
      另外,還發(fā)現(xiàn)客戶的監(jiān)聽日志已經(jīng)被填得很大,大概在1.6GB左右,過大的監(jiān)聽日志文件也會導(dǎo)致監(jiān)聽器不穩(wěn)定,這里將兩個節(jié)點的監(jiān)聽日志進行了重命名操作。
      《10g RAC監(jiān)聽器配置文件listener.ora中的IP=FIRST》:http://blog.itpub.net/23135684/viewspace-715967/
      《IP=FIRST作用說明》:http://www.xifenfei.com/2713.html

    posted on 2014-11-27 12:00 順其自然EVO 閱讀(245) 評論(0)  編輯  收藏 所屬分類: 測試學(xué)習(xí)專欄

    <2014年11月>
    2627282930311
    2345678
    9101112131415
    16171819202122
    23242526272829
    30123456

    導(dǎo)航

    統(tǒng)計

    常用鏈接

    留言簿(55)

    隨筆分類

    隨筆檔案

    文章分類

    文章檔案

    搜索

    最新評論

    閱讀排行榜

    評論排行榜

    主站蜘蛛池模板: 一边摸一边爽一边叫床免费视频| 免费在线黄色网址| 久久国产精品免费一区二区三区| 亚洲av乱码一区二区三区| 亚洲αv在线精品糸列| 男人的天堂亚洲一区二区三区 | 亚洲色图激情文学| 亚洲天堂在线播放| 亚洲精品无码久久久久| 亚洲国产精品成人网址天堂| 色吊丝最新永久免费观看网站| 中文字幕免费在线看线人| 午夜无码A级毛片免费视频| 精品国产污污免费网站入口| 曰批免费视频播放免费| 精品久久亚洲一级α| 亚洲精品成a人在线观看☆| 亚洲人成77777在线观看网| 亚洲成AV人片久久| 亚洲成a人不卡在线观看| 亚洲春色在线观看| 亚洲国产成人在线视频| 亚洲国产精品综合福利专区| 噜噜噜亚洲色成人网站∨| 久久久久亚洲精品无码蜜桃 | 最近中文字幕完整免费视频ww| 在线观看免费视频一区| 中文字幕视频在线免费观看| 黄色视屏在线免费播放| 国产在线国偷精品免费看| 羞羞视频免费网站在线看| 色www永久免费| 久久久久国产精品免费免费不卡 | 亚洲精品免费在线观看| 亚洲激情视频在线观看| 99久久亚洲综合精品成人网| 亚洲国产片在线观看| 亚洲最大福利视频| 久久亚洲中文字幕无码| 日韩少妇内射免费播放| 成人A毛片免费观看网站|