You can use below instructions for creating a database listener on Infiniband (IB) on SuperCluster. Creating such a listener enables database clients to connect to it over IB (as opposed to over the Ethernet Client Network). This is typical for an application running on the SuperCluster in an Application Domain. Listeners over IB can be setup to use the TCP protocol, the SDP(Sockets Direct Protocol) protocol, or both.
On SuperCluster, connections to the database over IB are established on the Storage IB Network that is setup using the IB partition 8503. This IB partition is created as part of the initial installation and configuration of the SuperCluster. There is no need to create it as part of the IB listener setup.
The example bellow adds an IB listener to an existing 2-node RAC. The Storage IB subnet is 192.168.30.0/22.
Add the vips for Client Network and Storage IB Network to /etc/hosts. Do this on both nodes:
x.y.132.103 node1-vip 192.168.30.65 node1-vipIB x.y.132.104 node2-vip 192.168.30.66 node2-vipIB
The addresses starting with x.y. are vips on the 10GbE Client Network and should already be listed in /etc/hosts. There is no need to add these addresses.
As root on node1 register the IB subnet in the Grid by running the following commands from the bin directory of the Grid home t(the usual Grid home is /u01/app/11.2.x.y/grid/bin). Also register the two vips:
root@node1# oifcfg iflist bondmgt0 x.y.183.0 bondeth0 x.y.132.0 bondib0 192.168.28.0
root@node1# srvctl add network -k 2 -S 192.168.28.0/255.255.252.0/stor_ipmp1 -w static -v root@node1# crsctl start res ora.net2.network root@node1# srvctl add vip -n node1 -A node1-vipIB/255.255.252.0/stor_ipmp1 -k 2 root@node1# srvctl add vip -n node2 -A node2-vipIB/255.255.252.0/stor_ipmp1 -k 2 root@node1# srvctl start vip -i node1-vipIB
root@node1# srvctl start vip -i node2-vipIB
As grid one node1, create a LISTENER_IB listener by running the following commands from the bin directory of the Grid home (if there is no grid account, use the oracle account):
grid@node1$ srvctl add listener -l LISTENER_IB -k 2 -p 1522 grid@node1$ srvctl start listener -l LISTENER_IB grid@node1$ srvctl status listener -l LISTENER_IB Listener LISTENER_IB is enabled Listener LISTENER_IB is running on node(s): node1,node2
As oracle one node1 check that the listener is properly registered by running the following commands from the bin directory of the Grid home:
oracle@node1$ lsnrctl status LISTENER_IB
LSNRCTL for Solaris: Version 11.2.0.3.0 - Production on 30-JUN-2014 15:49:2 Copyright (c) 1991, 2011, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_IB))) STATUS of the LISTENER
------------------------ Alias Version Start Date
Uptime Trace Level Security SNMP Listener Parameter File Listener Log File
LISTENER_IB TNSLSNR for Solaris: Version 11.2.0.3.0 - Production 30-JUN-2014 15:48:38 0 days 0 hr. 0 min. 50 sec off ON: Local OS Authentication OFF /u01/app/11.2.x.y/grid/network/admin/listener.ora
/u01/app/11.2.x.y/grid/log/diag/tnslsnr/node1/listener_ib/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_IB))) (DESCRIPTION=(ADDRESS=(PROTOCOL=sdp)(HOST=192.168.30.65)(PORT=1522))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.30.65)(PORT=1522))) The listener supports no services
As oracle on node1, edit tnsnames.ora located in the network/admin directory of the Grid home:
If the IB listener is to be used only with a specific standalone database then the tnsnames.ora from the Oracle home must be modified instead.
Get the database name ‘DBNAME’ with srvctl from the bin directory in Grid home. This value is assigned to SERVICE_NAME in tnsnames.ora:
oracle@node1$ srvctl config database DBNAME
Use vi ‘set list’ command to view invisible characters. These can create problems during the next step (executing sql statement) and should be removed.
## BEGIN
DBNAME_IB =
(DESCRIPTION =
(LOAD_BALANCE=on)
(ADDRESS = (PROTOCOL = TCP)(HOST = node1-vipIB)(PORT = 1522))
(ADDRESS = (PROTOCOL = TCP)(HOST = node2-vipIB)(PORT = 1522))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = DBNAME)
))
LISTENER_IBREMOTE =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = node2-vipIB)(PORT = 1522))
))
LISTENER_IBLOCAL =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = node1-vipIB)(PORT = 1522))
(ADDRESS = (PROTOCOL = SDP)(HOST = node1-vipIB)(PORT = 1522))
))
LISTENER_IPLOCAL =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = node1-vip)(PORT = 1521))
))
LISTENER_IPREMOTE =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = node2-vip)(PORT = 1521))
))
## END
As oracle on node2, add the following lines to tnsnames.ora in the network/admin directory of the Grid home:
DBNAME_IB = (DESCRIPTION =
(LOAD_BALANCE=on)
(ADDRESS = (PROTOCOL = TCP)(HOST = node1-vipIB)(PORT = 1522))
(ADDRESS = (PROTOCOL = TCP)(HOST = node2-vipIB)(PORT = 1522))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = DBNAME)
))
LISTENER_IBREMOTE = (DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = node1-vipIB)(PORT = 1522))
))
LISTENER_IBLOCAL = (DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = node2-vipIB)(PORT = 1522))
(ADDRESS = (PROTOCOL = SDP)(HOST = node2-vipIB)(PORT = 1522))
))
LISTENER_IPLOCAL = (DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = node2-vip)(PORT = 1521))
))
LISTENER_IPREMOTE = (DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = node1-vip)(PORT = 1521))
))
## END
As oracle on both nodes, register the new listener at the database level. Make sure to set ORACLE_SID to the proper value before running sqlplus:
oracle@node1$ export ORACLE_SID=DBNAME1 oracle@node1$ sqlplus / as sysdba SQL> alter system set listener_networks='((NAME=network2)(LOCAL_LISTENER=LISTENER_IBLOCAL)(REMOTE_LISTENER=LISTENER_IBREMOTE))','((NAME=network1)(LOCAL_LISTENER=LISTENER_IPLOCAL)(REMOTE_LISTENER=LISTENER_IPREMOTE))' scope=both;
oracle@node2$ export ORACLE_SID=DBNAME2 oracle@node2$ sqlplus / as sysdba SQL> alter system set listener_networks='((NAME=network2)(LOCAL_LISTENER=LISTENER_IBLOCAL) (REMOTE_LISTENER=LISTENER_IBREMOTE))','((NAME=network1)(LOCAL_LISTENER=LISTENER_IPLOCAL)(REMOTE_LISTENER=LISTENER_IPREMOTE))' scope=both;
As oracle on node1, restart LISTENER_IB and check its status:
oracle@node1$ srvctl stop listener -l LISTENER_IB oracle@node1$ srvctl start listener -l LISTENER_IB oracle@node1$ export TNS_ADMIN=/u01/app/11.2.x.y/grid/network/admin oracle@node1$ lsnrctl status LISTENER_IB
LSNRCTL for Solaris: Version 11.2.0.3.0 - Production on 04-JUL-2014 11:53:20 Copyright (c) 1991, 2011, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_IB))) STATUS of the LISTENER ------------------------
Alias Version Start Date
LISTENER_IB TNSLSNR for Solaris: Version 11.2.0.3.0 - Production 04-JUL-2014 11:52:32 0 days 0 hr. 0 min. 47 sec off ON: Local OS Authentication OFF /u01/app/11.2.0.3/grid/network/admin/listener.ora /u01/app/11.2.0.3/grid/log/diag/tnslsnr/rmb-zpr-db-
Uptime Trace Level Security SNMP Listener Parameter File Listener Log File int1/listener_ib/alert/log.xml Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_IB)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.30.86)(PORT=1522))) Services Summary... Service "DBNAME" has 2 instance(s).
Instance "DBNAME1", status READY, has 1 handler(s) for this service...
Instance "DBNAME2", status READY, has 1 handler(s) for this service... The command completed successfully
