Category Archives: Exa Family

Exalogic Installation Series -2- Importing Public Server Template

Exalogic Virtual Tea Break Snippets – Importing Public Server Template

Before we can create Virtual Servers within Enterprise Manager Ops Centre (EMOC) we will need to import an appropriate Server Template that will be used to create the Virtual Server. Server templates are associated with accounts and hence may be imported on an account by account basis or for common base templates we can import them once and make them Public. Once we have configured the imported template to be Public it will be available to all accounts and users.

Serve templates can be uploaded to an account in 3 simple steps although this initial upload will make the template private to an account its properties can be set to public by simply changing the “Public” flag within the account Server Template tab. The following steps are required to import a template and make it public.

To import a template you will need to navigate to your account and select “Upload Server Template”. This will initiate the Wizard below.

Identify a Server Template

The first screen simply allows you to name the image that will be imported and provide a meaningful description. In addition we can add some Searchable tags.

Upload Template

We can now specify the location of the template file to that will be uploaded into the OVM repository using either a local file or, more commonly, a URL pointing to a shared location on the internal Exalogic Storage.

e.g http://<ZFS IP>/shares/export/common/images/<DIR>/<FILE>

The URL can easily be found by navigating the ZFS BUI and locating the file on disk and accessing the http mode to copy the URL.


Finally a summary of the information entered will be displayed for validation and you can then Upload the file into the repository.

Set Template Public

Once the Server template upload has completed it will appear in the “Server Template” tab within the central pane for the selected Account. Following the initial upload you will notice that the template will have the Public flag set to no (see below). Selecting the Server Template will enable a number of additional menu items will be enabled.
To make the selected Template public we will need to select the “Register Server Template” button (highlighted below) and confirm the change. Once the change has been confirmed you will notice that the template is now Public and hence available to all accounts and registered Users.


Exalogic Installation Series -1- What Next After a Exalogic Clean Install

Having installed your Exalogic with the 2.0.1.x software you now have a fairly empty system that simply contains the Control Stack and nothing else. At this point we need to consider what your next steps should be and what we need to define and create before building you Virtual environment. This short blog will provide some recommendations for what you define / build and when based on a number of Exalogic installations. If this sequence is followed then you should have everything in place before your create your first vServer.

The key assumption of this blog entry is that you have spent time designing you virtual landscape and and know what External access etc is required for the resulting system.

Create ELAdmin and CloudAdmin Users

Following the installation of your Exalogic Environment the environment will be configured with a single root user. This should only be used a single time to create the ELAdmin and CloudAdmin users. These user should be created with the following Roles:

  • ELAdmin : Exalogic System Admin , Cloud Admin and User Management Admin Roles
  • CloudAdmin : Cloud Admin and User Management Admin Roles.

Once these are created the root login should not be used; the combination of the two users, above, provides all the functionality required on your Exalogic Rack.

Creating Client Access EoIB Networks

Before we can access the vServers we will create they will need 10GB Access to your network. By default during the Installation we create a single 10GB E0IB network that is used to access Exalogic Control Stack. This network will be on its own VLAN an must not be used to access your client vServer. Instead we will need to create 1 or more Client EoIB 10 GB VLAN tagged networks to be used with your client vServers.

Create vServer Types

Although during the installation of the Exalogic Virtual environment 3 default vServer Types are created these are likely not to match the requirements of your system. Therefore you should review your requirements and create vServer Types that are appropriate to your requirements.

Create Common Account

By design vServer Template files are installed on an Account by Account basis but this can lead to the same template being installed multiple times. Rather than install the same template in each account we can create a common account that uses no resources but is simply a place holder account that will be used to store commonvServer Templates.

Import Server Template

The install process for Exalogic Virtual will place an example vServer Template on the ZFS Storage appliance. This should be uploaded to the Common Account and switched to Public.

Create Custom Template

If the standard example vServer Template does not meet the needs of your environment then a new Template should be created based on the example template. We need to base this on the example because it has a number of Exalogic specific rpms and functionality built in.

Once you have created your template(s) then they should be uploaded to the common account.

Create Accounts (Tenancies)

Now we have all the common requirements in place we ca being to build the Accounts (Tenancies) that are specific to your environment.

Create Distribution Groups

Distribution Groups are the only method within Exalogic Virtual environment that allows you to force a cluster of vServers to run on separate Compute Nodes. These must be created before the vServers and a vServer will need to be added to a Distribution Group during it’s creation.

Create Private vNets

The vServers we create within an Account may need to communicate internally across the Infiniband network and to do this we will need to create one or many Private vNets that will be assigned during vServer creation.

Create Volumes / vServers

At this point we can now create our Volumes, if required, and vServers allocating all the appropriate resources defined above. Then install software ass appropriate.

Using LDAP for Shared Authentication (NFS v4 Requirement instead of NFS v3)

NFS v3 is a stateless protocol so in order to achieve file locking it relies on the Network Lock Manager (NLM) to maintain consistency of any file. The NLM protocol in turn relies on the client IP address to keep a track of the owner of a particular lock. Because an Exalogic is a multi-homed machine (many IP addresses) it is possible for a lock request to come from one IP address and a release from another IP, this causes the NLM to not honour the request and a lock is not released. This causes problems with Weblogic as it is then unable to get the lock and hence fail to startup. Using NFS v4 resolves this problem because NFS v4 is a stateful protocol, the client being given a client ID during the initial negotiation phase and then re-using the same client ID for all subsequent requests. This means that the client IP address is not used to track lock ownership and so the impact of multiple IP addresses on the client is not an issue and hence Exalogic operates as desired using NFS v4.
NFS v4 is a fairly significant improvement to the NFS v3 protocol, in particular around the area of security where a client accessing the share also includes the user’s security principle which is used to secure the file. This means that NFSv4 requires some form of shared repository of users so that it can check this to provide appropriate access to any particular file or directory. Two options are widly used in the industry for this functionality, historically the Network Information Service (NIS) also known as Yellow Pages (YP) the other option is to make use of a Directory Server via the Lightweight Directory Access Protocol or LDAP.
This posting explains how to install and get a very basic LDAP server operational and configure the Linux compute nodes and ZFS storage appliance of an Exalogic to use LDAP to access the users, passwords, hosts etc. from the directory.

Directory Server Architecture

In the first place a very simple non-HA approach is taken with one compute node of an Exalogic machine nominated as the host for the directory and all other nodes use its services to enable centralised authentication and management.  Future postings will cover of setting up the directory to be multi-master/master-slave configurations so that the system becomes fault tolerant.
We will also be considering security, both at the transport layer where we will change the directory to use LDAPS so that it is not possibe to pick up transports from the network interchange and within the directory structure so that users have access to various combinations of compute nodes.


First off create a group and user on your machine that we can use for testing the directory.
For example:-
# groupadd -g 500 donald
# useradd -g donald -d /home/donald -m -s /bin/bash -u 500 donald
Then login as this user to check that all is fine.

Installing openldap

The first step towards getting the shared repository setup is to get the directory server configured. In this example we are using openldap which is available as a package in the Oracle public Yum repository, alternatives include the likes of Oracle Internet Directory, Oracle Enterprise Directory Server, OpenDS etc. For openldap a number of packages must be installed onto the compute nodes. For the nodes that will host a directory server instance (Master or Slave) the following packages are required:-
  • openldap
  • openldap-clients
  • openldap-devel
  • nss_ldap
  • openldap-servers
and for the machines that will only operate as a client to the directory service the following packages should be installed:-
  • openldap
  • openldap-clients
  • openldap-devel
  • nss_ldap
In order to do this either setup your machine to connect to the Oracle public Yum repository or download the necessary packages.
For installing using yum:-
# yum install openldap-clients
Loaded plugins: security
Setting up Install Process
Resolving Dependencies
–> Running transaction check
—> Package openldap-clients.x86_64 0:2.3.43-12.el5 set to be updated
–> Finished Dependency Resolution

Dependencies Resolved

Package Arch Version Repository Size
openldap-clients x86_64 2.3.43-12.el5 el5_u5_base 221 k

Transaction Summary
Install 1 Package(s)
Upgrade 0 Package(s)

Total download size: 221 k

Is this ok [y/N]: y
Downloading Packages:
openldap-clients-2.3.43-12.el5.x86_64.rpm | 221 kB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : openldap-clients 1/1

openldap-clients.x86_64 0:2.3.43-12.el5

# yum install openldap-devel

# yum install openldap-servers
(On my test systems openldap and nss_ldap packages had already been installed.)

Configuring the Directory Server

The main configuration files for openldap are all held in /etc/openldap. Open slapd.conf and set the value of the suffix and rootdn to something appropriate for the environment. This file also contains the directory on the file system that will be used to host the directory instance. By default this is set to /var/lib/ldap, change this to match your directory name.
You can also set the password for the Manager user by putting it into the slapd.conf file. This can be set unencrypted but should really be an encrypted value for security. Openldap includes the utility slappasswd which will encrypt a password entered and give the encrypted value back to standard out. Use this facility to encrypt your password and then put the value into the configuration file.
For example:-
[root@vbelcn01 openldap]# slappasswd
New password:
Re-enter new password:
And copy the encrypted value into the slapd.conf file.
# ldbm and/or bdb database definitions

database bdb

suffix “dc=el01,dc=com”
rootdn “cn=Manager,dc=el01,dc=com”
rootpw {SSHA}Y0iT4bFZGz1QQ34NTjqwmc4N7BFpURsN
directory /var/lib/ldap/
Now create the directory that will be used to host the Directory Server instance and set the ownership to the ldap user. (The ldap user is created when the packages are installed.)
# mkdir /var/lib/ldap/
# chown ldap:ldap /var/lib/ldap/

Create the directory and populate with existing values

The directory is to be used to manage the users, groups, hosts etc., openldap comes with a number of scripts that can be used to copy existing values into the directory. These scripts are installed into /usr/share/openldap/migration. Start by editing the script and replace all instances of the domain with the domain that you are using for your directory.
For example, in our case we are using the domain so substitute all string occurances of “padl” with “el01”.
Run updatedb then locate DB_CONFIG to create the configration file for the database.
[root@vbelcn01]# updatedb
[root@vbelcn01]# locate DB_CONFIG
[root@vbelcn01]# cp /etc/openldap/DB_CONFIG.example /var/lib/ldap/
Run the script to populate the directory with the values from your various files.
[root@vbelcn01 tmp]# cd /usr/share/openldap/migration
[root@vbelcn01 migration]# ./ 
Creating naming context entries…
Migrating groups…
Migrating hosts…
Migrating networks…
Migrating users…
Migrating protocols…
Migrating rpcs…
Migrating services…
Migrating netgroups…
Importing into LDAP…
Migrating netgroups (by user)…
Migrating netgroups (by host)…
Preparing LDAP database…
=> bdb_tool_entry_put: id2entry_add failed: DB_KEYEXIST: Key/data pair already exists (-30996)
=> bdb_tool_entry_put: txn_aborted! DB_KEYEXIST: Key/data pair already exists (-30996)
slapadd: could not add entry dn=”dc=vbel,dc=com” (line=5): txn_aborted! DB_KEYEXIST: Key/data pair already exists (-30996)
Migration failed: saving failed LDIF to /tmp/nis.ldif.z17552

The error reported is linked to the services available on the server and as we are not planning on using these via LDAP lookups the error does not seem to impact the behaviour of the directory.   Be warned, if you are doing this in an environment that you have been using for a while take care to ensure you have no duplicate entries in your hosts file.  If you do then the migrate script fails to update the directory due to a duplicate entry and the script will stop.  The error reported will be similar to:-

bdb_tool_entry_put: id2entry_add failed: DB_KEYEXIST: Key/data pair already exists (-30996)
bdb_tool_entry_put: txn_aborted! DB_KEYEXIST: Key/data pair already exists (-30996)
slapadd: could not add entry dn=”,ou=Hosts,dc=el01,dc=com” (line=469): txn_aborted! DB_KEYEXIST: Key/data pair already exists (-30996)
Migration failed: saving failed LDIF to /tmp/nis.ldif.DA3373

Now we want to start up the directory. Prior to issuing the command to start the daemon we must change all the files in the directory to be owned by ldap:ldap.

[root@vbelcn01 migration]# cd /var/lib/ldap/
[root@vbelcn01]# ls -l
total 3312
-rw-r–r– 1 root root 2048 Jan 4 11:00 alock
-rw——- 1 root root 122880 Jan 4 10:59 cn.bdb
-rw——- 1 root root 24576 Jan 4 11:00 __db.001
-rw——- 1 root root 368640 Jan 4 11:00 __db.002
-rw——- 1 root root 270336 Jan 4 11:00 __db.003
-rw——- 1 root root 98304 Jan 4 11:00 __db.004
-rw——- 1 root root 557056 Jan 4 11:00 __db.005
-rw——- 1 root root 24576 Jan 4 11:00 __db.006
-rw-r—– 1 root root 921 Jan 4 10:54 DB_CONFIG
-rw——- 1 root root 86016 Jan 4 10:59 dn2id.bdb
-rw——- 1 root root 12288 Jan 4 10:59 gidNumber.bdb
-rw——- 1 root root 245760 Jan 4 10:59 id2entry.bdb
-rw——- 1 root root 10485760 Jan 4 11:00 log.0000000001
-rw——- 1 root root 8192 Jan 4 10:59 loginShell.bdb
-rw——- 1 root root 8192 Jan 4 10:59 memberUid.bdb
-rw——- 1 root root 8192 Jan 4 10:59 nisMapName.bdb
-rw——- 1 root root 36864 Jan 4 10:59 objectClass.bdb
-rw——- 1 root root 8192 Jan 4 10:59 ou.bdb
-rw——- 1 root root 20480 Jan 4 10:59 uid.bdb
-rw——- 1 root root 8192 Jan 4 10:59 uidNumber.bdb
[root@vbelcn01]# chown -R ldap:ldap *
Now start the DB and ensure it starts on boot.
[root@vbelcn01]# service ldap start
Checking configuration files for slapd: config file testing succeeded
[ OK ]
Starting slapd: [ OK ]
[root@vbelcn01]# chkconfig ldap on
[root@vbelcn01]# chkconfig –list ldap
ldap 0:off 1:off 2:on 3:on 4:on 5:on 6:off
Then check that it is accessible and contains the values expected by issuing an ldapsearch.
[root@vbelcn01]# ldapsearch -x -b “ou=People,dc=vbel,dc=com” ‘(uidNumber=500)’
# extended LDIF
# LDAPv3
# base <ou=People,dc=vbel,dc=com> with scope subtree
# filter: (uidNumber=500)
# requesting: ALL

# donald, People,
dn: uid=donald,ou=People,dc=vbel,dc=com
uid: donald
cn: Donald Forbes
objectClass: account
objectClass: posixAccount
objectClass: top
objectClass: shadowAccount
userPassword:: e2NyeXB0fSQxJHNtMHNQRVpLJG9XUm5VdGkzNnJJc0NkOEVMNTU5Mi4=
shadowLastChange: 15342
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash
uidNumber: 500
gidNumber: 500
homeDirectory: /home/donald
gecos: Donald Forbes

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Note – in this case I narrowed the search criteria down to specify my own user record that is identified with the UID of 500. The search criteria can be wider to return all the hosts, users, services etc.
[root@vbelcn01]# ldapsearch -x -b “dc=vbel,dc=com” ‘(objectclass=*)’

Configuring a client to use LDAP

Two approaches possible, either by using the authconfig command which will automatically set the values in multiple files or by manually editing the files.

The authconfig command will be similar to shown below:-

 authconfig –enableldap –enableldapauth –ldapserver=<LDAP server IP>:389 –ldapbasedn=”dc=el01,dc=com” –update

It is also important to edit the /etc/idmapd.conf file to add the domain as described in the section about editing individual files.

If you want to do this by hand then:-
First edit the /etc/ldap.conf file to specify the directory server host IP address and the base DN used.

base dc=el01,dc=com
Edit the /etc/nsswitch.conf file to specify all the entries that you would like to have resolved in ldap, specifying the resolution order. For example:-
passwd: files ldap
shadow: files ldap
group: files ldap
Setup the identity mapping ready for mapping NFS v4 users. This is managed via the idmapd daemon in Linux. Edit the /etc/idmapd.conf file and specify the domain name you wish to use. For example setting it to el01 would result in an idmapd.conf file that looks like:-
[root@vbelcn01 etc]# cat /etc/idmapd.conf

Verbosity = 0
Pipefs-Directory = /var/lib/nfs/rpc_pipefs
Domain = el01


Nobody-User = nobody
Nobody-Group = nobody

Method = nsswitch

Now restart the idmapd service.
# service rpcidmapd stop
# service rpcidmapd start

Test your users

Firstly check that you can log in as the user you created at the beginning of this exercise. If all is OK then we can can continue to create a new user in LDAP and see if we can authenticate as this user.
There are various command line tools that can be used to add users, modify them etc. – ldapadd ldapsearch etc. However for a test system you may well have access to the GUI tools, there are many available but for flexibility I have found the Apache studio to be very useful. ( Using one of these tools create a user similar to the one you created using the unix command useradd. (I exported the my user to an LDIF file, edited the file to change the dn, cn, uidNumber, uid and password then imported the changed ldif back into the directory. This approach means all the objectClass requirements are met.)
Having got the new entry in the directory then attempt to log in as that user, change its password and logout then re-authenticate.  (Note – If you are having a problem authenticating try setting the password in the directory to use crypt with a new salt.)

Setup the Shared Storage to use LDAP Authentication

Select the LDAP service and specify the root dn to search from, namely dc=el01,dc=com assuming this was the suffix you are using in your directory server. Select the Search Scope to be “sub-tree” which is a recursive search and will hunt all units below the base dn. Now click on the + sign by the servers to put in the IP address and port of the server running the directory.
Enable the LDAP service.
To configure the ZFS storage appliance so that it uses LDAP simply ensure that the NIS service is turned off and the NFS service has the option to use the DNS domain unselected. Then you need to put into the custom identity domain field the value of the domain as used in the /etc/idmapd.conf file on each client compute node. The default name for this is localdomain but it is recommended that you switch this to reflect your environment – eg. el01.

Testing the Shared Storage is configured correctly.

Having made the configuration changes to the shared storage the simplest mechanism to ensure that it has been configured correctly is to create a project and a share, giving the share “Root Directory Access” of one of the users & group that you have setup in the directory server. The software complains if the username/group are invalid. If this is Ok you can double check by “shelling” out to the command line and list the directory you have just created for the share, the file permissions should match the user/group you have in LDAP.
For example, the configuration details for a test share on /export/ldaptest/test are shown below:-
vbzfs:> shares
vbzfs:shares> select ldaptest
vbzfs:shares ldaptest> select test
vbzfs:shares ldaptest/test> ls
aclinherit = restricted (inherited)
atime = true (inherited)
casesensitivity = mixed
checksum = fletcher4 (inherited)
compression = off (inherited)
dedup = false (inherited)
compressratio = 100
copies = 1 (inherited)
creation = Wed Jan 11 2012 12:19:52 GMT+0000 (UTC)
logbias = latency (inherited)
mountpoint = /export/ldaptest/test (inherited)
normalization = none
quota = 0
quota_snap = true
readonly = false (inherited)
recordsize = 128K (inherited)
reservation = 0
reservation_snap = true
secondarycache = all (inherited)
shadow = none
nbmand = false (inherited)
sharesmb = off (inherited)
sharenfs = sec=sys,anon=ldaptest,rw=@,root=@ (inherited)
snapdir = hidden (inherited)
utf8only = true
vscan = false (inherited)
sharedav = off (inherited)
shareftp = off (inherited)
sharesftp = off (inherited)
sharetftp = (inherited)
pool = exalogic
canonical_name = exalogic/local/ldaptest/test
exported = true (inherited)
nodestroy = false
space_data = 31K
space_unused_res = 0
space_snapshots = 0
space_available = 69.9G
space_total = 31K
root_group = donald
root_permissions = 755
root_user = ldaptest
origin =

snapshots => Manage snapshots
replication => Manage remote replication
users => View per-user usage and manage user quotas
groups => View per-group usage and manage group
shadow => Manage shadow data migration

vbzfs:shares ldaptest/test> 

Note – This is simply to test the shared storage has the correct visibility of the directory server and is authenticating correctly.  Under normal circumstances the root user and group should be set to nobody:nobody for security reasons.  Only changing when there is a specific reason to do so.

Testing the NFS version 4 mounts

Now we are all ready to go and we would like to mount the nfs v4 shares. To do this persistently we need to firstly create the directory where we will mount the share.
# mdkir -p /u01/ldaptest/test
And then add an entry to the /etc/fstab file to specify the mount point:
vbzfs:/export/ldaptest/test /u01/ldaptest/test nfs4 rw,bg,hard,nointr,rsize=131072,wsize=131072 0 0
This can be mounted with the command
# mount -a
and unmounted
# umount -a -t nfs4 (to unmount all nfs4 partitions listed in the /etc/fstab file.)
# umount /u01/ldaptest/test (to unmount the one specific partition.)

Need to Remount File Systems After System Reboots When NFSv4 Is Used

When you use NFSv4, you are required to re-mount all file systems by running the mount -a command on Oracle Linux each time you reboot the compute node.

To automatically re-mount the file systems rebooting a compute node running Oracle Linux, complete the following steps:

1.Log in to the compute node as a root user.

2.Create a script named nfs4_automount_on_init in /etc/init.d/ directory (Log in as root user)

# vi /etc/init.d/ nfs4_automount_on_init

3.Add the following to the script:

#chkconfig: 345 80 05
#description: NFSv4 Automount
case “$1” in
mount -a

In chkconfig: 345 80 05, the first number 345 indicates the runLevels. The second number 80 is the start priority, and the third number 05 is the stop priority. The second number and the third number should be unique among all registered services, so you may need to adjust the values for your environment, if you have installed any other services.

4.Set up executable permission on the script:

# chmod +x /etc/init.d/ nfs4_automount_on_init

5.Run the following command to register the script:

# chkconfig –add nfs4_automount_on_init

6.Optionally, run the following command to verify the registration of the script:

# chkconfig –list