Category Archives: Exa Family

Exalogic Installation Series -7- Creating and using Distribution Groups

By default running your Exalogic in a Virtual provides you with, what to Cloud Users, is a single large resource and they can just create vServers and not care about how they are laid down on the the underlying infrastructure. All the Cloud Users will know is that they can create vServers. For example if we have a Quarter Rack (8 Nodes) and our Cloud User creates 8 vServers those 8 vServers may run on 8 distinct nodes or may all run on the same node.

Although in many cases we, as Cloud Users, may not be to worried how the Virtualisation Algorithm decides where to place our vServers there are cases where it is extremely important that vServers run on distinct physical compute nodes. For example if we have a Weblogic Cluster we will want the Servers with in the cluster to run on distinct physical node to cover for the situation where one physical node is lost.

To achieve this the Exalogic Virtualised implementation provides Distribution Groups that define and anti-aliasing policy that the underlying Virtualisation Algorithm will take into account when placing vServers.

It should be noted that Distribution Groups must be created before you create vServers because a vServer can only be added to a Distribution Group at creation time.

Creating A Distribution Group

AccountTo create a Distribution Groups we will first need to select the Account in which we want the Distribution Group to be created. Once we have selected the account we will see the Interface update and Account specific Actions will be displayed within the Action Panes. From the Action pane (or Right-Click on the Account) select the “Create Distribution Group” action.

Actions
This will initiate the create wizard as follows.

Distribution Group Details

Within the first Step of the Wizard we can specify the name of the distribution group and this should be unique. In addition we can provide a detailed description of the group.

Step 1

Distribution Group Configuration

The second step of the configuration wizard allows you to specify the number of elements that are required within this group and will specify a maximum of the number of nodes within you Exalogic. At this point it is always better to specify a group with spare capacity allowing for future expansion. As vServers are added to group the available slots decrease.

Step 2

Summary

Finally the last step of the wizard display a summary of the information entered.

Step 3

Exalogic Installation Series -6- Creating Accounts

Once we have created our Users and Networks we will want to enable the Virtual Data Centre (vDC) for access by the Cloud Users. To facilitate this we will need to create Accounts within the vDC / Cloud and allocate the users to these accounts. Once a Cloud User has been allocated to an account they will be able to access that account and hence create / manage Virtual Servers within that account / Pool.

menuTo create an Account within a vDC / Pool you will need to be logged into Enterprise Manager Ops Centre (EMOC) with the appropriate Role, and this is generally done using you Cloud Administrator, then simply navigate to the vDC Management Accordion, vDC, your Cloud and finally Accounts.

Once you have Accounts highlighted then select “Create Account” via one of the standard methods (Right-Click, Actions or button bar) to initiate the Create Account wizard. If this is the first time you will see an Introduction screen but for subsequent execution this can be disabled. The wizard screens are displayed below.

Specify Account Details

On this screen you will be able to name the Account you are creating and provide a meaningful description. In addition you have the option to provide a number of Name / Value pairs or tags that can be used for searching in the future. The name allocated here will be displayed to all users (Cloud Users) allocated in Step 4.

It should be noted that when an allocated Cloud User logs into EMOC they will only see the Accounts that they have access to below the vDC Management Accordion.
2

Specify Resource Limits

For each account we can specify a number of Resource Limits. These will be used to control the resources allocated to Virtual Servers created within the account. Once the limits have been reached then any attempt to create Virtual Servers within this account will fail and an appropriate message will be logged.
Following creation of an Account the information provided within these screens can be modified and hence the values increase as appropriate if it is found that more resources are required. As part of this screen EMOC will display the available resources within the Pool (Exalogic Rack) which may be over subscribed.

3

Having selected which Public Networks the Account will have access to you will need to specify the limit of IP Addresses available to the account. By default this is 0 which means there are no available IPs and attempting to allocate one will cause the Virtual Server creation to fail. This field can be easily overlooked and you should make sure that an appropriate limit is set.
3

Assign Users

The Assign Users Step will display all available users with either “Cloud Admin” or Cloud User” Roles. You will need to select all users that will require access to this account within this Step.
4

Summary

As a final step a Summary of the defined information is displayed and the user can verify this is correct before selecting Finish to create the account. As mentioned above one common mistake is to not specify a limit on the Public networks and this will default to 0 IP addresses being available on that network within this account.
5
5

Exalogic Installation Series -5- Creating Networks

In the majority or Real World scenarios we will need to access the Virtual Servers running within the Exalogic from an external client network. To facilitate this we will want to leverage the 10Gb Ethernet connection and hence we will need to create 1 or more EoIB networks that can be accessed by the Virtual Servers.

During the installation of the Exalogic 2.0.1 Virtual environment we create a single “EoIB-external-mgmt” network that we could, in theory, use to access the Virtual Servers we create. Although this is possible, assuming it has enough IP Address, this would be bad practice because this network is intended solely for management functionality and access to the Control VMs. Therefore to provide the Virtual Servers with external Ethernet access we will need to create additional EoIB interfaces. Each of these will need to be VLAN tagged to provide network isolation and partitioning.

MenuHaving created our Enterprise Manager Ops Centre (EMOC) users we will now have a Network Administrator account that we will use for the next process. We will need to select the Network Accordion and then either Right-Click on the default network and Select “Manage Network” or select the default network and choose the “Manage Network” Action.

Once we have initiated the Manage Networks (this will create a new managed network) wizard it will allow us to specify all the external network information. It should be noted that all the networks that will be need should be created prior to the creation of the Virtual Server. This is because in the first release we are not able to add networks, easily, through EMOC once the Virtual Server has been created.

Define Network

In this Wizard Panel we will need to define the network information and IP range available for configuration later in the wizard. You will see that we specify the initial IP address in a CIDR format, I do not intend to explain CIDR, then the gateway and MTU to be used for  this network.

You will notice that as well as the name and description you are able to defined a number of Tags, name/value pairs, that can be used for searching within EMOC.
Step 1

Specify Managed Ip Address Range

Here we define the IP addresses that are actually available within this network. You will notice that we can specify a number of ranges and hence you do not need to have large contiguous ranges. The only requirement is that they are permissible within the network range define in the first wizard panel.
Step 2

Assign Fabric

Once the IP addresses have been defined we will need to assign them to an existing Fabric and because we want to define EoIB we will use the existing EoIB fabric to allow us access to the external 10Gb network. As mentioned previously each of these networks must be VLAN tagged to allow the correct partitioning and routing across the fabric. So having selected the appropriate fabric we will need to enter the VLAN Id. We do not need to enter a Partition Key because EMOC will generate us a unique P-Key as part of the network creation process.
Step 3

Static Routes

If you need to define any static routing information for this network range we can do this in the next wizard panel and this will be automatically added to the appropriate tables. Within my example this is not the case.
Step 4

Network Services

We no specify additional information, such as the NTP server and Domain name. These are all optional fields and, as such, can be left blank.
Step 6

Assign Network

At this point we can assign the new network to a specific server pool(s)  within the rack or if we do not select a pool the whole rack. Thus we can restricted the network access to specific parts of the rack.
Step 6
If we specify a Server Pool we will be taken to Step 7 and allowed to configure the interfaces.

Proxy Controls

At this point we can also specify if the network is to be manage / associated with one of the Proxy Controllers. This is not a requirement and in general we can leave this network free from association with any of the controllers.
Step 8

Summary

The Summary panel displays all information entered, for review, and once Finish is selected a Job will be created to  create the network which will then appear in the list of available networks.
Summary
Details

Exalogic Installation Series -4- Creating Cloud Users

Creating Cloud Users and Administrators will be one of the first tasks when setting up a new Exalogic 2.1 environment. We will step through the simple process of creating users and describe a few key user types. Initially we will need to login as either the root user or the exl-admin user, that is a user with the User Admin Role.

Before adding users to the Exalogic 2.1 environment they must exist as either local users on the physical machine running the Exalogic Control Virtual Server or existing within an appropriate repository, LDAP etc, used by the machine for authentication. This is required because Enterprise Manager Ops Centre 12c (EMOC) does not store any account authorisation information instead this is left tot he underlying OS. It is assumed within this blog that this has been done.

To create a user simply open the “Administration” Accordion (Drawer), expand the Enterprise Controller then select “Local Users. This will present you with the following

User Administration

You can see from the image that we have 3 options for Adding a user and selection of any of these will display the following Dialog.

Add User

As mentioned earlier the User Name must match that of an OS based account to provide the authentication but we will need to specify the  EMOC account Roles and these will defined what functionality the new user can access.

Cloud Administrator

Cloud AdminThis user type can be created by adding the “Cloud Admin” to the selected roles, when creating a user, and will provide access to the Management functionality below vDC Management thus allowing for the creation of new accounts and resources. It should be noted that a Cloud Administrator can administer all user accounts within the system.

Cloud User

Cloud UserThe Cloud User is allowed to simply access the vDC accounts that they have been given access to by a Cloud Administrator. For each of the accounts they will be able to:

  • Create Private vNets
  • Create vServers
  • Manage vServer Life Cycle
  • ManageVolumes
  • Create Distribution Groups
  • Upload Templates

In general their will be many Users to limited Administrators.

Network Administrator

Network AdminThe Network Administrator will be used to create additional EoIB networks to be used by the Virtual Servers to access the external network. Although by default the installation of Exalogic 2.1 will provide a small EoIB management network this is not intended to be used for external access from within Virtual Servers. Instead 1 or more VLAN Tagged networks should be created prior to building the Virtual Server infrastructure.

Role Permissions

Roles

Exalogic Installation Series -3- Modifying the Default Shipped (Base) Template

Having installed your Exalogic Virtual environment by default you have a single template which can be used to create your vServers. Although this template is suitable for creating simple test or development vServers it is recommended that you look at creating your own custom vServers that match the environment you wish to build and deploy. Therefore this Tea Time Snippet will take you through the simple process of modifying the standard template.

Before You Start

To edit the template you will need the Oracle ModifyJeos Utility which can be downloaded from the eDelivery Site.

eDelivery
eDelivery
eDelivery

Once the ModifyJeos Utility has been downloaded we can install the rpms onto either an existing vServer or one of the Control vServers.

rpm -ivh ovm-modify-jeos-1.0.1-10.el5.noarch.rpm
rpm -ivh ovm-template-config-1.0.1-5.el5.noarch.rpm

Alternatively you can install the modify jeos packages on a none Exalogic OEL installation or a VirtualBox image. If you are doing this, assuming OEL 5u8, you will need the following rpms.

rpm -ivh ovm-modify-jeos-1.0.1-10.el5.noarch.rpm
rpm -ivh ovm-template-config-1.0.1-5.el5.noarch.rpm

Base Template

If you have installed the modify onto a vServer running on the Exalogic then simply mount the /export/common/images from the ZFS storage and you will be able to find the el_x2-2_base_linux_guest_vm_template_2.0.1.1.0_64.tgz (or similar depending which version you have) template file. Alternatively the latest can be downloaded from the eDelivery Site.

eDelivery
eDelivery
eDelivery

Now we have the Template tgz we will need the extract it as follows:

tar -zxvf  el_x2-2_base_linux_guest_vm_template_2.0.1.1.0_64.tgz

This will create a directory called BASE which will contain the System.img (VServer image) and vm.cfg (VServer Config information). This directory should be renamed to something more meaning full that indicates what we have done to the template and then the Simple name / name in the vm.cfg editted for the same reason.

Modifying the Template

Resizing Root File System

By default the shipped template has a root size of 4 GB which will leave a vServer created from it running at 90% full on the root disk. We can simply resize the template by executing the following:

modifyjeos -f System.img -T <New Size MB>

For example to imcrease the default 4 GB to 40 GB we would execute:

modifyjeos -f System.img -T 40960

Resizing Swap

We can modify the size of the swap space within a template by executing the following:

modifyjeos -f System.img -S <New Size MB>

For example to increase the swap from the default 512 MB to 4 GB we would execute:

modifyjeos -f System.img -S 4096

Resizing Note

If you are resizing both the root and the swap space you will need to execute the root resize first because it will reset the swap to a default size.

Changing RPMs

Adding RPMs

To add RPMs using modifyjeos, complete the following steps:

  1. Add the names of the new RPMs in a list file, such as addrpms.lst. In this file, you should list each new RPM in a separate line.
  2. Ensure that all of the new RPMs are in a single directory, such as rpms.
  3. Run the following command to add the new RPMs:
modifyjeos -f System.img -a <path_to_addrpms.lst> -m <path_to_rpms> -nogpg

Where <path_to_addrpms.lst> is the path to the location of the addrpms.lst file, and <path_to_rpms> is the path to the directory that contains the RPMs. The -nogpg option eliminates signature check on the RPMs.

Removing RPMs

To remove RPM s using modifyjeos, complete the following steps:

  1. Add the names of the RPMs (the ones you want to remove) in a list file, such as removerpms.lst. In this file, you should list each RPM in a separate line. The Oracle Exalogic Elastic Cloud Administrator’s Guide provides a list of all RPMs that must not be removed from the vServer.
  2. Run the following command to remove the RPMs:
modifyjeos -f System.img -e <path_to_removerpms.lst>

Where <path_to_removerpms.lst> is the path to the location of the removerpms.lst file.

Mounting the System.img

For all other modifications that are not supported by the modifyjeos command (adding you custom yum repositories, pre configuring NTP, modify default NFSv4 Nobody functionality, etc) we can mount the System.img and access it directly. To facititate quick and easy mounting/unmounting of the System.img I have put together the simple scripts below.

MountSystemImg.sh

#!/bin/sh

# The script assumes it's being run from the directory containing the System.img

# Export for later i.e. during unmount
export LOOP=`losetup -f`
export SYSTEMIMG=/mnt/elsystem
# Make Temp Mount Directory
mkdir -p $SYSTEMIMG
# Create Loop for the System Image
losetup $LOOP System.img
kpartx -a $LOOP
mount /dev/mapper/`basename $LOOP`p2 $SYSTEMIMG
#Change Dir into mounted Image
cd $SYSTEMIMG

UnmountSystemImg.sh

#!/bin/sh

# The script assumes it's being run from the directory containing the System.img

# Assume the $LOOP & $SYSTEMIMG exist from a previous run on the MountSystemImg.sh

umount $SYSTEMIMG
kpartx -d $LOOP
losetup -d $LOOP

Alternative MountSystemImg.sh

#!/bin/sh

# The script assumes it's being run from the directory containing the System.img

# Export for later i.e. during unmount
export LOOP=`losetup -f`
export SYSTEMIMGDIR=/mnt/elsystem
export SYSTEMIMG=System.img
export TEMPLATEDIR=`pwd`

# Read Parameters
while [ $# -gt 0 ]
do
 case "$1" in
  -i) SYSTEMIMG="$2"; shift;;
  *) echo ""; echo >&2 \
      "usage: $0 [-i <System Image Name (Default System.img)> "
      echo""; exit 1;;
  *) break;;
 esac
 shift
done

# Make Temp Mount Directory
mkdir -p $SYSTEMIMGDIR
# Create Loop for the System Image
losetup $LOOP $SYSTEMIMG
kpartx -a $LOOP
mount /dev/mapper/`basename $LOOP`p2 $SYSTEMIMGDIR
#Change Dir into mounted Image
cd $SYSTEMIMGDIR
echo "######################################################################"
echo "###                                                                ###"
echo "### Starting Bash shell for editing. When completed log out to     ###"
echo "### Unmount the System.img file.                                   ###"
echo "###                                                                ###"
echo "######################################################################"
echo
bash
cd ~
cd $TEMPLATEDIR
umount $SYSTEMIMGDIR
kpartx -d $LOOP
losetup -d $LOOP
rm -rf $SYSTEMIMGDIR

Modifying Files in the System.img

Once you have mounted the System.img script, above, you will be placed in a bash shell and located in the /mnt/elsystem directory. This directory represents the root of the System.img file and from here we can access the various files within, what will be, the vServers file system.

ls

One thing that we need to remember is to remove the / when editing files or creating links and so :

vi /etc/hosts becomes vi etc/hosts
vi /etc/fstab becomes vi etc/fstab

When creating links these must all be done as relative link so to set the System Time Zone we would normally execute:

ln -sf /usr/share/zoneinfo/GMT /etc/localtime

But this is replaced with:

cd etc
ln -sf ../usr/share/zoneinfo/GMT localtime

Packaging the Template

Once you have finished modifying the template it can be simply repackaged and then imported into EMOC .

To do this we will simply cd to the directory above that containing the modified files and execute the following:

tar -zcvf <New Template Name>.tgz <New Template Directory>

The resulting.tgz file can be copied to the images directory on the ZFS and uploadd using the IB network.