IBM pSeries (AIX) to Sun StorageTek 2540

We have a IBM pSeries 505 running AIX 6.1 that we use for product compilation and testing. The 505 is a 1U, entry-level POWER server with the capacity for two internal disks. In order to provision additional disk space so we can run Workload Partitions (wpars), we've added a single port Fibre Channel HBA.

The Common Array Manager (CAM) software that Sun provides to manage the 25x0 series of arrays (that form the heart of our SAN) allows the administrator to define an initiator which has a "host type" (i.e., what OS the host is running). Among the list of supported host types are the following for AIX:

• AIX
• AIX (with Veritas DMP)
• AIX (Discretionary Access Control)

The "right" option depends on the software running on the server. As I don't have Veritas DMP, and don't know what Discretionary Access Control is, I opted for AIX. Note, this doesn't appear to be documented in any of Sun's manuals...!

So with a volume setup on the array and mapped to the AIX server, it was time to see what the AIX server discovered.

In theory, booting the AIX server should find the new disks, but if you don't want to reboot, run the cfgmgr command. This appears to scan for new devices and the output can be checked by running:

bash-3.2# lsdev -Cc disk

hdisk0 Available 06-08-01-5,0 16 Bit LVD SCSI Disk Drive

hdisk1 Available 06-08-01-8,0 16 Bit LVD SCSI Disk Drive

hdisk2 Available 01-08-01 Other FC SCSI Disk Drive

hdisk3 Available 01-08-01 Other FC SCSI Disk Drive

hdisk4 Available 01-08-01 Other FC SCSI Disk Drive

hdisk5 Available 01-08-01 Other FC SCSI Disk Drive

hdisk6 Available 01-08-01 Other FC SCSI Disk Drive

hdisk7 Available 01-08-01 Other FC SCSI Disk Drive

hdisk8 Available 01-08-01 Other FC SCSI Disk Drive

hdisk9 Available 01-08-01 Other FC SCSI Disk Drive

hdisk10 Available 01-08-01 Other FC SCSI Disk Drive

hdisk11 Available 01-08-01 Other FC SCSI Disk Drive

The first two disks are internal SCSI. The disks labelled hdisk2 to hdisk9 are the management LUNs for the two 2540 arrays we have and can be ignored. The final two disks, hdisk10 and hdisk11, are two views of the LUN published from the array.

The reason there are two LUNs is because the 2540 has two controllers. Although the array is asymmetric active/passive, the server can see the LUN through both controllers at the same time.

To work around this, we need some multipath I/O software. Hunting around Sun's website found the "Dynamic Path Failover (DPF) Drivers for AIX Operating System 63 General Availability" of which there was a download for AIX 6.1. The download requires a software licence (more on that below...).

With the file downloaded to /tmp, the package could be installed by running smit and selecting the download directory (/tmp) and installing the driver. The software installs into /usr/lpp/cbxfc.

Now, a little diversion into the licensing of this driver. You need to register for a licence with Sun, providing your serial number, contract number and site id which proves you own a 2540 array. After that, the licence is free. Quite why you'd want the driver if you didn't own the array is beyond my powers of comprehension! A 30 day licence is provided in /usr/lpp/cbxfc and can be activated by copying the file "license.30day" to "license". It might be interesting to see what happens if you registered the licence a long way in the future and then changing the date back to today... (purely academic interest only, but might be useful if Sun take their time sending the licence through!).

Having installed the driver, it's another trip into smit, devices and unconfiguring the previously discovered SAN disks, hdisk2 through to hdisk11. When removing, specify the option to remove from the device database as well. After doing this, lscfg should not show the SAN disks.

With the SAN disks no longer visible to the AIX server, re-run cfgmgr and then run lscfg again. If everything is working correctly, the disks should reappear, but be labelled differently:


bash-3.2# lsdev -Cc disk
hdisk0 Available 06-08-01-5,0 16 Bit LVD SCSI Disk Drive
hdisk1 Available 06-08-01-8,0 16 Bit LVD SCSI Disk Drive
hdisk2 Defined 01-08-01 Sun StorageTek Universal Xport
hdisk3 Defined 01-08-01 Sun StorageTek Universal Xport
hdisk4 Defined 01-08-01 Sun StorageTek Universal Xport
hdisk5 Defined 01-08-01 Sun StorageTek Universal Xport
hdisk6 Defined 01-08-01 Sun StorageTek Universal Xport
hdisk7 Defined 01-08-01 Sun StorageTek Universal Xport
hdisk8 Defined 01-08-01 Sun StorageTek Universal Xport
hdisk9 Defined 01-08-01 Sun StorageTek Universal Xport
hdisk10 Available 01-08-01-01 StorageTek FlexLine with DPF V4.31P

Ignoring the management LUNs (hdisk2 to hdisk9), note how there is only one SAN LUN visible, hdisk10. Running /usr/lpp/cbxdpf/dputil utility shows the multipathed disk:

# Device Active Standby
hdisk10 cbx0 (fscsi0 0x010200,1) cbx1 (fscsi0 0x030200,1)

The new disk also appears as a physical volume:

bash-3.2# lspv
hdisk0 00048ada53748080 rootvg active
hdisk1 none None
hdisk10 00048ada40cd9d27 None

The disk can now be added to a volume group and used by the AIX system.

Special thanks to @cgibbo on Twitter who spotted my cry for help when I was struggling to get the multipathing working and got in touch. Thanks for the pointers Chris!
.