Storage - Fibre Channel zoning on a Cisco MDS.

I've recently been involved in setting up a new HP MSA2040 Array System. It has Fibre Channel SAN Controllers whereas previously I have set up MSA's with iSCSI Controllers.

There are a few differences between the two; with Fibre Channel you need hardware that supports the Fiber Channel Protocol (FCP) on both the switch side and the host side (FC HBA's). You have Fibre Optic cables plugged into Fibre Optic Transcievers (SFP+'s) into Fabric Switches (Cisco MDS in this case) in one end and the Fibre Optic cable plugged into an SFP+ into the Server FC HBA on the other end. So it can be significantly more expensive.

With iSCSI you can use your existing Ethernet hardware, using standard 1Gb Ethernet NICs and CAT5/6 patch leads, which is quite slow. The 10Gbe Ethernet Standard however is increasing in popularity and comes in standard Ethernet with Cat6 leads and 10Gbe NICs or using 10Gbe SFP+ Switches & 10Gbe SFP+ NICS connected with Direct Attach Cables (DAC) which have SFP+ connectors on either end to get the 10Gb speeds.

iSCSI is basically traditional SCSI commands encapsulated in the IP Protocol and usually over Copper cable, therefore you have the additional overhead of the IP layer on top of the SCSI commands which increases latency.

Fibre Channel was designed for storage, it has very low latency (being optical light) and the FC HBA's offload most of the processing of Fibre Channel Protocol frames (FCP).

Fibre Channel allows the transport of native SCSI commands as well, without the IP layer and via Fibre Optic cable.

There is another standard called Fibre Channel over Ethernet (FCoE) which also requires expensive switches (such as the Cisco Nexus 5000 series).
The idea with this standard is that you have both DATA and Storage protocols through the same switch hardware. As opposed with having separate switches for Storage (FC) and Data traffic (Ethernet), with CNA's instead of Fibre Channel HBA's in your Blade chassis through which both your DATA and Storage traffic pass through to a Cisco Nexus 5000 Switch from where it can split off to your Office network and to your Fibre Channel Switch (Cisco MDS). along with traditional Fiber Channel, Cisco Fabric Path

Another difference is iSCSI uses IP Addresses between initiator and target whereas Fibre Channel uses WWN's (World Wide Names) instead of IP Addresses. Think of WWN's as MAC Addresses, as they look similar and represent the same thing.

So here's how to set up zoning using Fibre Channel on Cisco MDS Fabric Switches. In this case my switch example is two Cisco MDS 9148 Fabric Switches connecting a HP MSA2040 with 2 controllers with 4 FC ports per controller.

I have connected the MSA2040 to the MDS like this:
Controller A ports A1, and B1 to fc1/1, fc1/3 on Switch 1
Controller A ports A3, and B3 to fc1/2, fc1/4 on Switch 2
Controller B ports A2, and B2 to fc1/2, fc1/4 on Switch 1
Controller B ports A4, and B4 to fc1/1, fc1/3 on Switch 2

Enter global configuration mode:
conf t

Configure the interface connected to SAN Controller A1 (i.e powering up)
interface fc1/1
switchport description SAN_Controller_A1
port-license acquire
no shutdown

Configure the interface connected to SAN Controller B
interface fc1/2
switchport description SAN_Controller_B1
port-license acquire
no shutdown

Configure the interfaces connected to your Servers (Or Blade Chassis interconnects). In my example I have Blade Chassis with two fabric interconnects.
interface fc1/7
switchport description Blade_Chassis_Fabric_A
port-license acquire
no shutdown

interface fc1/8
switchport description Blade_Chassis_Fabric_B
port-license acquire
no shutdown

Do the same on both switches, using the correct interfaces on the right switch.

Port WWN discovery
Now we need to identify the PWWN's (Port WWN's) connected to the above interfaces to add to our config, do this on both switches:

show flogi database
--------------------------------------------------------------------------------
INTERFACE VSAN FCID PORT NAME NODE NAME
--------------------------------------------------------------------------------
fc1/1 10 0x9c0700 21:70:00:xx:xx:xx:xx:xx 21:70:00:xx:xx:xx:xx:xx
fc1/2 10 0x9c0800 20:00:00:xx:xx:xx:xx:xx 20:00:00:xx:xx:xx:xx:xx

In my example above I have shortened the output and changed the real PWWN's to fake ones. But you get the idea. Make a note of the 'PORT NAME' on each connected interface.

Now we are going to create aliases, the alias will be used to simplify the zone config by using aliases instead of port names in the config. Aliases are easier to identify when you or someone else looks back at the config to understand what was done before to make changes in the future (It's also a best practice).

The syntax for the alias is as follows:

fcalias name <alias> vsan 10
member pwwn <Port Name>

For example:

fcalias name <hp_msa2040_A1> vsan 10
member pwwn 21:70:00:xx:xx:xx:xx:xx

From the above you can see that this alias is for the pwwn on fc1/1 (when you ran the show flogi database command)

Do this for all the pwwn's on both switches.

Create the vSAN for this switch fabric
Here we will define the VSAN to be used for the zoning. Each switch will have a different VSAN ID and a different fabric name (FABRICA and FABRICB). Do the same on the 2nd fabric switch and remember to specify a different vsan id and different fabric name.

vsan database
vsan 10 name "FABRIC_A"
vsan 10 interface fc1/1
vsan 10 interface fc1/2
vsan 10 interface fc1/7
vsan 10 interface fc1/8

Create the Zones

Now we are going to create the zone using the above information. Each zone name will contain one server alias and all the SAN Controller aliases defined on that particular switch. Each server will need to see all the ports of the SAN controllers connected to that particular switch for multipathing and redundancy.

zone name esx01_msa2040 vsan 10
member fcalias hp_msa2040_A1
member fcalias hp_msa2040_A3
member fcalias hp_msa2040_B1
member fcalias hp_msa2040_B3
member fcalias esx01_p1

zone name esx02_msa2040 vsan 10
member fcalias hp_msa2040_A1
member fcalias hp_msa2040_A3
member fcalias hp_msa2040_B1
member fcalias hp_msa2040_B3
member fcalias esx02_p1

zone name esx03_msa2040 vsan 10
member fcalias hp_msa2040_A1
member fcalias hp_msa2040_A3
member fcalias hp_msa2040_B1
member fcalias hp_msa2040_B3
member fcalias esx03_p1

Do the same on the 2nd switch but remember to give it a different VSAN ID (e.g. vsan 20) and only the SAN and Server ports connected to it (e.g. SAN Controller Ports A2, A4, B2, B4 and port 2)

Create the Zoneset

zoneset name esx_fabric_a vsan 10
member esx01_msa2040
member esx02_msa2040
member esx03_msa2040

Again, do this on both switches with the correct VSAN ID, correct zoneset name and members.

Activate the Zoneset

To activate the Zoneset you just do the following:

zoneset activate name esx_fabric_a vsan 10

Test connectivity and write changes
Now you need to configure your hosts and the SAN so that they can communicate, I won't cover that for the purpose of this guide as that depends on what SANs and Servers you are using.

Once the connectivity has been established you should write the changes:

copy running-config startup-config
or
write

comments powered by Disqus