Linux iSCSI Hints

I needed to deploy a linux file server but give it access to an Equallogic PS6000 storage array and a few LUNS that would become LVM Physical Volumes. The file server had two 1G links to the iSCSI VLAN ( a VLAN used for iSCSI traffic only). I reserved two IP’s for the server:

  • 192.168.130.30
  • 192.168.130.31

The host uses its iSCSI initiator name to access LUNS on the PS6000 disk array so I used FSPROD as the name. I configured the LUNs to be allocated to the file server using Initiator Name not IP address and then configured the LUNs for shared access (just in case). The SAN IP is 192.168.130.4

SETUP

First – Install the iSCSI software tools using YUM.

yum -y install iscsi-initiator-utils

Setup the Initiator name on the file server

cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1994-05.com.redhat:fsprod

Setup Ethernet interfaces

Ensure that the Ethernet ports for the iSCSI (eth2 and eth3 in our case) are defined in /etc/sysconfig/network-scripts/ directory AND are setup with static IP’s on the iSCSI VLAN network. Keep in mind the SAN will use initiator name to connect file server not IP but we still need to talk to the SAN.

#cat ifcfg-eth2
DEVICE=”eth2″
HWADDR=”B8:AC:6F:7F:17:14″
NM_CONTROLLED=”yes”
ONBOOT=”yes”
IPADDR=192.168.130.30
NETMASK=255.255.255.0
BOOTPROTO=static

#cat ifcfg-eth3
DEVICE=”eth3″
HWADDR=”B8:AC:6F:7F:17:16″
NM_CONTROLLED=”yes”
ONBOOT=”yes”
IPADDR=192.168.130.31
NETMASK=255.255.255.0
BOOTPROTO=static

Creating new disks

The process to retrieve a LUN from the SAN and make it a disk in LVM is as follows:

  • Discover new LUN using iSCSI tools
  • Allocate LUN to server via “Login”
  • verify new raw disk using ls -l /dev/sd*
  • fdisk the /dev/sd- to create a disk with a primary partition on it
  • pvcreate the partitioned disk
  • vgextend the volume group to hold the new disk
  • lvextend any existing LV’s as needed.

Finding New LUNS

iscsiadm -m discovery -t sendtargets -p 192.168.130.4

Success!

fsprod.intbne.gxs.com.au iscsi_host # iscsiadm -m discovery -t sendtargets -p 192.168.130.4
192.168.130.4:3260,1 iqn.2001-05.com.equallogic:0-8a0906-97771c10b-b400000000a4e9cd-vss-control
192.168.130.4:3260,1 iqn.2001-05.com.equallogic:0-8a0906-50e71c10b-18e00186a2a4ecd7-fsprod-lvm-1
192.168.130.4:3260,1 iqn.2001-05.com.equallogic:0-8a0906-53671c10b-e6900186a2d4ecd7-fsprod-lvm-2
192.168.130.4:3260,1 iqn.2001-05.com.equallogic:0-8a0906-36271c10b-6dc00186a304ecd8-fsprod-lvm-3
192.168.130.4:3260,1 iqn.2001-05.com.equallogic:0-8a0906-3e271c10b-59900186a384ecd8-fsprod-lvm-4
192.168.130.4:3260,1 iqn.2001-05.com.equallogic:0-8a0906-35e71c10b-88200186a3b4ecd9-fsprod-lvm-200-1
192.168.130.4:3260,1 iqn.2001-05.com.equallogic:0-8a0906-38971c10b-d4300186a3e4ecd9-fsprod-lvm-200-2
192.168.130.4:3260,1 iqn.2001-05.com.equallogic:0-8a0906-b8e71c10b-d3300186a414fd68-fsprod-lvm-200-5
192.168.130.4:3260,1 iqn.2001-05.com.equallogic:0-8a0906-34871c10b-84c00186a4451119-fsprod-lvm-300

Restart iscsi if needed:

(note – this may not be needed as the script calls iscsiadm to logout or login to the SAN).

vlan50-notused-175.intbne.gxs.com.au network-scripts # service iscsid restart
Stopping iSCSI daemon:
Starting iSCSI daemon:                                     [  OK  ]
                                                          [  OK  ]

LUN to Disk Process

We need the file server to log into the SAN for each LUN, the SAN then presents the LUN and we see it in /dev/sd*

iscsiadm -m node --target iqn.2001-05.com.equallogic:0-8a0906-50e71c10b-18e00186a2a4ecd7-fsprod-lvm-1 --login

Output from dmesg should show new disk!

dmesg | tail -20
scsi8 : Broadcom Offload iSCSI Initiator
scsi9 : Broadcom Offload iSCSI Initiator
scsi10 : Broadcom Offload iSCSI Initiator
iscsi: registered transport (tcp)
iscsi: registered transport (iser)
iscsi: registered transport (be2iscsi)
scsi11 : iSCSI Initiator over TCP/IP
 Vendor: EQLOGIC   Model: 100E-00           Rev: 5.1
 Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdb: 209725440 512-byte hdwr sectors (107379 MB)
sdb: Write Protect is off
sdb: Mode Sense: 91 00 00 00
SCSI device sdb: drive cache: write through
SCSI device sdb: 209725440 512-byte hdwr sectors (107379 MB)
sdb: Write Protect is off
sdb: Mode Sense: 91 00 00 00
SCSI device sdb: drive cache: write through
 sdb: unknown partition table
sd 11:0:0:0: Attached scsi disk sdb
sd 11:0:0:0: Attached scsi generic sg3 type 0

iscsiadm -m node –target iqn.2001-05.com.equallogic:0-8a0906-53671c10b-e6900186a2d4ecd7-fsprod-lvm-2 –login

Logging in to [iface: default, target: iqn.2001-05.com.equallogic:0-8a0906-53671c10b-e6900186a2d4ecd7-fsprod-lvm-2, portal: 192.168.130.4,3260]
Login to [iface: default, target: iqn.2001-05.com.equallogic:0-8a0906-53671c10b-e6900186a2d4ecd7-fsprod-lvm-2, portal: 192.168.130.4,3260] successful.

Preparing the disk for LVM

Once a newly allocated disk presents from the SAN we need to partition it as an LVM disk:

  • Use fdisk to create a new primary partition that consumes the whole disk ( N -> P -> 1 enter enter)
  • Set the partition type to LVM (partition type 8e) on the new disk ( t -> 8e -> enter )
  • Save the partition using “W” to write, fdisk will exit and the you can go and create a Physical Volume (PV).

Create Physical Volumes

Example uses /dev/sdb1 and /dev/sdc1

 # pvcreate /dev/sdb1
 Physical volume "/dev/sdb1" successfully created

 # pvcreate /dev/sdc1
 Physical volume "/dev/sdc1" successfully created
  • Add the PV’s to an existing Volume Group
# vgextend vgSAN /dev/sdc1
 Volume group "vgSAN" successfully extended

Output from “vgs” command:

# vgs
 VG    #PV #LV #SN Attr   VSize   VFree
 vg0     1   4   0 wz--n-  19.97G  15.97G
 vgSAN   2   0   0 wz--n- 199.99G 199.99G

DONE!

You can now slice and dice logical volumes as needed from the available LVM storage.

Hint to create a 10G logical volume called “lvNewVolume” from the available storage in our “vgSAN” disk – use lvcreate -L+10G -n lvNewVolume /dev/vgSAN to stripe the volume over 2 PV’s use -i 2 in the above example.

 

 

Advertisements