iSCSI and LVM

iSCSI – Brainstorm

iSCSI is a protocol that allows you to use SCSI commands over an IP network. You can use the storage over the network using iSCSI.

SCSI uses a client-server architecture. A “client” (ie: your system) is an initiator, it initiates requests. A “server” (ie: your storage device) is a target, it has something you want and answers requests from the initiator(s).

iSCSI Basics

All devices in an iSCSI enviroment will have addresses. Initiators will have addresses, and targets will have addresses. When you define a target you can specify the address yourself.

iSCSI uses the following form for addresses

iqn.2009-05.com.linuxtrove.avstorage:storage03

iqn (iSCSI Qualified Name)

The date field is the date of the first full month

The Naming Auth is the naming authority (domain name) for this target, reversed.

Following the naming authority is a colon, after which you can put anything you want to help you better remember.

I have used this one: iqn.2009-05.com.linuxtrove.avstorage:storage03

Now we will setup the target first then configure the initiator

<!– @page { margin: 2cm } P { margin-bottom: 0.21cm } H2 { margin-bottom: 0.21cm } H2.cjk { font-family: “DejaVu Sans” } H2.ctl { font-family: “DejaVu Sans” } –>

Setting up a Target

I have follow the following steps:

  1. Install the new machine with Ubuntu-8.4 OS, i have 1TB storage capacity on this machine.

We made the LVM partion because we are gonna to add it up in existing LVM storage.

IP Address for the machine is 192.168.200.23

Install the iscsitarget package using the following command.

apt-get install iscsitarget

3. Edit /etc/ietd.conf file

ietd.conf file contains the information which partition of the system is gonna to export using iSCSI

I have made the entry like this:

Target iqn.2009-05.com.linuxtrove.avstorage:storage03

Lun 1 Path=/dev/sda3,Type=blockio
Alias storage03


Second line Contains the path of the partiton you are going to use for storage
and the type of the device, it may be a file, but in our case it is a block device.

Third line contains the Alias, that is optiona
Save the file then restart the iscsi service using the following command.

#/etc/init.d/iscsitarget restart

NOTE: For more information please read the man pages or do googling.

Setting up an Initiator

  1. Install the open-iscsi package using the following command.

#apt-get install open-iscsi

In my case i have initiator already setup so i skipped the step 1.

  1. To list the targets, do the following

iscsiadm --mode discovery --type sendtargets --portal 192.168.200.23

Once the targets are listed above, they are automatically added into 
	 /etc/iscsi/nodes
  1. You can change the settings in the nodes directory and make it always automatic login on boot. The directories and files to be considered are

  • /etc/iscsi/iscsid.conf <- Make Node as automatic rather than manual
  • /etc/iscsi/nodes/yournodename <- Change the configuration and make it automatic e.g. node.startup=automatic
  1. Start into a node using the following and then monitor /var/log/messages for the block device name

iscsiadm –mode node –targetname iqn. 2009-05.com.vccorp.avstorage:storage03 –portal 192.168.200.23 –login

After that just check the output of fdisk command. New disk will be there as i got.

# fdisk -l

Disk /dev/sdd: 80.0 GB, 80026361856 bytes

255 heads, 63 sectors/track, 9729 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdd1 1 2550 20482843+ 8e Linux LVM

/dev/sdd2 * 2551 9430 55263600 83 Linux

/dev/sdd3 9431 9729 2401717+ 5 Extended

/dev/sdd5 9431 9729 2401686 82 Linux swap / Solaris

Disk /dev/sde: 500.1 GB, 500105217024 byte

255 heads, 63 sectors/track, 60800 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sde1 1 60800 488375968+ 8e Linux LVM

Disk /dev/sdf: 485.5 GB, 485595855360 bytes

255 heads, 63 sectors/track, 59037 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdf1 1 59037 474214671 8e Linux LVM

Disk /dev/sdg: 986.0 GB, 986005440000 bytes

255 heads, 63 sectors/track, 119875 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdg1 1 119875 962895906 8e Linux LVM

Above is just the sample output.

/dev/sdg1 is the newely added partiton by iSCSI.

Now we will add this new storage to our existing LVM.

# pvcreate /dev/sdg1

#pvscan

root@Orion:~# pvscan

PV /dev/sda1 VG recordings lvm2 [74.52 GB / 0 free]

PV /dev/sdb1 VG recordings lvm2 [74.52 GB / 0 free]

PV /dev/sdd1 VG recordings lvm2 [19.53 GB / 0 free]

PV /dev/sdc1 VG recordings lvm2 [335.34 GB / 0 free]

PV /dev/sde1 VG recordings lvm2 [465.75 GB / 0 free]

PV /dev/sdf1 VG recordings lvm2 [452.23 GB / 0 free]

Total: 6 [1.29 TB] / in use: 6 [1.29 TB] / in no VG: 0 [0 ]

Its not showing newely created physical volume.

Physical Volume will not be recognised untill you add it up in any volume group.

#vgextend recordings /dev/sdg1

# pvscan

root@Orion:~# pvscan

PV /dev/sda1 VG recordings lvm2 [74.52 GB / 0 free]

PV /dev/sdb1 VG recordings lvm2 [74.52 GB / 0 free]

PV /dev/sdd1 VG recordings lvm2 [19.53 GB / 0 free]

PV /dev/sdc1 VG recordings lvm2 [335.34 GB / 0 free]

PV /dev/sde1 VG recordings lvm2 [465.75 GB / 0 free]

PV /dev/sdf1 VG recordings lvm2 [452.23 GB / 0 free]

PV /dev/sdg1 VG recordings lvm2 [918.28 GB / 0 free]

Total: 7 [2.29 TB] / in use: 7 [2.29 TB] / in no VG: 0 [0 ]

Now its showing the new PV also.

root@Orion:~# vgdisplay

— Volume group —

VG Name recordings

System ID

Format lvm2

Metadata Areas 7

Metadata Sequence No 22

VG Access read/write

VG Status resizable

MAX LV                          0

Cur LV                              1

Open LV                            1

Max PV                              0

Cur PV                               7

Act PV                              7

VG Size                            2.29 TB

PE Size                            16.00 MB

Total PE                         91001

Alloc PE / Size              91001 / 2.29 TB

Free PE / Size                58770 / 986GB

VG UUID                       wz8pMV-RE63-Flwo-jpUI-G3t7-GCCz-uEAQDT

Now check the Logical volume space using the follwing command.

root@Orion:~# lvdisplay

— Logical volume —

LV Name /dev/recordings/disk1

VG Name recordings

LV UUID                             qy4ywC-oRqH-ccc5-PY47-NkCL-9Fcw-SfLpD1

LV Write Access                read/write

LV Status                              available

# open 1

LV Size                                 1.29 TB

CurrentLE                            149771

Segments                                7

Allocation                          inherit

Read ahead sectors                  0

Block device                          254:0

Now we will extend the size of Logical Volume by using the following command.

# lvextend -l +58770 /dev/recordings/disk1 /dev/sdg1

In the output of vgdisplay you can can the free extents and just add them like the above command.

Now please check the output of “df -h”.

root@Orion:~# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/sdd2 52G 16G 35G 31% /

varrun 502M 144K 502M 1% /var/run

varlock 502M 0 502M 0% /var/lock

procbususb 502M 444K 502M 1% /proc/bus/usb

udev 502M 444K 502M 1% /dev

devshm 502M 0 502M 0% /dev/shm

lrm 502M 33M 469M 7% /lib/modules/2.6.20-16-generic/volatile

/dev/mapper/recordings-disk1   1.4T 1.3T 222G 99% /recordings

Size of the logical volume still not increased. To make it visible here and usable, we have two ways.

  1. Reboot the system
  2. Online resizing

In second option we dont require reboot. So i prefer the second one. Use the following command to do that.

# resize2fs /dev/recordings/disk1

It will take some time to resize. Please have patience and wait for atleast 1 hour. If you want to check it that size is increasing or not you can check it by executing the df command without any flag.

  • martino62

    Can i use iscsitarget without LVM ?
    Thank You, Best Regards

I'm happy to use Increase Sociability.