Friday 2 September 2011

Identifying and Troubleshooting Congestion in Storage Area Network

Congestion of Fibre channel links will eventualy lead to performance issue and severe production impact. This occurs, when more frames are being transmitted over a FC link, also special care needs to taken while choosing number of ISL to forward the FC traffic or else this will lead to congestion of ISL links.
Lets take an example of FCIP links, if we see lot of retransmits this indicates that there is a congestion on IP links, suggestion to slow down the data being put on the IP link, or increase the commited bandwidth for FCIP link having issues. Other things to check would be window size, MTU size and compression.
ISL oversubscription is another issue which leads to congestion in SAN. Some of the ways to mitigate this issues are
  1.  Increase the number of ISLs or localize traffic to decrease ISL use.
  2. Increase the number of connections from the device to the fabric to spread the traffic across multiple ports
  3. Spread high usage devices across different VCs
How to monitor congestion.
  1.  Monitor the performance from host application perspective. There are various tools available to calculate the IOPS, there should be a significant drop during congestion.
  2. Need to check for ISL utilization and buffer starvation.
  3. Look for class 3 frame drops on F port from initiator
Listed below are the commands in Brocade to monitor congestion

portstatsshow [<SlotNumber>/]<PortNumber> - Monitor buffer credit and class 3 frame discards
portbuffershow - Monitor buffer credit allocation
porterrshow - check for class 3 discards

Tuesday 21 June 2011

Storage Virtualization Benefits

Storage virtualization is an emerging technology that creates logical abstractions of physical storage systems. With the help of Storage virtualization, the storage disks from multiple vendors can be pooled into a single device and thus enabling seamless data migration and storage tiering across heterogeneous storages.


 In the past I have seen very interesting use cases integrating server virtualization and storage virtualization. Listed below are some of the use cases


1) Seamless migration and tiering: 
     
    With the help of logical abstraction provided by the Storage virtualization appliance, host would be masked with a virtual volume and is completely unware of the underlying backend storage. Now with the help of native migration tool from the storage virtualization vendors underlying backend volumes that the virtual volume is made up of can be changed so that the host still has the access to virtual volume when the migration is being done. This helps in storage level tiering and migration across or within storage subsystem without a down time.


2) Vmotion across Datacenters:



What is vmotion ? its the capability to move a running VM from one ESX host to another, this is normaly done to offload the ESX server or to shutdown the host for maintenance. One of the requirement for this is a shared storage between the ESX hosts.
Now with the help of distributed device from storage virtualization solution from EMC which has legs spread across disparate datacenters, Vmotion can be done across Data centers (<100km | <5ms latency).  


What is distributed device ? Distributed devices are mirrored devices that spread across two VPLEX clusters connected together into a metro plex. Distributed devices are created on top of two devices, the devices should be from different VPLEX clusters. The geometry of distributed devices by default is Raid-1.