ESXCLI-* Storage commands every vSphere Admin needs to know

 

 

esxcli_storage_core_device_list

There are numerous events where the data or information showed in vSphere Client is not good enough for vSphere Admin to troubleshoot or root cause the issue particularly about Storage device, Multipathing information about LUN, Queue Depth etc…

In this post, I am going to explain one of the most frequently used storage commands help vmware admin to retrieve information about LUN/Disk and troubleshooting storage issues.

 

HOW TO RETRIEVE DISK/LUN INFORMATION CONNECTED TO ESXi HOST

 

esxcli storage core device list 

This command will list all SCSI devices connected to ESXi Host

~ # esxcli storage core device list

t10.F405E46494C454255443C6274495D247A78736D223956493

Display Name: OPNFILER iSCSI Disk (t10.F405E46494C454255443C6274495D247A78736D223956493)

Has Settable Display Name: true

Size: 1024

Device Type: Direct-Access

Multipath Plugin: NMP

Devfs Path: /vmfs/devices/disks/t10.F405E46494C454255443C6274495D247A78736D223956493

Vendor: OPNFILER

Model: VIRTUAL-DISK

Revision: 0

SCSI Level: 4

Is Pseudo: false

Status: degraded

Is RDM Capable: true

Is Local: false

Is Removable: false

Is SSD: false

Is Offline: false

Is Perennially Reserved: false

Queue Full Sample Size: 0

Queue Full Threshold: 0

Thin Provisioning Status: unknown

Attached Filters:

VAAI Status: unknown

Other UIDs: vml.010000000045346c7244592d747a78632d32594639564952545541

Is Local SAS Device: false

Is Boot USB Device: false

No of outstanding IOs with competing worlds: 32

 

Let’s understand the output of this command:

 

Display Name: This field displays useful information about:

  • Vendor Name: OpenFiler
  • Type of Disk: iSCSI Disk
  • LUN ID: Naa Unique identifier

Size: This field shows Size of SCSI Device

Multipathing Plugin : This Field shows information about Multipathing plugin used to discover this SCSI device.

Vendor : This field displays vendor name of SCSI Device manufacturer like EMC, IBM, DELL

Model : This Field displays Model information about LUN or Disk

Status: It shows the current status of SCSI Device. For ex

  • Not Connected: means LUN or Disk is not accessible
  • Degraded: Means loss of redundant path
  • Active: LUN or Harddisk is accessible

Is RDM Capable: This field display whether SCSI Device is RDM or Local Disk. It will be set to True if SCSI device is LUN or set to False if SCSI Device is Harddisk.

Thin Provisioning Status: This field shows whether LUN is configured with Thin provisioning or not.

VAAI Status: This field displays VAAI capability of Array.

Other UIDs : This field shows unique VML signature generated by VMware for each SCSI Device mapped to ESXi Host.

 

Explanation of Command Output:

iSCSI disk of 1GB size from Openfiler vendor has been mapped to ESXi Host. ESXi is using Native Multipating Plugin to manage this device. This Device is RDM device with no redundant paths configured. Unknown status of Thin Provisioning and VAAI primitives explains that Openfiler doesn’t support these features.

 

HOW TO FIND OUT QUEUE DEPTH

 

LUN specific information can be retrieved using -d <naa.id> option

esxcli storage core device list  -d <naa.id>

esxcli storage core device list -d t10.F405E46494C454255443C6274495D247A78736D223956493

Display Name: OPNFILER iSCSI Disk (t10.F405E46494C454255443C6274495D247A78736D223956493)

Has Settable Display Name: true

Size: 1024

Device Type: Direct-Access

Multipath Plugin: NMP

Devfs Path: /vmfs/devices/disks/t10.F405E46494C454255443C6274495D247A78736D223956493

Vendor: OPNFILER

Model: VIRTUAL-DISK

Revision: 0

SCSI Level: 4

Is Pseudo: false

Status: degraded

Is RDM Capable: true

Is Local: false

Is Removable: false

Is SSD: false

Is Offline: false

Is Perennially Reserved: false

Queue Full Sample Size: 0

Queue Full Threshold: 0

Thin Provisioning Status: unknown

Attached Filters:

VAAI Status: unknown

Other UIDs: vml.010000000045346c7244592d747a78632d32594639564952545541

Is Local SAS Device: false

Is Boot USB Device: false

No of outstanding IOs with competing worlds: 32

 

 

Is Perennially Reserved: This field displays whether perennially reservation flag is set on LUN or not

 

WHAT IS PERENNIALLY RESERVED

This Flag was introduced in ESXi5.0 & 5.1 to make LUN discovery process more efficient and faster in MSCS clustering environment.

In earlier days, when MSCS virtual machines deployed across multiple ESXi Hosts & VMs are sharing disk over RDM LUNs. If one of the ESXi Host from that cluster gets rebooted, it used to take longer time for ESXi Host to come up depending on no of RDMs mapped to Host. Since ESXi has to query each RDM LUN for storage discovery which slows down Boot Process of ESXi host.

With this flag enabled, we inform SCSI mid layer of VMkernel to not to query this device during Boot Discovery process. In Simple words, this flag tell ESXi hosts to not to waste any time in trying to query these devices on boot because they might be reserved by another ESXi host.

Check this VMware KB article 1016106 for detailed info on Perennially Reserved

 

HOW TO ENABLE PERENNIALLY RESERVED FLAG

 

esxcli storage core device setconfig -d <naa.id> –perennially-reserved=true

Queue Full Sample Size: This parameter related to Adaptive queuing. If its set to 0 means Adaptive queuing is disabled

Queue Full Threshold: This parameter is also related to Adaptive Queuing. If its set to 0 means Adaptive Queue is disabled.

WHAT IS QUEUING

Queueing defines how many Outstanding IOs can be send to disk. Queuing provides concurrency and improvise performance by allowing multiple IO request to be active on a LUN at same time.

 

WHAT IS ADAPTIVE QUEUING

Adaptive queueing mechanism adjusts the LUN queue depth in the VMkernel IO stack. Whenever Storage detects IO congestion on IO path, it sends SCSI Sense Code of BUSY or QUEUE FULL. When VMkernel receives BUSY or QUEUE FULL SCSI Codes, it throttles LUN queue depth depending upon Queue Full Sample & Full Threshold parameter configured.

 

No of Outstanding IOs with competing worlds: This fields actually displays LUN queue depth. In Our example, 32 outstanding IO can be active on LUN at same time.

Check this VMware KB article 1008113 for detailed info on LUN Queue Depth Throttling

 

In my Next Blog, I am going discuss another handy & very useful command to get Multipathing Information of SCSI device mapped to ESXi Host.

I hope you guys might have found this article useful and informative. 

If you like this article then please share your feedback in comments section.

Keep Sharing Keep Learning!!!

You May Also Interested In

govmlab on sabtwittergovmlab on sablinkedingovmlab on sabgooglegovmlab on sabfacebookgovmlab on sabemail
I am VMware Solution Architect with 10+ Years of enriching experience in Datacenter Virtualization Technologies, Storage Area Networks and Software Defined Datacenter, Networking and Storage.
I hold Numerous certification including RHCE, CCNA, VCP4.0, VCP5.1, VCP5.5, vCloud and EMC certification.
While spending countless hours exploring the product inside and out and learning everything about it, Eventually I discovered my passion for teaching and helping others learn from my knowledge and experience so turned to Trainer cum Blogger to educate every single person keen to learn Virtualization.

Leave a Reply