Looking at Exadata Cell Disks

An Exadata X2-2 storage cell contains 12 disks. While you should not need to look at them via iostat to measure performance, as metrics are available via dbconsole, enterprise manager and asm disk views, it can sometimes be useful to be able to take a quick look at the system.

Ways to map your Exadata devices to /dev/sd*

lsscsi

[root@ed2hcell01 ~]# lsscsi
[0:0:20:0]   enclosu SUN      HYDE12           0341  -
[0:2:0:0]    disk    LSI      MR9261-8i        2.90  /dev/sda
[0:2:1:0]    disk    LSI      MR9261-8i        2.90  /dev/sdb
[0:2:2:0]    disk    LSI      MR9261-8i        2.90  /dev/sdc
[0:2:3:0]    disk    LSI      MR9261-8i        2.90  /dev/sdd
[0:2:4:0]    disk    LSI      MR9261-8i        2.90  /dev/sde
[0:2:5:0]    disk    LSI      MR9261-8i        2.90  /dev/sdf
[0:2:6:0]    disk    LSI      MR9261-8i        2.90  /dev/sdg
[0:2:7:0]    disk    LSI      MR9261-8i        2.90  /dev/sdh
[0:2:8:0]    disk    LSI      MR9261-8i        2.90  /dev/sdi
[0:2:9:0]    disk    LSI      MR9261-8i        2.90  /dev/sdj
[0:2:10:0]   disk    LSI      MR9261-8i        2.90  /dev/sdk
[0:2:11:0]   disk    LSI      MR9261-8i        2.90  /dev/sdl
[1:0:0:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdm
[1:0:1:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdn
[1:0:2:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdo
[1:0:3:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdp
[2:0:0:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdq
[2:0:1:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdr
[2:0:2:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sds
[2:0:3:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdt
[3:0:0:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdu
[3:0:1:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdv
[3:0:2:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdw
[3:0:3:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdx
[4:0:0:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdy
[4:0:1:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdz
[4:0:2:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdaa
[4:0:3:0]    disk    ATA      MARVELL SD88SA02 D20Y  /dev/sdab

The LSI devices (hightlighted in red) are the physical disks, the MARVELL (in blue) are the Flash devices.

cellcli

The ‘list celldisk detail’ command gives information about the celldisks. Note, the command output is very long, so pipe it to a file!

[root@ed2hcell01 ~]# cellcli -e list celldisk detail
name:                   CD_00_ed2hcell01
 comment:
 creationTime:           2011-02-15T12:22:02+00:00
 deviceName:             /dev/sda
 devicePartition:        /dev/sda3
 diskType:               HardDisk
 errorCount:             0
 freeSpace:              0
 id:                     8f63def6-0ca7-448a-a3bd-76e5572af5f5
 interleaving:           none
 lun:                    0_0
 raidLevel:              0
 size:                   528.734375G
 status:                 normal

 name:                   CD_01_ed2hcell01
 comment:
 creationTime:           2011-02-15T12:22:06+00:00
 deviceName:             /dev/sdb
 devicePartition:        /dev/sdb3
 diskType:               HardDisk
 errorCount:             0
 freeSpace:              0
 id:                     41f37430-2beb-4062-9a03-ae801f411520
 interleaving:           none
 lun:                    0_1
 raidLevel:              0
 size:                   528.734375G
 status:                 normal

 name:                   CD_02_ed2hcell01
 comment:
 creationTime:           2011-02-15T12:22:06+00:00
 deviceName:             /dev/sdc
 devicePartition:        /dev/sdc
 diskType:               HardDisk
 errorCount:             0
 freeSpace:              0
 id:                     a9fc2181-10fc-498e-a902-9bb5cb47441c
 interleaving:           none
 lun:                    0_2
 raidLevel:              0
 size:                   557.859375G
 status:                 normal

<snip>

Note the lines highlighted in red:- on the first 2 disks the celldisks sit in the partition /dev/sd*3 to allow space for the software and operating system. On the remaining disks the cells have access to the whole disks.

This means that in iostat you cannot separate out the load for different ASM diskgroups whose griddisks share the same celldisk. You can see which celldisks are used by your griddisk by using

[root@ed2hcell01 ~]#cellcli -e list griddisk detail

Again the output to this command can be long, and it is better piped to a file.

name:                   DATA_DM01_CD_00_ed2hcell01
 availableTo:
 cellDisk:               CD_00_ed2hcell01
 comment:
 creationTime:           2011-02-15T12:26:29+00:00
 diskType:               HardDisk
 errorCount:             0
 id:                     a8c155f9-64c4-4604-adfd-3771ac93c35a
 offset:                 32M
 size:                   392G
 status:                 active
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s