Pooladm can hang you up!

In my last post I wrote about using pooladm to change the number of CPUs assigned to a zone. I’ve been using this command a lot more recently and I’ve hit upon some quirky behaviour when the contents of /etc/pooladm.conf is inconsistent with the reality on your system. So, I have a T5-8 LDOM. This originally had 32 cores (256 threads) of T5-8 CPU and a zone using a pool of 240 vcpus. I used supported tools to increase the number of cores in my LDOM to 60 (480 threads) and rebooted.  After reboot I try to increase the pset size for my zone..

# poolcfg -c 'modify pset pset_sc5acn01-d8.blah.com_id_6138 ( uint pset.min=448 ; uint pset.max=448 )' /etc/pooladm.conf
# pooladm -c

And then I wait, and I wait and I wait… and still nothing happens. The OS is up and running ok, I can do other jobs but my pooladm isn’t returning. When I truss it, it seems to just sat there doing nothing..

41937: brk(0x0035FA30) = 0x00000000
41937: brk(0x0035FA30) = 0x00000000
41937: brk(0x0035FA30) = 0x00000000
41937: brk(0x00361A30) = 0x00000000
41937: brk(0x00361A30) = 0x00000000
41937: brk(0x00361A30) = 0x00000000
41937: brk(0x00363A30) = 0x00000000

I gave up after 5 minutes of waiting.. I tried rebooting the LDOM again, and after reboot the zone hasn’t automatically restarted as the system pools service is stuck at transitioning to online

offline* 9:59:15 svc:/system/pools:default

After a bit of experimentation, I discover that I can increase the number of cores up to 240, but if I make the jump to 296 I get the hang. Then I look at my pooladm.conf

<property name="pool.sys_id" type="int">1</property>
<property name="pool.scheduler" type="string">TS</property>
</pool>
<pool name="pool_default" active="true" default="true" importance="1" comment="" res="pset_-1" ref_id="pool_0">
<property name="pool.sys_id" type="int">0</property>
<property name="pool.scheduler" type="string">TS</property>
</pool>
<res_comp type="pset" sys_id="1" name="pset_sc5acn01-d8.osc.uk.oracle.com_id_6138" default="false" min="448" max="448" units="population" comment="" ref_id="pset_1">

You can see here that my system  default pset size is still shown as 256 despite the changes in the underlying LDOM. If I do

# pooladm -s

To save the current running config to /etc/pooladm.conf this value increases to match the new size of the LDOM. Now I can manipulate the size of the zone’s pool and enable it successfully using pooladm -c So – if you need to change the underlying number of CPUs in your LDOM then you must take care! – If increasing, using pooladm -s to refresh the pooladm.conf with the new total number of cpus – If decreasing  (I’m only guessing here how it will behave here) reduce the zone’s pool size first.

UPDATE: There is also some information in the MOS note SuperCluster – Avoiding system/pools maintenance state when modifying zone pool CPU count after using setcoremem (Doc ID 1991360.1)

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s