Weekend Reading: Concurrent Manager

Maris Elsins posted several great posts many months ago on the internals of the Concurrent Manager processing, some performance settings available, and the dangers of CMCLEAN being run.  Why have I not posted these sooner you ask?  Well, I'm anxiously waiting for the next entries in this series but I found myself reading these again so I decided there was no better time to share them with you!  Very easy to read, but explaining some advanced mechanisms with the Concurrent and some great visual aids as well.

CMCLEAN: Zombie Hunter?

The other day I was tracking down a mystery (more on that later) when one of our Exadata database nodes hard crashed and we had to pick up the pieces.  While I was tracking down that mystery, I happened to be on our concurrent manager nodes and saw that neither node had more than 3 processes at the OS layer marked as zombie processes.  What do you think I found after the crash?

We now had 38 zombie processes "alive" on one of the nodes!  As we were cleaning up the system, I kept asking "what about the zombie processes?" to which the DBA told me "don't worry, CMCLEAN will take care of it" but he didn't exactly tell me how or if he really knew for a fact that CMCLEAN would take care of the zombies.  After we took down the concurrent manager nodes and ran CMCLEAN, guess what: all the zombie processes were gone.

So this leaves me to wonder if they disappeared when the nodes were taken down, or if CMCLEAN actually did the job.  Next time, I'll have to watch a bit closer to find out for sure but if you know the answer share in the comments!

EDIT:  For some reason the Blogger iOS app doesn't include changing the "Published On" date, and Safari on my phone wouldn't load the site right/completely, so this showed up yesterday on the blog as created on 9/12/13 instead of when I posted it 9/18/13.  Sorry for burying it on the site!