Weekend Learning: Defining the TCP Socket (KGAS) wait event

I have talked about this wait event before as a potential warning sign of network issues, as well as an indicator when you have database traffic misbehaving, but what does Oracle say about the TCP Socket (KGAS) wait event?  I am SO glad you asked!  Wait no more, check out MOS article "What are 'TCP Socket (Kgas)' Waits?" from document ID 416451.1 for the official word on why this wait event is not a cause to open an SR with Support, and what issues it could be representing!

Weekend Learning: InfiniBand interfaces

Are you running InfiniBand devices with your Oracle infrastructure?  Do you want to know a bit more about it, and problems that can arise?  Of course you do!  Look no further than My Oracle Support note "InfiniBand IPoIB Interfaces Down and IBSTAT Reports HCA Ports in INITIALIZING State" via document ID 1950483.1 which sounds a LOT like an issue we had with InfiniBand interfaces a few months ago, so I wonder if this is based off of our case!  Have you ever used the ibstat command?  Do you monitor /var/log/messages for potential issues?

Network monitoring tool: iperf

Looking to add to your toolbox of commands and measurements for your network?  Then this article is for you!  Learn how to use this open source command to measure either client or server connectivity but also look at the comments section, as at least one person seems to have found a strange behavior which you may be able to replicate as well.

ALERT: Ethernet memory leak may cause network outage

Following on the heels of the ASM alert the other day, today I share with you MOS note ID 1622661.1 that outlines what firmware versions of the following devices are affected by a serious bug:

• Sun Blade 6000 24-Port 10-Gigabit Ethernet Switched Network Express Module (NEM) (2073A-N)
• Sun Network 10GE Switch 72p (2074A-F)

What's the issue here you ask?  Well, it seems that memory leaks each time an admin logs in and out of the switch on these platforms if they are NOT configured for HA (High Availability).  This causes an eventual critical issue where the switch will reboot causing network outages at durations which may not be regular and make it very hard to pinpoint the cause of the issue.

Network canary in the coal shaft

Following part 1 and part 2 of getting to the bottom of wait events in your system, when we had a "network event" at work the other day we could see the after effect of this disruption on our system by the wait event "TCP Socket (KGAS)" being on the top of our alert.  Seems like a "no duh?" moment right?  Yet it got me to thinking, that this wait event is really the canary in the coal shaft for your network.

What if we modified our previous alert to run every minute, looking only for spikes in this wait event? Wouldn't we get proactive notifications for incidents like this, or be able to easier identify "network spikes" or other disruptions in communication between our systems themselves or the systems and other parts of the infrastructure?  I think that just might be the case, so I'm going to create a simple alert and mock it up so the next time we have any network disruption we can start fine tuning the alert to get at what are the really important measurements needed in a scenario like this.