Friday, January 4, 2013

Weird Network Problem at Home

Well, I got home from work the other night, and proceeded to fire up my PC to check my email.

Hmmm.....no network connectivity.

I powered up my Ham Radio PC to see if it had a network connection, and got the same type of error message.

I went and reset the Ethernet switch in my little "Network Closet", and things worked for about 2 minutes, and then the network connection dropped out.

About this time, my stepson arrived home, and said "Hey, could you check the Internet for me? I can't get on".

I told him I was in the process of doing that, and went and power cycled our Verizon router, and waited for it to come back up.

STILL no Internet connectivity, so then I tried to connect to my weather server, which is loacted in the living room, but on a different segment of our LAN.

Uh-Oh.....can't get to it, either. Hmmm....my little $20 "Dynex" Gig-E switch in the closet must have rolled over and gone casters up.

Since my Audio/Video PC is still set up on the dining room table, I powered it up, and was able to get on the Internet to see what Best Buy had in stock for a new Ethernet switch. I would up buying a Netgear 8-port switch like the one I have on my desk in the Radio Room to tie everything together with, and proceeded to swap out the "dead" Dynex switch.

Oh, boy......STILL no Internet access from the Radio Room.

Thinking perhaps the cable from the router to the switch had gone flakey, I then pulled the cat6 cable going to the router, and ran a cable directly from the router to the switch on my desk.

STILL no Internet! I went back to the living room, and tried connecting to the Home Theater receiver and/or the Oppo Blu-Ray player (they're both "Net Aware"), and discovered I couldn't reach either of them.

Hmmmm...something is definitely rotten in the network segment with the Home Theater, Radio Room, and stepson's bedroom, but a direct connection to the router works, as my wife's PC, and the Audio/Video editing PC have full connectivity.

Back to the network closet, where I unplug everything except the cable back to the router, and the cable to the Home Theater gear, which has it's own little D-Link 4-port 10/100 switch.

NOW I can get to the Onkyo receiver and Oppo Blu-ray player from either PC that has a direct connection into the router.

I reconnect the cable to the Radio Room segment and all works well.

I reconnect the cable to the back bedroom, and all works well.

I reconnect the cable to stepson's bedroom, and BLAM.....the whole network GOES DOWN.
And for the first time I noticed via the LED's on the network switches, that there's a tremendous amount of traffic flying around.

I pull the cable to stepson's bedroom, and all the traffic disappears.

HMMMMMM......WTF is going on in there? His PC and his X-Box are turned OFF, and I know I have the BIOS in his PC set so you can't remotely hit the PC with a Magic Packet to turn it on.

X-Box go berserk? Cable get crushed/shorted?

I go into his room, and he helps me pull his PC from under the desk, and I see the special Netgear "Home Theater and Gaming Series" Ethernet switch I bought for him some time ago. This switch has the traffic on certain ports prioritized, so that your X-Box or PS3 gets priority over any other device connected to it. It supposedly reduces the "Ping Times", something critical for networked games, but otherwise it's just an Ethernet switch.

It has five ports. One port goes to the Network Closet in the Radio Room, one port goes to his PC, and one port goes to his X-Box.

But all 5 ports have cables in them. It's then I notice that two of the ports are "bridged" with a yellow cat5 jumper cable.

I seem to remember having a yellow Cat5 cable in there so his girlfriend could use her laptop until we got all the wireless issues solved, but I thought I'd pulled it out after we got her laptop working on the wireless network segment.

I ask him why he has the cable connecting the two "unused" ports, and he says he doesn't know anything about it, but then his girlfriend chimes in and said she saw the free end of the yellow cable "just dangling there" and decided it had to go *somewhere*, and since it looked like it would fit into the unused port, she plugged it in!

Now, I don't remember specifically why you never connect two ports togther, as it's lost among all the dust of my brain somewhere under "Networking 101", but I know it's a Bad Thing to do so.




Ahh....found it. It's called a "Switching Loop" and causes the switch to broadcast out of *every* port it has,  causing a "Broadcast Storm", and will bring even a well-managed network to it's knees.


As soon as I unplugged one end of the mysterious yellow cable, "traffic" on the LAN went to about zero, and all connectivity was restored.

In retrospect, I should have fired up Wireshark (formerly Ethereal) on my Linux box and observed what type of traffic was bringing the network down, but by the time I "fixed" the problem, and restored everything to normal, it was 2130, and my bed time!


9 comments:

  1. LOL - "just dangling there" and decided it had to go *somewhere* - :D

    Over the years working in aerospace, that sort of problem pops up quite often, I'd say.

    ReplyDelete
  2. Good detective work on your end, though. Glad you got it all working again.

    ReplyDelete
  3. @Minstrel - Yeah, I should have tagged it "DO NOT CONNECT!" or something similar. Or, I could have just removed it, but it's one of those things that you never expect to come back and bite you!

    @Rev - That's what I get the big ( ? ) bucks at work! Besides all the radio ("RF") stuff I do, I help out the network and IT guys when they need it.

    Having more than one 'specialty' comes in handy at times.

    ReplyDelete
  4. OH man... Lucky you 'know' troubleshooting, otherwise that could have been expensive!!!

    ReplyDelete
  5. Yep, something I learned at an early age.
    I always heard it called either the "Half-Split Method", or the "Divide and Conqueror" method.

    For example, if it turns on, and lights up, it's probably NOT the power cord!

    ReplyDelete
  6. Glad you found it! It helps make up for the lost time somewhat. =)

    ReplyDelete
  7. Further questioning of the female suspect revealed that she was "cleaning things up a bit" Monday morning before she left for her job, and that's when it happened.

    Everything had been running just fine Sunday night when I shut down for the evening, and it was a bit of a surprise Monday afternoon when I had ZERO connectivity.

    My stepson further indicated he wasn't able to get on the Internet Monday when he came home for lunch.

    ReplyDelete
  8. What Old NFO said - good thing you're a problem finder rather than just a chaser!

    ReplyDelete
  9. And I wound up with a "spare" 8-port Netgear GigE switch!

    ReplyDelete

Keep it civil, please....