Pre-Troubleshooting Is Key To InteropNet's Success - InformationWeek
IoT
IoT
Interop
News
4/9/2015
12:00 AM
Tim Titus
Tim Titus
Event Updates
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Pre-Troubleshooting Is Key To InteropNet's Success

InteropNet can't fail during the Interop conference. The work we did at Hot Stage will help make sure the network is fully operational.

Working on InteropNet takes root-cause troubleshooting to a new level because no one on the NOC team wants hear "Hey, what happened to the network?" once Interop begins.

During the Hot Stage phase of InteropNet, our goal was to make sure the network was always operational and available. And, instead of troubleshooting existing network issues, we did something I like to call pre-troubleshooting; that is, actively preventing problems from existing once Interop opens.

To do this, we used PathSolutions TotalView, our root-cause troubleshooting product. During the network build out, it took about twelve minutes to fully deploy and get all of the equipment added to the configuration.

Keep in mind that the equipment represents a multi-vendor network that has to be assembled, configured, and deployed in Las Vegas in less than four days—not the norm for most company networks, but something the NOC team prides itself on accomplishing every year.

One of the first things I did is look at the Issues tab, which showed all immediately discovered problems in the network.

This allowed the pre-troubleshooting activities to start so we could work with the other NOC team members to remove the bugs before they became weird mysteries that required serious digging through network device configurations.

Switches and routers have thousands of counters that track all sorts of performance, error, and configuration information. TotalView discovers and analyzes all this information. We listen to what the equipment is trying to tell us through its error counters, and produce plain-English answers to the problem.

For example, during Hot Stage a high-throughput test wasn't showing as much data as should be passing from one point to another.

TotalView's Path mapper identified the layer-1, layer-2, and layer-3 path through the network the load test was using, and then spotted one interface that was discarding a ton of packets due to jumbo frame misconfiguration.

With that interface reconfigured to permit the jumbo frames, the test produced expected results.

Now that we're done at Hot Stage and on our way to Las Vegas, our goal is to have a rock-solid network capable of handling anything Interop attendees and exhibitors throw at it.

"As rapidly as the InteropNet is deployed, you can't just plug things in and hope and pray that they work," said Glenn Evans, Lead Network Engineer of InteropNet.

"You have to be on top of what the equipment is complaining about and rapidly address the problems or you’ll be stuck in constant firefighting mode."

For InteropNet, pre-troubleshooting problems is key to staying on top of any possible issues just as for company networks, root-cause troubleshooting is key to rapidly resolving network issues. To see TotalView in action, drop by the NOC or the PathSolutions booth #1651.

Interop Las Vegas is April 27 through May 1. Register now to learn the latest technology developments, accelerate your career, and network with your peers in the IT community.

Tim Titus, CTO of PathSolutions, is a network geek and proud of it. Before starting his own network troubleshooting company, he spent more than 25 years working for small and large companies in a variety of roles: from network engineer, to network manager, to network ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
2018 State of the Cloud
2018 State of the Cloud
Cloud adoption is growing, but how are organizations taking advantage of it? Interop ITX and InformationWeek surveyed technology decision-makers to find out, read this report to discover what they had to say!
Commentary
5 Machine Learning Resolutions for 2019
Lisa Morgan, Freelance Writer,  12/13/2018
News
What Tech Mega Hubs Will Do to East Coast Cities
Kayla Matthews, Technology Writer,  12/11/2018
News
Daimler Financial Services CIO Says: Don't Get Comfortable
Jessica Davis, Senior Editor, Enterprise Apps,  12/5/2018
Register for InformationWeek Newsletters
Video
Current Issue
Enterprise Software Options: Legacy vs. Cloud
InformationWeek's December Trend Report helps IT leaders rethink their enterprise software systems and consider whether cloud-based options like SaaS may better serve their needs.
White Papers
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Sponsored Video
Flash Poll