Although precious few companies are in a position to deploy and manage their own business networks, educational and governmental research institutions have bandwidth and performance requirements that go beyond what commercial network providers can supply at reasonable prices. The retail cost for enough bandwidth to handle the data transfers needed for research and experimentation would bankrupt most institutions.
To avoid retail costs, universities and nonprofits have formed consortia to purchase access to dark fiber--fiber that's in place but not being used--and build high-speed metropolitan, regional, and national optical networks. In addition to 10-Gbps IP network connections, these fiber paths can carry non-IP traffic, a capability few conventional Internet service providers offer. Even better for researchers, the collapse of the telecom market meant much of this fiber could be purchased at deeply discounted rates.
UW-Madison gives data link the old college try.
Photo by Morry Gash/AP
At the University of Wisconsin-Madison, we've lit the fiber section from our city to a major network peering point in Chicago with a DWDM optical link that carries our Internet2 traffic over one of these 10-Gbps connections. Internet2, an alternative to the commodity Internet, links research and instructional networks at higher speeds and is used for exploring and testing next-generation applications that would require this connectivity.
Following the collaborative investment model of LambdaRail, regional and metropolitan partnerships have sprung up to fill out connectivity needs in localized markets. In our neck of the woods, the Northern Tier Network Consortium aims to extend the fiber infrastructure across the northern states, from Wisconsin to Washington and all points in between, offering extraordinary network connectivity to educational and governmental institutions, communities, and nonprofits along that route.
Other examples are the Pacific Northwest Gigapop and the Boreas-Net, a regional consortium acquiring private fiber links between various connecting points in Wisconsin, Minnesota, Iowa, Missouri, and Illinois. These setups give university partners high-speed connections between campuses, as well as backup routes to Internet2 and other network providers.
The University of Wisconsin system has acquired fiber from the Starlight PoP in Chicago, through Wisconsin along Interstate 94 to the border of Minnesota. That fiber also will be made available to the Northern Tier Network Consortium and Boreas-Net.
Why Not Business?
Companies could purchase regional or national unused fiber to connect to a geographically distant backup data center. With some good partnerships, multiple companies or institutions could acquire fiber together and share the resource using DWDM equipment. This connectivity also could be leveraged to connect business partners that interchange large data sets, such as customer relationship data.
Just 10 years ago, the University of Wisconsin at Madison was connected to the Internet via WiscNet--a public-service ISP and partner of UW-Madison--using a half-dozen T1 pipes. Although the traffic included research and educational data, it was only marginally distinguishable from general traffic in network load. But by 2000, our connection to Chicago had grown to a full OC-12 ATM (622- Mbps) link, partitioned out with circuits for transit to Internet1, collaborative peering partners, and custom research connections.
Now our traffic and connections have diversified to meet the growing demand for unique connections. WiscNet still maintains an OC-12 connection to support commodity Internet traffic for the University of Wisconsin's 26 campuses, as well as other public institutions, such as local school systems serving grades K-12. WiscNet also maintains diversified routes to the Internet through additional high-speed connections in different locations in Wisconsin.
Large data interchanges with national and international research collaborators mean network demands at UW-Madison have exceeded the needs of other members of WiscNet. As a result, a DWDM optical link carries our Internet2 traffic over a 10-Gbps connection. UW-Madison shares this network connectivity with our WiscNet partners. Before the private fiber connection to Chicago, we would have had to bid out our 10-Gbps network project to a commercial ISP. Based on initial discussions with our state-contracted provider, we believe the long-term cost would have been an order of magnitude more than our implementation with our peer partnerships.
With DWDM, we can add capacity for a fraction of the initial implementation cost. In combination with our current equipment, one pair of single-mode fiber can carry 32 wavelengths--that's 32 times 10 Gbps. As the Northern Tier Network Consortium project continues to identify fiber paths farther to the west, the same fiber pairs and equipment in use by UW-Madison can be used to carry other institutions' traffic at different wavelengths.
A recently completed project makes a direct 10-Gbps network connection from the UW-Madison high-energy physics department to the data sets generated by CERN CMS, a scientific project at the Fermi National Accelerator Lab in Illinois. This project is expected to generate up to 10 petabytes of data per year. These large data sets are propagated from CERN out to Tier 1 data stores. UW-Madison is a Tier 2 site that fetches the data from the Tier 1 center at Fermilab in Batavia, Ill.
|(click image for larger view)|
National LambdaRail could connect more than two dozen cities.
Note that the Fermilab already receives this data directly from the CMS project and isn't the only lab getting that first distribution. New 10-Gbps and faster links enable data sharing at a level never before possible.
Once the Boreas-Net is completed, regional higher-education partners will be able to light up connectivity to the LambdaRail and Internet2 connecting points. We'll also be able to directly peer with neighboring universities in bandwidths that are multiples of 10 Gbps and share large data sets as though they were stored locally on our campus.
Beyond the standard network interchange, possible uses for those private fiber connections are endless. For instance, partner institutions could make agreements to supply each other with server platforms and co-locate the other's backup data center. The network would have a slight latency, but bandwidth of 10 Gbps should be able to handle most needs. If not, simply add another lambda.
Because current gear--Cisco ONS 15454 MSTP--supports 32 wavelengths, many institutions can share the same pair of fiber for any variety of geographic paths, as well as any variety of network traffic. Technological advances will let us continue to put more wavelengths on a single pair of fiber strands.