NTP's Fate Hinges On 'Father Time' - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
IT Life
News
3/11/2015
06:06 PM
Connect Directly
Twitter
RSS
E-Mail
100%
0%

NTP's Fate Hinges On 'Father Time'

The Network Time Protocol provides a foundation to modern computing. So why does NTP's support hinge so much on the shaky finances of one 59-year-old developer?

(Image: Geralt via Pixabay)

(Image: Geralt via Pixabay)

In April, one of the open source code movement's first and biggest success stories, the Network Time Protocol, will reach a decision point. At 30 years old, will NTP continue as the preeminent time synchronization system for Macs, Windows, and Linux computers and most servers on networks?

Or will this protocol go into a decline marked by drastically slowed development, fewer bug fixes, and greater security risks for the computers that use it? The question hinges to a surprising degree on the personal finances of a 59-year-old technologist in Talent, Ore., named Harlan Stenn.

The Network Time Protocol is important enough that the likes of Google and Apple speak up if they find a bug in the protocol that needs fixing, or a modification they think is needed. But NTP has worked so well for so long that few people think there's any problem.

Not all is well within the NTP open source project. The number of volunteer contributors -- those who submit code for periodic updates, examine bug reports, and write fixes -- has shrunk over its long lifespan, even as its importance has increased. Its ongoing development and maintenance now rest mostly on the shoulders of Stenn, and that's why NTP faces a turning point. Stenn, who also works sporadically on his own consulting business, has given himself a deadline: Garner more financial support by April, "or look for regular work.”

(Image: Margaret Clark)

(Image: Margaret Clark)

Stenn's shaky personal finances illustrate one very real risk to the future of the Internet. A number of widely used foundations of the Internet -- such as OpenSSL, the Domain Name System, and NTP -- are based on open source code. Open source means no one owns the software, anyone can use it, and it's maintained through a collaborative process of people submitting changes to a central governing group. Some open source projects, such as the Android mobile OS, have a rich uncle like Google that pays people who maintain the code as a side job. Or, the project is trendy enough that working on it helps to spur consulting work. But a project like NTP, which is buried deep in the infrastructure, doesn't have a clear-cut financial backer. That leaves support up to people like Stenn.

For the last three-and-a-half years, Stenn said he's worked 100-plus hours a week answering emails, accepting patches, rewriting patches to work across multiple operating systems, piecing together new releases, and administering the NTP mailing list. If NTP should get hacked or for some reason stop functioning, hundreds of thousands of systems would feel the consequences. "If that happened, all the critics would say, 'See, you can't trust open source code,'" said Stenn.

Sam Ramji, CEO of the Cloud Foundry Foundation, cited Stenn’s work in an address at the Open Compute Summit 2015 in San Jose Mar. 11. He dubbed him "Father Time," and said he was "scraping by" as he continued to work on NTP.

Stenn is hardly the only open source coder living in such straits. Ramji also mentioned Werner Koch in Germany, the author and maintainer of Gnu Privacy Guard, which is used in three popular email encryption programs. In a Feb. 5 article, Koch told ProPublica that he was "going broke" on $25,000 a year since 2001. Chet Ramey, part of the networking infrastructure team at Case Western Reserve, has been the primary maintainer of the Bash shell for Unix since 1990 with minimal support.

Ramji noted that OpenSSL developers had been receiving less than $2,000 a year in donations when the Heartbleed exploit of OpenSSL broke out last April. "Secure code is hard to write and maintain," Ramji noted. Users have to decide whether they want to leave these projects to survive as best they can.

Next Page: Watching the timekeeper

Charles Babcock is an editor-at-large for InformationWeek and author of Management Strategies for the Cloud Revolution, a McGraw-Hill book. He is the former editor-in-chief of Digital News, former software editor of Computerworld and former technology editor of Interactive ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Previous
1 of 5
Next
Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
hstenn
100%
0%
hstenn,
User Rank: Apprentice
3/11/2015 | 6:58:32 PM
It's fine to send big money directly to NTF!
Wow, thanks for the great article, sir!

I've always considered myself either the "janitor of time" or maybe NTP's babysitter.

From my point of view, we want lots of companies and individuals to "join" Network Time Foundation.  We are also very happy to receive direct donations.  As long as we get enough donations from enough sources to meet the IRS Public Support Test we're happy, and all is well.

 
Doug Henschen
100%
0%
Doug Henschen,
User Rank: Moderator
3/12/2015 | 12:50:53 PM
Re: It's fine to send big money directly to NTF!
Keep up the good work, as best as you can as long as you can, but by all means cultivate a backup, particularly if Charlie's article helps spur more funding. This seems like it's too imporant to rest on your shoulders alone. What if you get hit by a truck? Too bad you can't set up a little tarrif for accessing NTP!
sejtam
50%
50%
sejtam,
User Rank: Apprentice
3/13/2015 | 6:46:02 AM
Re: It's fine to send big money directly to NTF!
For individual donations, a campaign on GoFundMe or Kickstarter or somesuch site would be useful (or maybe just Amazon Payments)?
curts88
50%
50%
curts88,
User Rank: Apprentice
3/15/2015 | 11:59:20 AM
Free PTP implementation for Windows?
Last time I checked (sometime in 2014) there were no free implementations of PTP for Windows. This situation probably needs to change if we expect PTP to gradually replace NTP. Maybe Microsoft should include PTP support in Windows 10?
Charlie Babcock
100%
0%
Charlie Babcock,
User Rank: Author
3/11/2015 | 11:02:50 PM
Are you ready to roll back the next Leap Second?
Harlan Stenn is the only person I know who has already laid plans to cope with the Leap Second that will need to be subtracted from the solar day on June 30, when the discrepancy  between UTC and TAI will reach 36 seconds. For that, I'm afraid we must call him Father Time, even though he doesn't much like the moniker. (Leap Seconds occur irregularly, averaging one every 18 months.)
Todder
100%
0%
Todder,
User Rank: Moderator
3/12/2015 | 9:47:20 AM
Re: Are you ready to roll back the next Leap Second?
Very good article Charlie. One of the best I've read in a long time. Hopefully this stirs up some funds for Harlan and the project. I'm also surprised that academia hasn't jumped right back in on NTP.
Somedude8
100%
0%
Somedude8,
User Rank: Ninja
3/12/2015 | 11:41:52 AM
Let it fail?
Companies will pay as little as they have to for any given service. Its not greed, its just their nature. They have been used to getting NTP for free for so long, they aren't likely to start coughing up funds out of the goodness of their hearts.

Harlan's efforts sound like they have been truly heroic, and at that heroic pace for a very long time. But I think he is flat out being taken advantage of, and will continue to be until he puts his foot down. Maybe just flat out refuse to work more than 15 or 20 hours per week on it, putting the rest of his time in to his consulting business or something. I know that $7k per month looks good on paper, but I gaurantee its not nearly as much in practice.

Let it fail. That will bring the needed attention as funding. Its harsh, and in many ways against the open source spirit, but the guy has been taken advantage of long enough. When Google and Apple and others aren't even chipping in a little bit, especially taking in to account the huge consideration he gave Apple, then the whole thing is just wrong.
I_just_wanna_say_
50%
50%
I_just_wanna_say_,
User Rank: Apprentice
3/12/2015 | 11:12:21 PM
Re: Let it fail?
Or alternatively, Apple, who just announced the Apple Watch, could set up a small endowment to support the project in perpetuity and use that as a marketing hook.  Just think, the Apple Watch, a product that seems completely pointless to most people could suddenly become a badge of geeky insider knowledge ---

OK I'm naive.
JanK591
50%
50%
JanK591,
User Rank: Apprentice
3/13/2015 | 3:21:56 AM
Re: Pending Review
Over the years I came to a conclusion that attempting a donation-based model for open source/free software is futile effort. Only big endavours seem to be able to support themselves their way. Same goes for business model based on giving product for free and selling support. I think the FLOSS community was completely wrong in presenting this model as an alternative to traditional software business.


As I see it, there are only two options available for FLOSS projects. First option is fully voluntary work by people who are able to support themselves in other way. Second option is big business who uses FLOSS in their operations - not necesarily in their product, but also in infrastructure. Working together with other enterprises on commonly needed features is clearly beneficial. The second model is already prominent, but need more time to sink in. CEOs need to understand that cooperation is mutually beneficial and doesn't equal "giving away for free". Even when majority of users don't contribute code, they still do a tremendous job of spreading the word, thus giving the project more potential developers.
JanK591
50%
50%
JanK591,
User Rank: Apprentice
3/13/2015 | 3:30:37 AM
Re: Let it fail?
I think it's completely in line with open source spirit. Various projects rise up and fall all the time. FLOSS was always about freedom of choice and if ntpd will be missed, your solution will resolve the problem, just as you said.
gmerriman112
100%
0%
gmerriman112,
User Rank: Strategist
3/12/2015 | 12:33:55 PM
Typical of the way things are going
I think the problems noted here are symptomatic of a larger problem. It seems that everyone these days is focused on getting rich with the next "Yo!" app or shiny new wearable gizmo that nobody is worried about keeping the foundations on which all this new technology depends. If the industry as a whole does not come to grips with this problem soon I fear we are in for a rough time ahead.
ThomasW840
50%
50%
ThomasW840,
User Rank: Apprentice
3/16/2015 | 1:41:34 AM
Why not centrlize critical app/protocol financial support at the distros?
Each GNU/Linux distro should have a post install "donate to FOSS" option that allows users of FOSS or Linux distros to donate to these critical, base apps and sysyems.  Very simple issue to solve here folks..

Tweeks
Tony A
100%
0%
Tony A,
User Rank: Moderator
3/12/2015 | 1:58:58 PM
Truly Disturbing
This is the most unsettling thing I've read since a report many years ago that said all international Internet traffic was being routed through a garage in Virginia. Truly amazing how vulnerable the entire infrastructure is in spite of the fact that it is now the  single most important vehicle for both business and defense.

So nobody wants to pay for an infrastructure service that is crucial to the Pentagon, the stock market, major search providers, ISP's and satellite communications? I say you send Goldman Sachs, Google, Microsoft and a few others - don't forget our friends at Verizon and Time Warner who oppose net neutrality - a letter written on a Smith Corona typewriter saying that the servers will be going down for one second at 12:00:00 a.m. next Monday, to save energy costs. Then see if they come up with some funding.

"We don't contribute to open source projects" - that's a classic. Google spends more on sushi and GS on annual bonuses than they are willing to spend on a service that their entire revenue streams depend on. As if the Internet were a natural resource like air and water and they are just entitled to it. It's unbelievable what we put up with from these companies. They should fund a pension for Mr. Stenn and his family in addition to ponying up a few million $ to keep the service stable and secure.
JanK591
50%
50%
JanK591,
User Rank: Apprentice
3/13/2015 | 3:26:43 AM
Re: Truly Disturbing
There are no problems with server costs, in fact I am sure that most of companies mentioned here do maintain their NTP servers, probably in large amount. The problem is with funding the development of software that runs on those servers. Cessation of development won't bring any immediate catastrophe and hence doesn't make a convincing argument to people who could pay for it.
EJW
50%
50%
EJW,
User Rank: Apprentice
3/12/2015 | 4:59:11 PM
No Free Lunch
I have just read Charles Babcocks excellent article about NTP.org and Harlan Stenn.

I wasn't aware of the situation: it is utterly shameful that the likes of Apple, Google and others have allowed this to occur while freeloading on the software that allows them to make millions.

And it is not just the IT industry:

"...the NTP time stamp is one of the few ways equities firms have of proving to regulators they were in compliance of making a trade..."

So where are the contributions from Wall Street?

There is no free lunch!

Sooner or later everything must be paid for by someone; from NTP to PBS and even whole economies (just ask the Greeks).

I just made my contribution.

Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/12/2015 | 5:56:51 PM
Yes, some contribute, some don't
Good comment from EJW, an IT manager in the Calif. State University system, and Tony J, thanks. I would like to note that Google is a contributor to the Linux Foundation's Core Infrastructure Initiative, now supporting Stenn. The CII includes Amazon, IBM, others and supports Werner Koch's Gnu PG in Germany & other projects as well. But there aren't enough $$ to go around. Stenn's non-profit is at www.nwtime.org. Checks can be sent to Network Time Foundation, PO Box 918, Talent, OR. 97540.
hstenn
IW Pick
100%
0%
hstenn,
User Rank: Apprentice
3/15/2015 | 11:12:08 PM
Re: Yes, some contribute, some don't
Thanks a bunch, Charlie!

Slight clarification: Network Time Foundation is not "my" non-profit, I'm just the founder and president. it's there for public benefit.

There are "donate" and "join" links at www.nwtime.org and we do also accept PayPal.  We're looking at some other "ways to send money".

The feedback and support we've already seen is heartwarming, and it will currently cover about 2 more weeks of my time.  We've also heard from a few companies that have said "we saw the article and we're looking to help, we'll be in touch soon."

Network Time Foundation has no anonymous institutional or governmental supporters.  If you don't see their name on our site, they're not supporting us directly.  The reason Linux Foundation is not there is they insisted on sending their money directly to me and PHK, instead of to NTF.  I can appreciate their reasons.  Having said that, if you are using software or equipment that uses network time and you don't see that company listed, please contact them and ask them to support us!  They will listen to you more than they'll listen to us...
ThomasW840
50%
50%
ThomasW840,
User Rank: Apprentice
3/16/2015 | 1:44:10 AM
Re: Yes, some contribute, some don't
So why don't all the FOSS distro vednros band together and start a FOSS foundation, and allow the distro installer to donate to suport critical base apps & systems (and optionally along with other groups and efforts)?  Even give them 501 tax status and writeoffs! You KNOW their donations would get a bump every April. :)

 

Tweeks
sejtam
50%
50%
sejtam,
User Rank: Apprentice
3/13/2015 | 6:59:39 AM
pool.ntp.org
While I agree with this article 100% i think it should have been mentioned that there are a number of organizations (and individuals) who donate a server and bandwidth to the pool.ntp.org project which provides much of the net-wide NTP distribution.
Jamescon
50%
50%
Jamescon,
User Rank: Author
3/13/2015 | 10:17:07 AM
A look behind the scenes
Charlie. Great idea to look into the inner workings of technology (the code and the people) that most of us never see. Well done.
GIGABOB
50%
50%
GIGABOB,
User Rank: Strategist
3/13/2015 | 2:12:21 PM
Is there really a problem?
As a prior Oregon developer who needed a real job I appreciate Stenn's dilemma.  I am less concerned about shipping Stenn a few bucks than creating a better vehicle to support critical open source protocols like NTP. 

Stenn really needs help i nunderstanding how to monetize his efforts.  I suggest a microcent per millisecond.

At the end of the day do you see a lack of industry support for this activity or a vicious fight for gatekeeper rights?
Li Tan
50%
50%
Li Tan,
User Rank: Ninja
3/14/2015 | 7:04:49 AM
Re: Is there really a problem?
Such kind of important open source protocol deserves more attention - it's so important that everybody took it for granted. Then it's a real trouble if one day it stops working.
Susan_Nunziata
50%
50%
Susan_Nunziata,
User Rank: Strategist
3/16/2015 | 7:48:06 AM
Re: Is there really a problem?
@Gigabob: Your comment caught my eye, especially this: creating a better vehicle to support critical open source protocols like NTP.

Having been thru a similar experience yourself, what would you say is required to create such a vehicle for NTP (and other critical open source projects).

 

 
Mr. Gigabob
50%
50%
Mr. Gigabob,
User Rank: Strategist
3/18/2015 | 12:45:27 PM
Re: Is there really a problem?
The process is straightforward - an industry group with a vested interest steps in and enlists support from an eco-system by starting a "Time Committee" with contributions from those organizations in the form of team members and fiscal backing levels.  DLNA, USB, WiFi all started this way as a prelude to creating and adopting a standard.  The more groups the get behind supporting NTP - the more that will build in NTP into their systems.  

Ideally, increased investment in time synchronization for security, log management and other roles will add many paths to orbiting atomic clocks in GPS satellites to increase accuracy of NTP so it eclipses PTP - Precision Time Protocol (IEEE1588).

NTP and PTP approach the problem from different angles.  PTP uses hardware to provide a precise local clock with accuracy to 100ns and very little software sophistication.  NTP uses software and statistics to get time from local motherboards and other sources then distribute across a network.  Accuracy varies widely from micro-seconds to 10's of milliseconds, as distribution delays across shared network links are impacted by busy workloads.  Until there is a ubiquity of precision time sources available with known latency, we need both.

As an example of industry standard support - suppose members of the "TIME ASSOCIATION" included all the major home network router vendors.  Their support for NTP might include some local intelligence and a dedicated port channel for distributing time information that would have a prioritezed Quality of Service level enabling it to consistently provide microsecond accuracy in the home.  This would be advertised as a selling point and if embraced by users would prolieferate across the Customer Premises landscape.

Ironically we have access to precision time in to 100ns today.  Everyone with a GPS chip in their mobile phone leverages the GPS time in the orbiting satellites.  Perhaps it is time to codify that into a new standard.
jeff_logullo
IW Pick
100%
0%
jeff_logullo,
User Rank: Apprentice
3/13/2015 | 6:53:30 PM
UTC = Coordinated Universal Time
Great article - thanks for shedding light on the issue, especially that of the unsung heroes of the internet and the open source community!

One small comment: the abbreviation "UTC" stands for "Coordinated Universal Time". You might wonder how that acronym makes sense... seems it should be "CUT" instead.

We English speakers call it Coordinated Universal Time -- which would make the acronym CUT.

French speakers, however, call it Temps Universel Cordonné -- which would result in TUC.

What to do? Compromise! Instead of CUT or TUC, the alternative UTC was chosen. It plays no favorites! Strange but true.

The wikipedia entry for Coordinated Universal Time has more details, including a reference to the IAU resolution in 1976 when this decision was made.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/13/2015 | 7:25:01 PM
Oops, Coverity is a hidden contributor
I've learned there are a few hidden contributors to NTP. For example, Stenn uses almost 100% open source code but I knew he liked to check his code against the Coverity's security and bug detecting software, a commercial service. So the first version of this story listed Coverity as a service he had to pay for from his slender resources. It turns out that Coverity contributes its service to NTP. Stenn has also used BitMover's BitKeeper, commercial software for source code management, which he likes better than open source git. "Because (CEO) Larry McVoy appreciates the NTP Project, they've freely given my entire team licenses to bk, and they've given us free enterprise-class service as well, for nearly '14 years' time,'" Stenn wrote in a follow-up message.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/15/2015 | 5:11:27 PM
If UTC stands for Coordinated Universal Time, then why TAI?
I have always wondered why Coordinated Universal Time is abbreviated UTC. There's a gem of an explanation below by Jeff_Logullo, who happens to be a pre-sales engineer for the Oracle's Public Sector Systems division. Can anyone confirm what he's saying? Jeff doesn't remember where he first heard the story.

Then, 2), can someone explain to me why TAI is used as the acronym for International Atomic Time? (Don't tell me it's the French, again--temps atomique international?)
hstenn
50%
50%
hstenn,
User Rank: Apprentice
3/15/2015 | 10:39:30 PM
Re: If UTC stands for Coordinated Universal Time, then why TAI?
In English, TAI is "International Atomic Time".  In French it's "Temps Atomique International".

 

In English, UTC is "Coordinated Universal Time", while in French it's "Temps Universel Coordonné".  This way the French and English speakers are equally unhappy with the acronym.
vorlonken
50%
50%
vorlonken,
User Rank: Apprentice
3/16/2015 | 1:28:34 PM
Here's the proper solution
Author: "So, Mr Stenn, what will you do if huge companies like Google, Microsoft, IBM, Apple, Cisco, Intel, etc don't start contributing? They could each donate $10 million/year with the change culled from under the driver's seat of the CEO!" Stenn: (shrugs) That's how the article should end. I hope everyone out there got my very unsubtle reference.
mbperezpinilla
50%
50%
mbperezpinilla,
User Rank: Apprentice
3/17/2015 | 7:07:24 AM
Radioactive Cesium-133???
Cesium-133 is the only stable isotope of Cesium!
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/17/2015 | 2:07:46 PM
You're right, Cesium-133 is stable, not decomposing
mbperezpinilla, A second as measured by an atomic clock is "9,192,631,770 cycles of radiation" reflecting the transition in energy levels of the Caesium-133 atom, according to the International System of Units. I didn't realize radiation in this case doesn't mean (ouch) radioactive. I've always thought atomic cloicks were using a measure of radioactive decomposition as a precise time-keeper. Instead, it's vibrations of the stable Cesium-133 atom that's keeping the beat. It's Cesium-137, used in medical imaging, that's radioactive. Oh boy, time to brush up on my physics.

 

 
pzjones
50%
50%
pzjones,
User Rank: Apprentice
3/18/2015 | 12:28:30 PM
Demonstrates change in motivation
I think this article clearly demonstrates what attracts people to IT now is not what drove many of us into IT 20+ years ago. It wasn't about the "job" or the "salary." It was about the love of this new technology, about being a pioneer in this industry, about collaboration, about conquering and innovating.

 It was nice that it came with a salary but that wasn't the driving force. I've seen many come because they thought they would make the big bucks but didn't have the heart or the passion and now they have gone...some stick around because "it's a job" and they don't want to go back to school. For those like Stenn, it is much more than that...it's in the blood. We need to figure out how to ignite that fire in the younger generation that has come to rely on technology without a desire to be part of it.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/18/2015 | 1:46:40 PM
Just a minute, Mr. Gigabob
Your answer is straightforward, Mr. Gigabob, except for the part about how we've had for years many companies with a vested interest in sychronizing time and they haven't done what you say should happen.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/23/2015 | 11:51:33 PM
Has this issue kept you up late at night?
This may not be something you've worried about lately, but the 32-bit counter in the Network Time Protocol's time stamp is able to designate any second that's occurred since Jan.1, 1900. The only thing bad about covering such an expanse of time is that the counter runs out of numbers sometime in 2036. Like I said, maybe you haven't worried about it -- yet. Harlan Stenn is up late at night thinking about the solution... Better keep him on the case.
kstaron
50%
50%
kstaron,
User Rank: Ninja
3/25/2015 | 3:52:11 PM
So the people using it aren't wiling to pay?
Can I guess that by the lack of this guy's wallet, that the companies who claim to care, like Google, have not stepped up and given the the guy funding to make sure the clock keeps ticking? If he has not yet, I would suggest he approach each of the companies that uses NTP and tell them it's in danger of being unsupported without financial backing. Wake up the guys who use it and let them know the free ride is about over.
akostadinov
50%
50%
akostadinov,
User Rank: Apprentice
8/18/2015 | 10:50:33 AM
alternatives
chronyd anybody? (chrony.tuxfamily.org)

works better at least for some use cases...

Competition is good, otherwise things rot anyway.
Commentary
Why It's Nice to Know What Can Go Wrong with AI
James M. Connolly, Editorial Director, InformationWeek and Network Computing,  11/11/2019
Slideshows
Top-Paying U.S. Cities for Data Scientists and Data Analysts
Cynthia Harvey, Freelance Journalist, InformationWeek,  11/5/2019
Slideshows
10 Strategic Technology Trends for 2020
Jessica Davis, Senior Editor, Enterprise Apps,  11/1/2019
White Papers
Register for InformationWeek Newsletters
State of the Cloud
State of the Cloud
Cloud has drastically changed how IT organizations consume and deploy services in the digital age. This research report will delve into public, private and hybrid cloud adoption trends, with a special focus on infrastructure as a service and its role in the enterprise. Find out the challenges organizations are experiencing, and the technologies and strategies they are using to manage and mitigate those challenges today.
Video
Current Issue
Getting Started With Emerging Technologies
Looking to help your enterprise IT team ease the stress of putting new/emerging technologies such as AI, machine learning and IoT to work for their organizations? There are a few ways to get off on the right foot. In this report we share some expert advice on how to approach some of these seemingly daunting tech challenges.
Slideshows
Flash Poll