Retired Microsoft engineer Dave Plummer has taken to his YouTube channel to explain Redmond's missteps with Windows Longhorn and the background to the company's failed attempt at an XP follow-up.
It's been just over 20 years since the Longhorn project, as initially envisioned, was reset. Plummer worked on the project – it was one of his last at Microsoft before retirement. "Longhorn is a tale of big ambitions, technical nightmares, and an extraordinary comeback," he said.
We'd argue that many users who were content with XP but frustrated by Windows Vista would take issue with that last point.
"Microsoft wanted Longhorn to reimagine what Windows could be," Plummer said. This meant a completely new user experience, some fundamental changes behind the scenes, and, of course, WinFS (short for Windows Future Storage).
Plummer described WinFS as one of the "crown jewels" of the new operating system. Rather than the file-based approach used up to that point, WinFS would utilize a database, meaning users could search for files by content tags and relationships rather than by name and folder.
It was, Plummer noted, "a bold plan."
There was also a shift to managed code with .NET, a new presentation subsystem, and a new framework for communications and web services. "Windows XP's colorful Luna interface was going to be yesterday's news," said Plummer. "Longhorn would be sleek, modern, and visually rich."
"It's not often that an operating system promises to reinvent multiple core pillars all at once. Longhorn was aiming for the sky but – and you knew a 'but' was coming – ambitions often come with big challenges."
Plummer recalled the storm clouds gathering as development ramped. Microsoft staffers connected to the project could see it buckling under its own weight.
"One major problem," he said, "was that with so many new features being developed simultaneously, the Windows codebase became fragile and bloated." WinFS was a major culprit, but pretty much the entire stack was tottering. "The integration of all these pieces became a nightmare.
"We had a tradition at Microsoft called 'Dog Fooding,' meaning using our own daily builds of the OS to do our actual work. For Longhorn, dog fooding was tough. Builds were often too unstable to run for very long.
"The degree to which eating dog food is tolerable can be measured in direct proportion to how good the dog food actually is. And this was not good dog food."
It's a familiar story. Developers would check in code, not realizing that their changes would break something else. Getting a stable build where everything worked together without major bugs proved increasingly difficult. Plus, there was the need for backward compatibility.
"The Longhorn team was walking a tightrope," recalled Plummer. "Innovate aggressively, but don't break Windows. Not an easy balancing act."
Then came the infamous memo in 2002, in which Microsoft's head honcho, Bill Gates, declared security to be the company's top priority. Securing Windows was the right call, but it meant that development on Longhorn slowed as engineers, including Plummer, were called upon to patch holes in Windows XP.
Even with teams scattered, Plummer said Longhorn feature creep continued. "It's demoralizing as an engineer when you can see the light at the end of the tunnel getting further away and not closer."
Around this time, Plummer took a sabbatical from Microsoft and decided to retire. He said the Longhorn fiasco played only a small part, but "it almost seems like I dodged a bullet."
"I'm an incrementalist, not a revolutionary," he said. "I prefer to add things one solid feature at a time rather than reinventing everything all at once as part of some grand vision."
The fate of Longhorn is well documented. While the consumer team continued to add features, taking the attitude, according to Plummer, that "Consumer Windows could be less rigorously engineered than Server Windows," the server team took a methodical approach to ensure stability.
It all came to a head when, as Plummer recounts, Dave Cutler – who had led the development of Windows NT – noted the severity of the situation and suggested that the codebase used for Longhorn be switched to the one the server team had been working on.
"It's never pleasant to consider throwing away years of work, and it certainly bruised some egos to admit that the Longhorn project had gone too far off track," said Plummer.
The death knell came with the reset of August 2004 and the public announcement that the existing Longhorn codebase would be scrapped. Instead, the Windows Server 2003 SP1 codebase would be used, with the most important, or most complete, Longhorn features added.
Plummer retired before the course correction, but said the announcement was shocking. "Imagine telling hundreds of developers and testers, 'All that work that you've done for the past three years, we're going to set that aside and do something else.' It was devastating, yet oddly enough, a relief for some.
"Devastating because nobody wants to see their hard work tossed out, but a relief because, by this point, everybody knew it wasn't working. The project had become a death march of sorts."
The result was Windows Vista. The operating system retained some of Longhorn's features, such as translucent window effects, but discarded others, including WinFS.
"Was the Longhorn project a failure?" mused Plummer. "Well... yes and no."
"It failed to deliver on its original promises. WinFS was never shipped, many of the more radical ideas were cut, and the schedule slipped dramatically. It was also a management failure as it took a near-crisis for Microsoft to course-correct.
"But Longhorn's legacy is complex and, in many ways, a positive one. The reset and recovery from Longhorn shaped Microsoft's culture and engineering practices going forward."
Plummer regards the story of Longhorn as a cautionary tale. "It shows that even a company with virtually limitless resources and a track record of shipping big products can get in over its head.
"The story of Longhorn is one of hubris and humility; Microsoft reached for the stars, stumbled, but then humbled itself to regroup and ultimately deliver something worthwhile." ®