Fixing the Foam: Preventing Disaster, Getting Clear Picture

This story is Chapter 3 in an 11-part series by Florida Today.

NEW ORLEANS - The stuffthat changed everything at NASA feels like the foam in life jackets on afishing boat. It's super light, mostly air. Hold it in your hand and it defiescommon logic. How could a chunk of this foam bring down a mighty space shuttle?

Even the engineers whospent decades working on the foam, which prevents ice from growing on theoutside of the fuel tank that holds a half-million gallons of two of thecoldest substances on Earth, never feared the kind of catastrophe they saw in2003.

They'd seen foam come offthe tank for more than 20 years, usually popcorn-size pieces, and they had longago decided they were only dealing with a maintenance problem. How could theequivalent of a foam beer cooler hurt an aircraft? But the shuttle turns realityinside out. The environment it flies through those first nine minutes fromEarth to space is a different realm, and for a few fleeting seconds, even foambecomes a potential killer.

Steve Holmes is stillcoping with that new reality. Holmes, who lives with his wife, Pam, and twokids in Huntsville, Ala., is a shuttle fuel tank engineer who spent the lasttwo years trying to figure out why chunks of foam insulation come off theexternal fuel tank and how to stop it.

NASA had tried to solve theproblem about a dozen times with varying success, but never eliminated it. Now,with the foam implicated in the loss of a $2 billion spaceship and sevenpeople's lives, trying wasn't good enough. Holmes and the tank team had onlyone option: success.

The prime fix was obvious-- get rid of the two big wedge-shaped ramps of foam that protect a V-shapedstrut connecting the tank to the orbiter's nose.

One of the two triangularfoam blocks ripped free of the tank, probably because of an air pocket or otherflaw hidden inside the hand-molded ramps.

With the shuttle roaringupward, the lightweight foam slammed into the spaceship with the force of a tonof bricks -- literally.

But that wasn't the onlyfix. Post-accident testing of shuttle heat-shielding tiles and wing panelsshowed the materials were not as strong as NASA believed. Indeed, pieces offoam as small as a cupcake -- something a tenth the size and weight of what hitColumbia -- could be fatal.

So Holmes and hundreds ofhis co-workers had to stop anything bigger than three-hundredths of a pound --a square about the size of a breakfast muffin -- from breaking off the tankduring flight. It was a tall order beyond what NASA believed possible.

Since 2003, Holmes hasshuttled from the tank's design center in Huntsville and the factory near NewOrleans. He has missed soccer games and school functions, a lot of what'shappening in the lives of his children, Madeleine, 14, and Thomas, 6.

"I've tried to makethe birthdays and major holidays," said Holmes, a self-proclaimed spacecadet who went to work for NASA in 1989 and joined the external tank project adecade later.

Members of the tank teamwent to work early every morning and came home late every night.

They constantly passed theposters plastered on hallways featuring shuttle commander Eileen Collinsholding her grinning daughter, Bridget. The slogan: "Are you ready for usto go? Think safety."

No one publicly pointedfingers at the men and women of the tank program, but the team felt the painanyway. And they were determined never to feel the agony of the Columbiaseven's loss again.

"A lot of people did alot of soul-searching about what could have been done different," Holmessaid. "This is something that could have been taken care of a long timeago."

Among the long-acceptedproblems that had to be fixed: a foot-wide strip of extra-thick foam near wherethe round barrel of the tank meets the rounded pointy end. The ridgeconsistently shed Frisbee-sized pieces of foam.

Across 95 percent of thetank, robots spray a near-perfect layer of the white liquid foam, which hardensinto the orangish color people are accustomed to seeing on the launch pad.

In hard-to-reach placeswith bumps, grooves or other odd surfaces, technicians spray or pour the foaminto place. That's where most flaws hide.

The ridge -- called aflange -- is one of those handmade spots and one of the tank's weak points.

Engineers, tank buildersand foam sprayers worked the problem side by side; everyone had ideas.

They sliced and dicedanother tank's foam looking for air pockets, cracks and defects.

They chilled the inside ofmetal panels coated with foam, trying to re-create the extreme temperatures thestuff endures at the launch pad and during flight.

Fixes designed by engineerswere perfected in countless practice sessions by men who've been coating thetanks for 20 years.

The "sprayers" would work over and over on new techniques,experiencing failure upon failure, before finding something that worked.

One theory was that tiny cracks or holes could let gas get behind the foam,then rapidly expand as the temperature skyrockets during launch.

The resulting pressure pops foam off in chunks large and small. Experimentafter experiment ended in frustration. Nothing worked.

Then, one day, epiphany. Spraying a test panel, and watching it react, thegroup figured it out.

Gas crept through the threads of bolts holding together two parts of tank. Thebolts were hard to spray around, and gas could sneak into tiny paths around thethreads.

"I think we've got it!" one of the guys shouted.

Repeated tests proved they were right. Engineers went off to figure out how tostop it.

The answer: Turn the bolts upside down so they were easier to reach with foamspray guns and fill the bolt threads with material to prevent gas from creepingpast.

The sprayers practiced and practiced before they sprayed real foam on the realtank.

Finally, just before the New Year, the managers, engineers and factory workerscelebrated as they loaded ET No. 120 onto a barge that crossed the Gulf ofMexico, sailed around the southern tip of Florida and ultimately drifted into abasin at Kennedy Space Center the first week of January.

Nobody relaxed, though. Discovery's tank was delivered, but every tank has tobe perfect now.

"We shipped one tank, and we're getting ready to ship another one,"Holmes said shortly after shipping the first tank. "It's going to beanother three or four months before we're going to see any more breathingroom."

KENNEDY SPACE CENTER
Engineer keeps cameras focused on shuttle

Armando Oliu, whose films first spotted the debris that doomed Columbia, is nowin the middle of making sure the tank is fixed.

The trick for Oliu is balancing new camera shots against making sure he atleast gets what he always had gotten before.

Oliu's team upgraded to high-definition television cameras. They repaired orreplaced giant lenses -- really telescopes -- that are attached to some camerasalong the coast north and south of the pad.

Cameras will be on boats, airplanes and even the shuttle itself. Never beforehave shuttle engineers had so many looks, so many angles, such detailed imageryof a launching spacecraft.

Once Discovery's in space, Oliu and

a team of engineers will spend more than two days studying thousands of frames.

That's just if everything looks good.

Oliu and colleague Bob Page will brief mission managers each of the first fewdays of the flight on what they see.

This time, they'll have the best of the best equipment to do the analysis. Apowerful bank of computers hums in the film lab now. Giant screens and banks ofmonitors are everywhere.

Oliu painted the floors black on the advice of film experts because it wouldreduce the glare so people can see better -- the same reason movie theaters areblack or very dark colors.

They bought a projector used at modern movie houses playing digital versions ofHollywood films.

"There are a few extremely rich individuals who would have this at home.Bill Gates, maybe," Oliu said.

Oliu never expected to become so engrossed in cameras. He came to NASA afterthe Challenger accident in 1986. A systems engineer, he spent time working onthe external tank and the International Space Station before being asked tojoin the ice and debris team.

The engineers not only are experts on various shuttle systems; they're knownfor excruciating attention to detail in reviewing films for trouble.

They also do one of the most dangerous jobs in the program other thanastronauts.

As the shuttle is fueled before launch, they walk up and down the evacuated300-plus-foot tower looking for debris, cracks in the external tank'sinsulating foam, large chunks of ice, anything that could hit the delicateorbiter during liftoff.

Now, Oliu is in the middle of the most important inspection ever done on ashuttle.

The hours have been longer. The travel has been more extensive. But Oliu's teamtries to balance work and family.

Oliu works hard. His wife, Jennifer, understands. But he's no workaholic.

"I have a level of sacrifice to make. But first is my family," hesaid. "I'll work to get the job done, but I'm not going to destroy myfamily for the space program. . . . Every individual's got to weigh that pullfrom work with what's right for the family."

He met Jennifer at a NASA picnic. She was a nurse who worked with the wife ofone of Oliu's buddies. Armando and Jennifer hit it off and got married in 1997.

A few years later, Victoria was born. She's almost 6 now, a kindergartener inCocoa Beach. She knows her dad works on the shuttle, but he says, "she'sjust getting old enough to understand. She still thinks I'm an astronaut, forGod's sake."

Oliu's job, the investigation, the return-to-flight effort have put him in thespotlight. He has been on television several times. She notices a rocket liftingoff or something space-related on the tube at home in Rockledge and she asks,"Daddy, are you going to be on TV again?"

"Hopefully not," he jokes, somewhat uncomfortable with being putforward to reporters to represent the work of dozens of others.

Off camera, he's still making the kind of frank, no-nonsense calls people havecome to expect from him.

If he thinks your idea is bad, he'll say so. No hard feelings. Get the jobdone.

There's been plenty of that since Columbia.

As incredible as the shuttle tracking camera system always has been, it's notgood enough anymore. The loss of seven astronauts exposed a host of seeminglyinnocuous problems that, left unfixed too long, turned deadly.

Broken tracking gear, out-of-focus cameras, glitchy remote control systems andhuman error all contributed to pictures that Oliu complained to bosses were"simply unacceptable" and "unusable" for engineers tryingto make life-or-death decisions.

In the years since, Oliu and others have put every camera and related piece ofequipment under the microscope.

Some wanted to replace almost everything with the latest technology.

For instance: There's a camera north of the pad near a place called Shiloh, theone and only camera that captured the leak of fuel from the Challenger boosterrocket and helped solve the mystery of that accident. Some wanted to shut itdown. A new one nearby gets the same look.

"Let's not take that one out just yet," Oliu said.

Why? Because shuttle engineers know this: You never know. Oliu wants topreserve at least the images that engineers have come to expect, at least fornow. Once he sees the new cameras work, he'll feel safer.

"Let's not start deleting cameras," Oliu said. "We know what wecan get out of this camera. We've gotten used to certain views from certaincameras. If you do everything brand new, it's going to be a mess."

Publishedunder license from FLORIDA TODAY. Copyright ? 2005 FLORIDA TODAY.No portion of this material may be reproduced in any way without the writtenconsent of FLORIDA TODAY.

        Fixing NASA: Complete Coverage ofSpace Shuttle Return to Flight