CAPE CANAVERAL, Fla. - The inaugural Boeing Delta 4-Heavy rocket suffered premature engine shutdowns during its December test launch because of bubbles in the liquid oxygen plumbing, investigators have concluded, and now corrective measures are being devised to prevent a repeat problem during the next launch in October.

The three engines were snuffed out several seconds early after internal sensors were fooled into believing the liquid oxygen fuel supply had been expended. That left the rocket with a massive underspeed in which the vehicle's upper stage could not overcome and resulted in a final orbit lower than planned.

"The root cause of the anomaly has been identified as a fluid cavitation within the liquid oxygen feed system," the Air Force said Wednesday in announcing the investigation's findings.

The cavitation, or bubbling, is a localized condition where the super-cold oxidizer changed from liquid to vapor within the feed lines running from the rocket's tanks to engines.

The Delta 4-Heavy is the largest member in Boeing's next-generation rocket family. It takes three Common Booster Cores, each featuring a cryogenic main engine, and straps them together to form a vehicle capable of launching hefty cargos into space.

Trouble on trip to space

The three Common Booster Cores were ignited during the final seconds of the December 21 countdown, generating 1.9-million pounds of thrust to propel the 23-story rocket away from pad 37B at Cape Canaveral Air Force Station, Florida. It was meant to be a full dress rehearsal flight -- with only a dummy payload aboard -- to test the rocket before critical national security satellites begin using the vehicle later this year.

About 50 seconds into flight, the center booster's main engine throttled back to 58 percent thrust as a fuel conservation effort. The starboard and port boosters continued to operate at their maximum power setting of 102 percent thrust, each guzzling a ton of propellant per second.

The strap-on boosters were scheduled to fire until T+plus 4 minutes, 5 seconds when the Rocketdyne-made RS-68 engine on each stage would cut off. About three seconds later, the 15-story starboard and port boosters, which provided the vast majority of thrust during the first four minutes of flight, would peel away from the center stage and tumble into the Atlantic Ocean below.

But the engines shut down 8 seconds early after sensors temporarily indicated "dry" fuel conditions despite the stages having plenty of propellant remaining to accomplish the scheduled firing time. The sensors returned to "wet" readings after the shutdown sequence was already activated.

Once the outer boosters were shed, the center stage's RS-68 engine revved back to full throttle. Although the booster was identical to the outer strap-on stages, carrying the same propellant supply and engine package, it employed a more conservative fuel consumption strategy by the lower-throttle setting for the past three minutes and saved enough propellant to operate almost 90 seconds longer.

But the same sensor "phenomenon" repeated on the center booster, causing its engine to shut down 9 seconds prematurely, according to investigators.

After the center booster had been jettisoned, the Delta 4-Heavy rocket's upper stage found itself with a speed deficit of 1,500 feet/second due to the early shutdowns of the main engines. The upper stage ignited for the first of three firings planned over the 6-hour mission to geosynchronous orbit.

That first burn of the Pratt & Whitney RL10 upper stage engine was supposed to last seven minutes to reach an initial parking orbit around Earth where a pair of university-built nanosatellites would be released into space. The rocket motor was designed to extend its firing time to compensate for any performance shortfalls experienced by the Common Booster Cores, and it did that. But even through the stage fired much longer than planned it still failed to reach a stable orbit, deploying the nanosats into a suborbital trajectory that took them into the atmosphere before completing a lap around the planet.

The upper stage then reignited for its second scheduled burn, shaping the rocket's track into a highly elliptical egg-shaped geosynchronous transfer orbit. It was in this orbit that the vehicle coasted for five hours to reach the high point about 19,600 nautical miles above the planet where the final engine blast would occur.

This firing should have lasted three minutes to circularize the orbit. However, the stage's precious fuel supply was greatly impacted by the extended maneuvers battling back from the Common Booster Core problem. The stage ran out of fuel about two-thirds of the way through the burn, leaving the instrumented satellite simulator payload -- the rocket's main cargo for this test flight -- with an orbit featuring a high point of 19,600 nautical miles (36,400 km), low point of 9,600 nautical miles (19,000 km) and inclination of 13.5 degrees. The orbit's low point was 10,000 miles off the target and inclination was 3.5 degrees higher than planned.

Tracking down the glitch

Each Common Booster Core has a large liquid hydrogen tank and a much smaller liquid oxygen tank for its RS-68 engine. The liquid oxygen tank is located at the top of each rocket stage, with a long feedline running down the booster's side to reach the engine.

"Analyses show that the cavitation originated at the entrance of the propellant feedline, where a filtration screen and turning elbow restrict the propellant flow as it accelerates leaving the tank. This feedline restriction has been present in all previous Delta 4 flights, but the unique combination of vehicle acceleration, liquid level in the tank, and propellant flow rate for this mission, reduced the fluid pressure enough to enable the creation of gaseous oxygen at this location as the tanks emptied," Wednesday's Air Force statement said.

"Further draining of the liquid oxygen tank worsened the conditions at the feedline inlet, causing the cavitation effect to extend down the feedline until it reached the liquid depletion sensors and caused them to momentarily toggle 'dry.' This action was sensed by the flight computer, which initiated the sequence to throttle-down and shut off the main engines as it is programmed to do. Flight data shows that sufficient propellant remained in the tank to complete the planned first stage burn time."

A Fault Tree analysis was used to examine potential causes of the problem, including propulsion, avionics, structures and flight environments. Forty-nine of 50 Fault Tree branches were "closed" after being ruled not credible.

"All closures were thoroughly documented, citing multiple sources of supporting evidence drawn from flight data, a range of focused technical analyses and computer simulation results," the Air Force said.

Other propellant phenomena like sloshing and "vapor pull-through" were analyzed and determined to be highly unlikely.

"This investigation has followed a deliberate process to ensure no potential causes were missed," said Maj. Rod Houser, investigation lead for the Air Force. "Our attention is now focused on the final open branch of the Fault Tree dealing with cavitation within the liquid oxygen feed system."

Engineers have spent the past two months examining various scenarios to explain the cavitation occurring in the region near the engine cut-off sensors.

"Our team used computer models to simulate the flow in the liquid oxygen feedline between the bottom of the propellant tank and the engine cut-off sensors, approximately five feet downstream," said Mark Baldwin, Boeing's Delta propulsion analysis manager.

"The team enhanced its simulation models incrementally to include the more complex internal features of the liquid oxygen tank and feedline. Simulation runs have been completed with the higher fidelity models, resulting in an increasingly accurate simulation of the flow conditions experienced during the Heavy demonstration flight. These conditions correlated well with measurements taken by the sensors onboard the vehicle."

Boeing is examining options to fix the bubbling problem. Throughout this month, additional computer simulations are being performed to fully analyze the liquid oxygen flow between the bottom of the tanks and the engine cut-off sensors to assist in picking and verifying the corrective actions, the Air Force said.

"Boeing is evaluating future missions across its Delta 4 family of launch vehicles so that adequate margin for cavitation exists under the worst case conditions," a company spokesman said. "Cavitation margin adjustments, if required, can be made by changing the flight profile to throttle the RS-68 earlier, and can also be made by pressurizing the oxygen tank to a higher ullage pressure later in flight."

Boeing is scheduled to launch the GOES N civilian weather satellite from Cape Canaveral atop a Delta 4-Medium rocket on May 4. A Medium vehicle uses just one Common Booster Core -- a configuration that has flown three times without fault.

That will be followed by the first Delta 4 launch from Vandenberg Air Force Base in California, also flying in the Medium version. It is targeted for late August to loft a classified National Reconnaissance Office Payload.

The first operational Delta 4-Heavy with a real satellite payload is planned for late October when the 23rd and final Defense Support Program missile-warning spacecraft is launched directly into geostationary orbit.