Settled Science

Let's look at the crux of the problem, a few examples from "settled science" and then apply that to the settled science that is pilot procedures and technique.

1 — Examples of "settled science"

2 — The scientific method

3 — Faulty statistics

4 — Aviation example: stall recovery

5 — Aviation example: "Dive and drive"

6 — Aviation example: circling

7 — Aviation example: balanced field length versus runway available

8 — Aviation example: "travel not local in character"

9 — Putting an end to the 97% nonsense

10 — One last lesson in skepticism

Einstein's Static Universe

We have all heard that, according to Albert Einstein:

$$E=m{c}^2$$

Many of us also know what each of those constants mean. Basically, all the energy in the universe is equal to the mass of the universe times the speed of light squared. It appealed to Einstein's quest to find a nice, tidy explanation for everything. He wanted an elegant equation to explain an elegant and static universe. The formula itself is called the "Theory of Mass Energy Equivalence" and is a part of Einstein's 1915 "Theory of Special Relativity." So novice scientists (like me) call the formula itself the theory of special relativity. (Wrongly so.)

The problem was that his theory relied upon the universe being static, neither expanding or contracting. That meant there wasn't enough observable mass in the universe to make the math work out. So, in 1917, Einstein amended his theory to something like this:

$$E=m{c}^2+\mathrm{ "A\; fudge\; factor"}$$

The new equation wasn't as elegant but necessary to agree with the known science at the time. You could say that 97% of all the world's physicists were on board and it was settled science.

Then, in 1924, one of science’s greatest astronomers, Edwin Hubble, delivered a bombshell to science in general and astronomy in particular. Using a new 100 inch reflecting telescope on Mount Wilson, in California, Hubble announced that the universe is not defined by the galaxy in which we reside. The reality is that there are untold millions of sister galaxies distributed throughout a universe orders of magnitude larger than previously believed. So Einstein's static universe wasn't so static after all. So now, doing the math, that fudge factor was no longer needed and we are back to:

$$E=m{c}^2$$

Or are we? Now scientists are saying the discovery of "dark matter" means we need the fudge factor again. Or do we? The point here is that it was settled science until it wasn't. And no matter what you read today on the subject, you cannot rule out that something in the near or distant future will change everything we know about this again. It could be that E=mb², just as cartoonist Sydney Harris theorized so many years ago.

Martian canals

I grew up in the 1960's hearing fantastic stories about ancient canals on Mars, proof positive that there was at some point water on that planet. To this day I read now and then about how it was "global warming" that boiled off the water on Mars, a fate that awaits us to. More "settled science."

The canals were first “discovered” in 1877 by Italian astronomer Giovanni Schiaparelli. After other stargazers corroborated his claim, the canals became something of a phenomenon. Scientists drew detailed maps tracing their paths, and soon wild speculation began on their possible origins and use.

Percival Lowell, a mathematician and astronomer with worldwide cachet, concluded that the canals were a sophisticated irrigation system developed by an unknown intelligent species. His ideas were discredited by some scientists but the idea of canals on Mars were widely accepted. More settled science.

As better and better telescopes became available the new theory was that the canals are merely an optical illusion caused by streaks of dust blown over the surface. It wasn't proven one way or the other until the 1960s when unmanned spacecraft made flybys over Mars and took photos. So then it became settled science that there were no canals and therefore no water. But recent unmanned landings brought that into question too. Settled science unsettled.

Cold Fusion

When nuclear power plants first came on line, many science fiction geeks started to postulate about the day there would be a nuclear reactor in every house (at best) or that nuclear power would become so widespread that power companies would stop using meters to measure the output, since it would be so cheap to produce (at worst). But this never happened. Why? The conditions needed to create nuclear energy usually require extreme temperatures. What if we could do it at room temperature? What about "cold fusion?" If a nuclear reaction could be generated at room temperatures, then an abundance of energy could be created without the dangerous waste that results from nuclear power plants.

In 1989, chemists Martin Fleischmann and Stanley Pons published experimental results suggesting that they had achieved cold fusion in an experiment where an electric current was run through seawater and a metal called Palladium. Pons and Fleischmann became the darlings of the media and the scientific community. The term "cold fusion" became ubiquitous with the day and age where science had an answer for everything. Cold fusion was settled science, waiting for engineering to catch up with it.

But soon other scientists tried and failed to replicate the experiment. Fleischmann and Pons were accused not only of sloppy, unethical science, but were even said to have stretched the truth of their results. The settled science failed to achieve the acid test of repeatability. But to this day, there is still talk about the day engineers will finally be able to build something to achieve the promise this settled science has "proven."

So why is it that we, the non-scientific public in general and the news media is specific, always fall for the "settled science" ruse and seem too reluctant to give it up when the facts come in? I think it is because most of us are science-illiterate and want to establish our science bonafides by jumping aboard the latest science fad; but we don't understand that the scientific method means being wrong more often than being right. (And that is a part of the process.)

The purpose of science, when you get right down to it, is to explain the "how" and "why" of everything. When doing that, scientists will propose a hypothesis. The scientific method is used to collect measurable, empirical evidence in an experiment related to the hypothesis. The results of the experiments are used to support or contradict that hypothesis. A key point here is that no matter if the hypothesis is proven correct or false, the fine tuning of the hypothesis and testing continue.

Note that most hypothesis are proven wrong. It is easier to come up with an idea than it is to validate it. So why is it that we never hear of such things? Well, plainly speaking, negative results don't make the news.

A common rebuttal to the definition of the scientific method is that if "97% of all scientists agree, that is a consensus and therefore the science is settled."

To that I offer the common (but true) statement: "Figures lie and liars figure." This isn't to assign malicious intent, only to say very few of us really understand statistics.

Rather than dive into the math, let's simply posit this: the results of an accurate statistical poll rely more on who and what you ask than how you ask. You can get a 97% response to just about anything, depending on how you ask the question. Here is a made up example, just to illustrate the point further.

Let's say there is life on Mars and you are a Martian sent to Earth simply to observe from afar. Using your best surveillance cameras from low Earth orbit, you notice that there are males and females and that in general the males are larger. Your superiors back on Mars are considering an invasion and want to know the percentage of males versus females. They give you permission to land in a remote region of the most populous country of the world, and ask you to spy on a hospital newborn nursery for one day to report back on the percentage of male births versus female births.

You pick a small town in a remote region of China where the only hospital averages four births a day. You place your spy in a room adjacent to the nursery with a pad of paper. Your spy reports back 24 hours later that there were indeed four births that day, three males and one female. You report back to Mars that 75 percent of the population on Earth is male. The Martians are alarmed, recall their spies, and decide to cast their hopes of conquest on another planet. Earth is saved.

Okay, the faulty statistics are obvious here, but let's list them:

1. Sample size wrong.

You cannot learn valid statistics when the sample size is too small to capture a sense of the entire population. The Martian hypothetical, for example, was an accurate statistic for that particular day and that particular hospital.

2. Sample source wrong.

You cannot extrapolate a population's behavior on a smaller sampling, unless that smaller sampling is representative of the population being examined. Let's say that particular rural town in China still has the one child per couple rule and many couples chose to abort females. Your statistics could very well be valid for that entire town, but certainly not for the entire world.

3. Question wrong.

You cannot learn from the answers to a question that does not relate to the purpose of the statistical analysis in the first place. Our Martian warlords were worried about whom they would have to fight in an invasion. The percentage of newborn babies may not necessarily reflect those who achieve war fighting adulthood.

4. Assumptions wrong.

The framing of the question determines the answer and if you don't ask the right question the answer really doesn't matter. At the beginning of the space shuttle program, NASA’s own pre-launch estimates were that there was a 1 in 100,000 chance of shuttle failure for any given launch. The historical result was 2 in 135. A space shuttle pilot after those two shuttle failures said, "I don't care what the odds are but if it happens to me, it's 100% Just tell me what I need to know to handle it."

For the first 30 years of my career flying jet aircraft, there was only one accepted way to recover from a stall: full power and minimize altitude loss. It was settled science and to say otherwise was to risk a failure on your simulator checkride. For those of us with experience stalling actual jets (versus simulated jets), we knew this was a recipe for disaster. But as simulators took over fewer and fewer pilots had actual large aircraft stall experience. It was only a matter of time before the settled science killed someone.

How it used to be (old settled science)

In just about every simulator aircraft initial or recurrent program I've ever attended prior to 2009 the drill was the same:

1. Student briefs the procedure.
2. Student trims aircraft for level flight at a high altitude, brings thrust to idle (or other agreed upon setting), and maintains level flight. (Student is graded on altitude control.)
3. As aircraft stall warning activates, student applies maximum available thrust and maintains altitude. (Student is graded negatively for any altitude loss.)

Why the settled science was replaced

• 7 January 1994 — Lesson ignored

Case Study: United Express 6291

• 14 October 2004 — Lesson obscured

Case Study: Pinnacle Airlines 3701

• 12 February 2009 — Lesson learned (finally)

Case Study: Colgan Air 3407

How it is now (for now, anyway)

In late 2012, the FAA released AC 120-109 Stall and Stick Pusher Training which acknowledges "Reduction of AOA is the most important response when confronted with a stall event." It further outlines the following stall recover template:

1. Autopilot and autothrottle ... Disconnect

While maintaining the attitude of the airplane, disconnect the autopilot and autothrottle. Ensure the pitch attitude does not increase when disconnecting the autopilot. This may be very important in out-of-trim situations. Manual control is essential to recovery in all situations. Leaving the autopilot or autothrottle connected may result in inadvertent changes or adjustments that may not be easily recognized or appropriate, especially during high workload situations.

2. Nose down pitch control ... Apply until stall warning is eliminated
   Nose down pitch trim ... As Needed

Reducing the angle of attack is crucial for recovery. This will also address autopilot-induced excessive nose up trim. If the control column does not provide sufficient response, pitch trim may be necessary. However, excessive use of pitch trim may aggravate the condition, or may result in loss of control or high structural loads.

3. Bank ... Wings Level

This orients the lift vector for recovery.

4. Thrust ... As Needed

During a stall recovery, maximum thrust is not always needed. A stall can occur at high thrust or at idle thrust. Therefore, the thrust is to be adjusted accordingly during the recovery. For airplanes with engines installed below the wing, applying maximum thrust may create a strong nose-up pitching moment if airspeed is low. For airplanes with engines mounted above the wings, thrust application creates a helpful pitch-down tendency. For propeller-driven airplanes, thrust application increases the airflow around the wing, assisting in stall recovery.

5. Speed brakes/Spoilers ... Retract

This will improve lift and stall margin.

6. Return to the desired flight path.

Apply gentle action for recovery to avoid secondary stalls then return to desired flight path.

In 1979, the Air Force taught its pilots how to fly instruments at the same time they were learning basic stick and rudder, aerobatics, and formation. We tended to boil everything down into steps to keep everything straight. A "non-precision" instrument approach required we dive the airplane from the Final Approach Fix (FAF) altitude at a healthy rate (between 1,200 and 1,500 feet per minute), level off at the Minimum Descent Altitude (MDA), and drive along until we spotted the runway. This is the way it had been done from the very beginning of instrument flight and 97% of all pilots knew this to be the best way to do this. It was settled science.

How it used to be

This is from the instrument flight procedures manual I was issued as an Air Force second lieutenant pilot in 1979. The procedures were the law of the land for the next 20+ years. They were settled science.

Proof the settled science was wrong

Some of us began our instrument jet experience without the benefit of an autopilot or flight director and flying an approach without vertical guidance was an invitation to making fatal mistakes. Even with the best technology, flying an approach without vertical guidance can be a problem:

• 12 November 1995 — Lesson: "A non-precision approach must be flown with great precision."

Case Study: American Airlines 1572

• 19 October 2004 — "Just because you spot the runway doesn't mean it is time to descend."

Example: Corporate Airlines 5966.

• 24 November 2001 — "Just because you spot the ground doesn't mean you are on a visual approach."

Example: Crossair 3597.

How its done now

Continuous Descent Final Approach.

The aviation scientists who first came up with the circling approach were dealing with relatively slow aircraft with limited avionics and the need to get airplane on the ground with limited options. The circling approach was the best they could come up with so that's what we have been using since the dawn of powered flight. So how do you train pilots to fly these circling approaches in simulators? You require them to demonstrate circling approaches in the worst conditions, which will make the easier conditions, well, easier.

That's how we've always done it, it is settled science. I bet if you conducted a poll of FAA check airmen, you would discover that 97% of them agree. Settled science.

A personal story about how the training is unrealistic

I have been flying circling approaches into Memphis International Airport for a very long time. Until recently, all of this experience has been in the simulator. While the actual mechanics of the approach have changed over the years, the basic process is the same:

1. Fly the available non-precision approach on Runway 27, dive down to the MDA, look for the airport.
2. Once you spot the runway and are within the published circling approach radius, turn right 30° or 45° (your preference) and start timing. Once you've met your time, turn left to parallel the inbound course.
3. Identify land marks (which you've memorized) or look for the approach lights on Runway 18L and then 18R. When you reach a certain point, which depends on your aircraft category, your experience level, and your tolerance for this kind of thing, turn to Runway 18R and start descending.
4. Roll out below your stable approach criteria and attempt to put the airplane down in the touchdown zone. Don't worry about the stable approach thing, because the check airman knows a stable approach is virtually impossible for this set up.

Having done this, over and over again, I thought a real instrument approach to Runway 27 circle to 18L or 18R should be easy. I got my chance in 2016. It was a clear but moonless night. When we began the ILS to Runway 27 the winds were 240/10 and everything looked great. By the time we got to glide slope intercept, the winds had shifted to 090/15, beyond our tailwind limits. It was about midnight and we were already on tower's frequency. We told them we could no longer land on Runway 27 and were immediately cleared to circle for landing on Runway 18L. Should be easy, right? Without any moonlight, the area north of the airport is very dark indeed. I could not make out Runway 18L at all until we were on final. The only reason we ended up on final was the airplane's synthetic vision drew an extended centerline and the green airplane symbol on our Jeppesen approach chart showed us exactly where we were at all times. As hard as the circle was in the simulator at minimums, it would have been even more difficult in real life had it not been for all our electronic gizmos.

A. How it is done in the simulator during training and checkrides

Find an airport with enough runways at right angles to provide visual cues, drop the weather to minimums, evaluate the pilot's ability to keep the airplane at the MDA with a tolerance of 0 feet low 50 feet high while maintaining distance to the runway no greater than the published circling approach radius.

B. How it is done in the real world

Maneuver the airplane above the MDA to as high as normal traffic pattern altitude and as wide as needed to provide a stable approach on roll out.

C. How it should be done in the simulator during training and checkrides

See B. above.

Why?

• 15 April 2002 — Recognizing the problem

Case Study: Air China 129

• 28 July 2010 — Still a problem

Case Study: Air Blue ABQ 202

• 15 May 2017 — Some never get the word

Case Study: Learjet 35 N452DA

How do we fix this mess?

Looking at various circling approach accidents it is clear that the circling approach radius used to be too small under US TERPS but that has been largely fixed. But we still have problems. The circling to land maneuver is often confused with a circling approach at minimums because we train in our simulators to always circle at the lowest possible altitude while forbidding pilots to fly outside the published circling approach radius. But all of this for a maneuver even our check airman acknowledge is a visual maneuver.

My flight department forbids circling below 1,000 feet or with visibilities lower than 3 statute miles. Our simulator checks are always done at the published minimums because our training vendor is required by the FAA to do just that. The simulator, for many of us, has become a video game with "hacks." We learn the hacks at the limited number of locations approved for circling in simulators and that reinforces in our minds that circling is done at the MDA very tight to the airport. The one day when we are faced with a circling approach at minimums, we are likely to be surprised at just how difficult it can be.

One major airline fixed the problem by simply removing the circling approach from their list of approved maneuvers. This airline flies to a fixed number of airports and went through the process to have straight-in approaches added at airports where the winds normally required circling. I like this solution but it isn't possible for many of us corporate operators.

I think a possible solution is to continue the circle demonstration at KMEM, KJFK, or KIND to a missed approach and then at VFR minimums to emphasize the ability to maneuver so as to make the pattern as close to a normal visual approach as possible.

The settled science is hard to overcome here. As the accident investigation into Learjet 35 N452DA reveals, the FAA and the NTSB do not understand the root cause of circling approach accidents. Until that 97% of these aviation scientists become unsettled, this problem will continue to kill people and break things.

I've always thought that as an engineer I am more susceptible to this problem than most, but over the years I realize that isn't true. I've come to the conclusion that there are two kinds of pilots when it comes to figuring how much runway is needed for takeoff. First, you have the technically inclined that like to chase through the spaghetti charts, factor in all the extra variables, and come to an answer. For example: "Our balanced field length is 6,473 feet and the runway available is 7,000 feet, so we are good to go." The second type of pilot punches the numbers in, sees that one number is less than the other and says, "Yup, we are good to go." I think if you add everyone in the first group to everyone in the second ground you get 97% of all pilots. So what about that other 3 percent?

I think I am in that third group. We look at those neatly printed spaghetti charts and ultra-precise iPad applications and then think about the cuts on our tires, the faded look of the hydraulic pressure accumulators, and then think back about how our reaction times aren't as good as they used to be. Then we add a thousand feet to everything.

How 97% of all pilots do it

Let me first say, if you are in this 97% group it isn't your fault. "We" made you that way. The "we" is the FAA and all of those FAA approved training vendors. Here's how that happened. In the old days you spent hours going through charts to produce a number that predicted your required runway length to the nearest 100 feet. In a sure sign that nobody at the FAA knows how to use the spaghetti charts either, they now permit you to plug in the conditions into a computer and rely on the numbers it produces. So now, with your trusty iPad or FMS, you have your required runway to the nearest foot. Then you go to the simulator and practice. You are (1) expecting the engine failure, (2) ready to throw smoothness out the window and slam on those brakes, and (3) "flying" a simulator programmed to work like an airplane without years and years of abuse on its logbooks. So, in your subconscious, you believe you can make the numbers work.

What can go wrong

• 10 September 1980 — Sometimes the math doesn't add up

Case Study: Losing one at V₁

• 24 May 2008 — You cannot predict how you are going to react under pressure

Case Study: Kalitta 207

• 13 March 2014 — Simulator training is often negative training

Case Study: US Airways 1702

How it should be done

Let's say you need to abort a takeoff on a day things aren't going your way. Let's say the other pilot figures out you need to abort first but your SOP says you have to make the decision. So the SIC simply says "fire light, number two engine." Now you have to make the decision. Then you have to actually perform the abort. But you haven't actually done one of those in the airplane at high speeds and you end up not standing on the brakes the way the designers and your simulator instructor asked you to. Maybe you need more runway. Assuming the first officer utters the words "fire light, number two engine" precisely at V₁, here is how much more time you need:

• Your reaction time: 1 second
• Your action time: 1 second

And that is on a good day. (For how these times were determined, see: Abort Authority.) So let's say you were doing 130 knots at V₁ but accelerate further to 140 knots in the seconds it takes you to react and then act. That means you take another (2)(140)(nm/hr)(6076)(ft/nm)(3600)(sec/hr) = 472 feet before you actually start the abort.

But if you've never done this before, chances are your braking effort will not be optimal. And even if it is, there is a tendency to relax on the brakes just before the airplane comes to a stop. Why? See: V₁. I think you can safely add 500 more feet to your braking effort as a result.

My technique: add 1,000 feet to all takeoff and landing distances. The FAA would have you add 15% and that is better than nothing. But my answer is 1,000 feet and, yes, I have reduced fuel loads because of this. Oh yes, it gets better. I believe in using reduced (flex) thrust but I insist on at least a 2,000 foot buffer when doing so.

There was a lawyer somewhere in the bowels of the FAA charged with striking a compromise between safety and airline scheduling that came up with the idea that if a pilot commutes to work, that time should not count against the limited duty time and, conversely, should be considered a part of rest time. Hence the "not local in character" provision of 14 CFR 135, §135.263(c). I imagine 97% of all FAA examiners can quote you that CFR from memory.

Back in the days when an airline pilot's commute meant from the suburbs to the airport, all was good. But since deregulation we have legions of pilots crisscrossing the country in order to make their flight assignments. When the pilot has the judgment (and the cash) to check into a hotel, no problem. But when that same pilot spends "rest time" in row 14F of a turboprop regional airline or, worse yet, on the couch of the airline's pilot's lounge, you are courting with catastrophe. If you are one of those pilots you may be tempted to say you can handle it and it is a risk you are willing to take. But if my kids are sitting 50 feet behind you as you begin your instrument approach, it is a risk you should not be allowed to take.

How it is usually done

If you've never flown commercially allow me to translate. Transportation "not local in character" means during layovers away from your domicile. So transportation to your domicile is "local in character" and may be considered a part of a rest period. Have you ever sat next to an exhausted airline pilot in the back of a crowded airplane? You know, the one in the rumpled shirt trying to sleep? That sleep counts as an official (and legal) rest period. Some pilots will end up in the pilots lounge on a couch (if they are lucky) and that also counts as a rest period.

What can go wrong?

• 18 August 1993 — The captain had been awake for 23.5 hours in the last 28 hours . . .

Case Study: American International Airways 808.

• 26 July 2002 — Dozing in the company sleep rooms for an hour and a half . . .

Case Study: Federal Express 1478.

• 12 February 2009 — Sleeping in the pilot's lounge . . .

Colgan Air 3407

How it should be done

If I were king, crews who don't live where they are domiciled should have to prove they were in the local area in a suitable location with suitable rest facilities during the rest period prior to flight. If they can't afford to do that, they should find another job. Airlines that don't pay enough to entice pilots to move locally should be required to provide suitable rest facilities; that means a good hotel.

So we have a problem. Because we've always done things a certain way we get used to the way it has always been done and are loathe to change. Others like us, 97% of us, believe "everyone" agrees and feel that it isn't our duty to blaze a new trail. Besides, the minute we do something else, someone is going to say, "Hey! 97% of all pilots agree it should be done this way, so you must be wrong!"

Here are a few pointers to help you avoid this group think.

1. Remember that there is no such think as settled science.

Just because E=mc² today doesn't mean it will tomorrow. We are constantly finding better mouse traps in aviation. And that isn't just the hardware and installed software. It is also basic pilot procedures.

2. Remember that the 97% is true of only a specific population.

Just because you got 97% of a group of people to respond in a certain way doesn't mean they are representative of the population that matters or that they were asked the correct question to begin with. If you've done nothing but fly turbo props for a living don't tell me how to recover from a high altitude stall; and I won't tell you how to deal with a prop that won't feather.

3. Remember that the seasoned simulator instructor is only as good as his or her syllabus.

Just because the FAA approves the training program doesn't mean the training program can't be improved. Take everything thrown your way with a grain of salt and be skeptical.

4. Remember the way we've always done it is often because we didn't know better.

There are many techniques that used to be gospel that are now forbidden. Failing to "dive and drive" could bust you a checkride thirty years ago. These days, the dive and drive is more often than not the wrong thing to do. Where we used to teach the "duck under," those who still practice that ancient art are unlikely to admit it.

I like to think a good scientist, engineer, and pilot will have at least one thing in common: a skeptical mind. A scientist needs skepticism to continue learning the truth. The engineer needs skepticism to truly investigate the best way to turn those ideas into reality. And a pilot needs skepticism because there are so many things out there ready to prevent the pilot's prime purpose: flying safely.

I offer this final story of true skepticism.

As the American frontier was finally opened up to the rest of the world with the advent of the transcontinental railroad, scientists from Europe started coming over to expand their knowledge of the new world. Three such scientists were aboard a train which came to a stop at a watering station. As they were stopped, the scientists spotted something they had never seen before, a donkey tied to a hitching post. The donkey stood, parallel to the train so that only its left side was visible.

"What a strange looking horse," the first scientist said.

"That's not a horse," the train attendant said while passing through the cabin. "That's a donkey."

"Ah!" the three scientists said together. "We must record our observations."

"Donkeys have four legs and a tail," the first scientist said. "And they are brown in color."

"I wouldn't go so far," the second scientist said. "You are coming to conclusions the data does not support. I would say that donkeys have four legs and a tail, yes that is true. But as far as we know, only the left side of the donkey is brown."

"You too have jumped to a conclusion," the third scientist says. "I would agree only to say, at least one donkey in Iowa has four legs, a tail, and a left side that is brown. For all the others, I will have to wait for more evidence."

AC 120-109 Stall and Stick Pusher Training, 8/6/12, U.S. Department of Transportation

Air Force Manual (AFM) 51-37, Instrument Flying, 1 December 1976

United States Standard for Terminal Instrument Procedures (TERPS), Federal Aviation Administration 8260.3C, 03/14/2016