The school of philosophy called Antipodianism briefly flourished on the fringes of the Hellenistic world more than 2,000 years ago. The sect held that every person has an opposite number, a mirror image who inverts all our beliefs, feelings, actions and attitudes. If I smile, my antipodian counterpart frowns; when I wake, she sleeps. If I’m a Mac, she’s a PC. For every liberal Democrat there’s an antipodian Tea Party Republican. In this way the universe is held in balance. It’s an enforced equilibrium, which none of us has the power to upend, hard as we might try.

The earliest of the Antipodians believed that every such matched pair (commonly designated A and ∀) live at diametrically opposite points on the surface of the earth. This arrangement ensures that A and ∀ can never meet—thereby averting a cosmic catastrophe. A later quantum-field version of Antipodianism relaxed the geographic constraint by allowing for the creation and annihilation of A∀ pairs, but that idea never really caught on.

One day an Antipodian master was teaching an exchange student from New Zealand. The child was crafty.

“Is it not true,” she asked, “that A and ∀ always do the opposite thing?”

“Yes, antisymmetry demands it,” the master replied.

“If A walks north, ∀ must walk south?” the child asked.

Again the master assented.

“If A goes east, ∀ must go west?”

“Yes.”

“If A turns to the right, ∀ must turn to the left, no?”

The master agreed, although he sensed trouble coming.

“I’m afraid the universe is out of joint,” said the child. “If A goes north and turns to the right, while ∀ goes south and turns to the left, afterwards they are both walking east. They are doing the same thing.”

Needless to say, this was a moment of crisis in Antipodian doctrine. The Pythagoreans, you may recall, resolved a similar impasse by resorting to violence. When some upstart challenged their precept that “all is number” by showing that no known number can be the square root of 2, the Pythagoreans tossed the troublemaker out of the boat. But in this case the Antipodian master kept his calm.

“Ah my little Kiwi,” he said to the student. “You are clever but not wise. Your own statements refute your claim. Did you not begin by saying that A and ∀ always do the opposite thing? When A walks 10 paces north, ∀ walks 10 paces south. When A turns right, ∀ turns left. But now you would have us believe they both take a step forward, contradicting the most basic law of their nature. What really happens is that A walks forward and ∀ walks backward. Thus A goes eastward and ∀ westward, and all is well with the world.”

Through this brittle sophistry the master extricated himself from the classroom—though he may have had to walk backwards to make good his escape. He never taught again. The Kiwi student went on to a brilliant career studying the weak interactions of neutral K mesons. As for Antipodianism, it vanished without a trace.

Or maybe it left a tiny trace. I’ve never visited the antipodes, but I hear that corkscrews Down Under turn the other way.

The printed program for the recent STOC meeting in Cambridge announced the following talk:

Unfortunately, the fourth author could not be present (and he is not listed as an author on the paper itself), so the talk was given by Gregory Valiant.

Here is a motivating example. Take a tape measure, and go record the heights of a few thousand adults chosen at random. You’ll come back with a distribution that looks something like this:

How come the curve is so lumpy and lopsided? Isn’t height a variable with an approximately normal distribution? Of course it is. The problem is that we have mixed up two subpopulations—men and women—with different height distributions:

Question: Given the lumpy combined distribution, and the knowledge that it represents a mixture of exactly two normal distributions, can we somehow recover the component distributions? Specifically, can we determine the means and standard deviations and the Gaussian curves, as well as their relative weights, or contributions to the total?

The answer is yes. In a series of papers published in the 1950s and 60s, Henry Teicher (then at Purdue, now emeritus at Rutgers) proved a kind of unique-factorization theorem for distributions. He showed that no mixed distribution can be decomposed into two or more different sets of normal distributions, just as no composite whole number can be formed as the product of two or more distinct sets of primes. (Teicher’s theorem generalizes to many distributions other than normal ones. There are also a few caveats; for example, none of the component distributions can be pointlike, with a standard deviation of zero.) Teicher’s work implies that a mixed distribution is identifiable: If you can break it down into a set of primitive distributions, then that decomposition is unique.

If STOC were a mathematics meeting, that might be the end of the story. But STOC is the Symposium on the Theory of Computing, and the question here is not “Does a solution exist?” but rather “Can you solve it in polynomial time?” Teicher’s result offers no such guarantee. It turns out that his proof of identifiability depends on the behavior of the distributions far out in the tails. Measuring data frequencies in these sparsely populated outlying regions requires an exponentially large number of samples. But other approaches to the problem don’t run into this snag; Valiant and his colleagues show that “robust polynomial identifiability” is indeed possible.

A key idea in their proof goes back at least as far as the 1890s, to work done by the indefatigable statisticians W. F. R. Weldon and Karl Pearson. In 1892 Weldon spent a summer on the Bay of Naples, measuring various features on the carapaces of crabs. One such feature yielded a distinctively lumpy distribution much like the height curve shown above. Weldon thought that the asymmetry might signal the incipient splitting of the crab population into two races or species, each of which if taken individually would have a normal distribution. For help with the analysis of his data he turned to Pearson, who was able to identify two component Gaussian curves that sum up to the observed distribution. (The figure comes from Weldon’s 1893 paper; see below for references.)

Pearson’s method was based on calculating the first six statistical moments of the given distribution. (The nth moment of a distribution is the expectation value of \((x-\bar{x})^n\), where \(x\) is a random value drawn from the distribution and \(\bar{x}\) is the mean value.) The process called for solving a ninth-degree polynomial, which was a heroic feat in 1893.

Valiant et al. show that a procedure like Pearson’s will always succeed in the following sense: If two mixed distributions can be distinguished at all, then they can be distinguished by examining their first six moments, and those moments characterize the component Gaussians. Calculating the moments requires only a polynomial number of samples, and the running time of the overall algorithm is a polynomial function of various parameters such as the required accuracy. Moreover, the process can be extended from one-dimensional distributions to multidimensional Gaussians.

Although the algorithm has polynomial running time, that’s not a guarantee of practicality. (After all, this is a theory conference.) One stage of the process is essentially a brute-force search through a very large (though polynomially bounded) space of parameter values for the six moments.

Sources:

The STOC proceedings with the Kalai-Moitra-Valiant paper are now available online, but only by subscription. A PDF preprint is posted on Valiant’s web site; also an expanded version.

For Henry Teicher’s work see: “Identifiability of Mixtures,” The Annals of Mathematical Statistics 32(1):244–248 (1961) and “On the Mixture of Distributions,”  The Annals of Mathematical Statistics 31(1):55–73 (1960).

For Weldon and Pearson see: W. F. R. Weldon: “On Certain Correlated Variations in Carcinus maenas,“ Proceedings of the Royal Society of London 54:318–329 (1893) and Karl Pearson: “Contributions to the Mathematical Theory of Evolution,” Philosophical Transactions of the Royal Society of London A 185:71- 110 (1894).

My thanks to Virginia Gold and Irene Frawley at ACM, Lance Fortnow, Paul Oka, Jennifer Chayes and Christian Borgs, all of whom helped make it possible for me to attend some of the STOC sessions.

The snarXiv is a ran­dom high-energy the­ory paper gen­er­a­tor incor­po­rat­ing all the lat­est trends, entropic rea­son­ing, and excit­ing mod­uli spaces. The arXiv is sim­i­lar, but occa­sion­ally less ran­dom.

Inspiring! Soon bit-player, too, will be generated by a context-free grammar, and no one will know the difference.

The explosion and fire that destroyed the drilling rig Deepwater Horizon on the night of April 20 was a run-of-the-mill industrial accident. In saying this, I don’t mean to make light of the disaster, in which 11 workers perished. Nevertheless, it’s important to recognize that events like this one have happened before. They happen all the time. Just two weeks earlier, an explosion in a West Virginia coal mine killed 29 workers. A refinery fire in Anacortes, Washington, killed seven on April 2. In February six workers died in a natural gas explosion at a power plant under construction in Connecticut. In January, leaking phosgene gas killed an operator at a West Virginia chemical plant. Looking back a few years further, a 2005 fire at a BP refinery in Texas City, Texas, killed 15 workers and injured 170 more. There was another fatality at the same refinery in 2008. Texas City was also the site of a fertilizer explosion in 1947 that destroyed much of the port and left almost 600 dead. Then there’s the Bhopal catastrophe, where toxic fumes from an insecticide plant suffocated thousands of residents of nearby neighborhoods.

Still another accident that belongs in this sad catalogue is the destruction of the Piper Alpha platform in the North Sea oil fields. Piper Alpha was not a drilling rig but a production platform; it pumped oil and natural gas from completed wells to a terminal in the Orkney Islands. On the night of July 6, 1988, a huge explosion shattered the main part of the platform, and a subsequent fire consumed the rest; 167 crew members died. The apparent cause was a miscommunication: The night shift tried to start up a pump, not knowing that the day shift had removed a crucial valve for maintenance, leading to a massive leak of flammable gas.

It’s a bitter truth: Industrial accidents are business as usual. Death on the job is a cost that we (as a society) are evidently willing to accept and pay. When a mine collapses or a sugar refinery explodes, we read the story in the newspaper and we watch the film at 11, and then we grit our teeth and move on. The Deepwater Horizon sinking is just another in this long series of mishaps.

And yet it’s also totally different. This accident has consequences that reach beyond the workers and their families. A hole in the bottom of the ocean has been spewing hydrocarbons for six weeks. The rate of loss could well be a million gallons a day. Efforts to control the well are still under way, but if they fail the spill might go on for another three months. All that makes for a nightmare that doesn’t drop out of the news cycle after the first 24 hours.

In this respect, Deepwater Horizon may turn out to be not another Piper Alpha but another Three Mile Island. The 1979 accident at that nuclear power station had lasting consequences: We haven’t built another nuclear plant in the U.S. in the 30 years since. (There are other reasons for the long lassitude of the nuclear industry, but TMI was a major factor.) Likewise, in the aftermath of Deepwater Horizon, it seems a fair guess that we won’t be drilling another deep offshore well anywhere near the U.S. coastline for years to come, and maybe decades. Maybe never.

A moratorium on offshore drilling, enforced by economic as well as legal and political pressure, looks like a perfectly sensible response to the current situation. But we deserve more. By all means, let’s do whatever necessary to avoid repeating this particular mistake, but at the same time let’s address the broader issue of why things all over the industrial landscape keep blowing up and falling down.

Will we ever learn what happened on the Deepwater Horizon? There are certainly a lot of investigators trying to find out. Three Congressional committees have heard testimony already. A board of inquiry formed by the Coast Guard and the Minerals Management Service is holding its own series of hearings. So is a state panel in Louisiana. On May 11 President Obama asked the National Academy of Engineering to look into the cause of the accident, then on May 22 he appointed a special commission to carry out yet another investigation. Meanwhile the Justice Department is studying possible civil or criminal penalties.

For all that effort, we sure haven’t learned much yet. The well is leaking, but BP isn’t. It’s amazing how they’ve managed to keep the public and the press at a distance; the one art they seem to have mastered is secrecy. Even friendly trade publications (e.g., Oil and Gas Journal, Offshore) have so far failed to penetrate the security cordon. In the end, though, the story will come out. We’re going to learn why that blowout preventer didn’t prevent a blowout. But I’m not so confident we’ll learn how to prevent the next blowout.

The only blowout preventer I’ve ever seen up close and personal was on a drilling rig near Oklahoma City in 2004. At the time, I thought it was quite a brawny-looking piece of gear, with all those rings of torqued bolts clamping the flanges together, and the big red hydraulic rams like pincers clasping the pipe. Apparently, this unit is puny compared with the BOP stack installed on the Deepwater Horizon well (“five stories tall” in news accounts). Still, the principle of operation is the same: If the well “kicks”—meaning that gas and oil begin to push their way toward the surface—the rams close off the well and keep everything sealed tight. Some rams are meant to choke off the annular space around the drill pipe; some are “blind rams” used when no drill pipe is present in the bore; the last resort is a pipe ram or shear ram meant to crush or cut off the drill pipe.

It sounds foolproof, but evidently not. Controls can stick, valves can leak, parts can break. I’ve read speculation that the shear ram on the BOP beneath the Deepwater Horizon might have failed because it happened to strike a thicker-walled section of drill pipe where two lengths of tubing are threaded together; I really hope that’s not a serious concern, because it would mean that every BOP on the planet has a failure probability of 3 percent. (Drill pipe comes in 30-foot lengths, and the thickened sections are about a foot long.)

A 1997 article in Offshore lists a fascinating variety of other failure modes for deep-water wells. The article is by Larry H. Flak of Boots and Coots—a company often called in to deal with crises like this one. Flak is writing for an audience of oil-patch insiders, and I don’t follow all of his jargon. For example, there’s this wonderfully opaque passage:

Broached blowouts could happen with casing failure. Recently, an ultra-deepwater operator swabbed in a kick resulting in over 9,000 psi on the subsea BOPs. Fortunately, the casing was sound and set just on top of the sand. This allowed safe kick bullheading.

Even for those of us who have never bullheaded or swabbed in a kick, the message comes through: A lot can go wrong. A well is not just a hole in the ground; it’s a fairly complicated structure with multiple concentric layers of casing and drill pipe, which create several interconnected annular spaces; fluids under pressure can find many pathways to the surface. Flak points out that even the massive steel blades of a hydraulic ram can be torn to pieces by high-velocity streams of abrasive fluids, such as drilling mud. He concludes: “Blowout control options in ultra-deepwater are very limited. Blowout prevention is of paramount importance.”

When the story of the Deepwater Horizon accident is finally told, we’re going to hear at least two interpretations. In one telling, BP and its contractors were incompetent or negligent or criminally greedy; they screwed up. But the basic technology of offshore drilling is sound, and if only the companies had followed established industry practices and common sense, none of this mess would have happened. In the other version, BP and its contractors were incompetent or negligent or criminally greedy; they screwed up. But even if they had followed approved industry practices, a disaster like this was waiting to happen, because the technology of offshore drilling is fatally flawed. I really wish I knew which of these stories to believe. I worry that we lack the institutional means to decide between them.

•     •     •

I’m a great admirer of the National Transportation Safety Board. When an airliner crashes or a couple of commuter trains collide, the NTSB mounts a focused, scientific effort to understand the cause of the accident. Investigating accidents is all they do, and this turns out to be a key to their success; they are insulated from the responsibilities and temptations of regulating the industry, adjudicating legal culpability, assessing penalties or even enforcing their own safety recommendations. Separating the investigative and regulatory functions has worked brilliantly, and the board’s findings command wide respect. But the NTSB won’t be investigating the Deepwater Horizon blowout, because the event doesn’t fall within their statutory purview; it’s not a transportation accident. There’s also a Chemical Safety and Hazard Investigation Board (CSB), which has had a similar role in the chemicals industry since 1998. You might think that petroleum would count as a chemical substance, but as far as I can tell the CSB will also be sitting this one out. Their mission statement says: “The CSB conducts root cause investigations of chemical accidents at fixed industrial facilities.” The Deepwater Horizon wasn’t “fixed.”

So here’s my call for action: We need a single agency, modeled on the NTSB and the CSB, with authority to investigate all accidents involving industry or infrastructure—everything from well and refinery fires to water-main breaks, power blackouts, mine cave-ins, bridge and dam failures, boiler explosions and sewage spills. I believe we’re under a moral imperative to learn all we can from every such accident, in the hope of preventing a recurrence. A permanent, independent agency with investigative authority, in-house expertise and technical resources looks to me like the best way to reach this goal.

After the 2005 refinery fire in Texas City, BP hired James Baker III (the former Secretray of State) to convene a special of inquiry into the company’s safety practices and culture. The Baker panel report observed that BP had an admirable record on personal safety but cited deficiencies in process safety. Personal safety says: Wear your hardhat, your eye protection and your steel-toed boots when you go out in the plant to open a valve. Process safety says: Make sure you open the right valve, not the one that’s going to create a fireball that engulfs the whole plant. The report remarks: “BP mistakenly interpreted improving personal injury rates as an indication of acceptable process safety performance.” On April 20, BP executives flew out to the Deepwater Horizon to celebrate a personal safety milestone on the rig: no loss-of-work accidents in seven years. The executives were still on board when somebody turned the wrong valve.

Note: I have posted a PDF of the oil and gas chapter from my book Infrastructure: A Field Guide to the Industrial Landscape on the book’s web site.

Update 2010-06-21: A superb report on the failure modes of blowout preventers was published this morning in The New York Times. The article is by David Barstow, Laura Dodd, James Glanz,  Stephanie Saul and Ian Urbina. Nothing they say is reassuring.