Be Skeptical of Both Piketty and His Skeptics

By Nate Silver, FiveThirtyEight

27 May 14

ata never has a virgin birth. It can be tempting to assume that the information contained in a spreadsheet or a database is pure or clean or beyond reproach. But this is almost never the case. All data is collected and compiled by someone — either an individual researcher or a government agency or a scientific laboratory or a news organization or someone or something else. Sometimes, the data collection process is automated or programmatic. But that automation process is initiated by human beings who write code or programs or algorithms; those programs can have bugs, which will be faithfully replicated by the computers.

This is another way of saying that almost all data is subject to human error. It’s important both to reduce the error rate and to develop methods that are more robust to the presence of error. And it’s important to keep expectations in check when a controversy like the one surrounding the French economist Thomas Piketty arises.

Piketty’s 696-page book “Capital in the Twenty-First Century” has become an unlikely best-seller in the United States. That’s perhaps because it was published at a time when there is rapidly increasing interest in the subject of economic inequality in the U.S. But on Friday, the Financial Times’ Chris Giles published a list of apparent errors and methodological questions in the data underpinning Piketty’s work. Piketty has so far responded to the Financial Times only in general terms.

My goal here is not to litigate the individual claims made by Giles; see The New York Times’ Neil Irwin or The Economist’s Ryan Avent for more detail on that. Rather, I hope to provide some broad perspective about data collection, publication and analysis. A series of disclosures: First, my economic priors and preferences are closer to The Economist’s than to Piketty’s. Second, I haven’t finished Piketty’s book, although I’ve spent some time exploring his data. Third, I’m no expert on macroeconomic policy or macroeconomic data. Fourth, this comment rather liberally takes advantage of our footnote system; there’s a short version (sans footnotes) and a long version (avec).

My perspective is that of someone who has spent a lot of time compiling and analyzing moderately complex data sets of different kinds. Also, I’m someone who, like Piketty, has seen his public profile grow unexpectedly in recent years. I consider myself extremely fortunate for this — however, I know that attention can sometimes yield disproportionate praise and criticism. Throat-clearing aside, here’s what I have to offer.

Piketty’s data sets are very detailed, and they aggregate data from many original sources. For instance, the data Piketty and the economist Gabriel Zucman compiled on wealth inequality in the United Kingdom for their paper “Capital is Back: Wealth-Income Ratios in Rich Countries, 1700-2010? contains about 220 data series for the U.K. alone which are hard-coded into their spreadsheet. These data series are compiled from a wide array of original sources, which are reasonably well documented in the spreadsheet.

This type of data-collection exercise — many different data series over many different years, compiled from many countries and many sources — offers many opportunities for error. Part of the reason Piketty’s efforts are potentially valuable is because data on wealth inequality is lacking. But that also means his numbers will not have received as much scrutiny as other data sets.

An extreme contrast would be to something like Major League Baseball statistics, almost every detail of which have been scrubbed and scrutinized by enthusiasts for decades. Even so, they contain errors from time to time. There are, however, usually larger gains to be had when data or methods or findings are relatively new — as they are in Piketty’s case. (An analogy is the way a vacuum’s first sweep of the living-room floor picks up a lot more dust and dirt than the second and third attempts.) Perhaps Piketty is guilty of coming to some fairly grand conclusions based on data that has not yet received all that much scrutiny.

What error rate is acceptable? The right answer is probably not “zero.” If researchers kept scrubbing data until it were perfect, they’d never have time for analysis. There comes a point of diminishing returns; that Hack Wilson had 191 RBIs during the 1930 season rather than 190 ought not have a material impact on any analysis of baseball player performance. At other times, entire articles or analyses or theories or paradigms are developed on the basis of deeply flawed data.

I don’t know where Piketty sits on this spectrum. However, I think Giles (and some of the commentary surrounding his work) could do a better job of describing Piketty’s error rate relative to the overall volume of data that was examined. If Giles scrutinized all of Piketty’s data and found a handful of errors, that would be very different from taking a small subsample of that data and finding it rife with mistakes.

All of this is part of the peer-review process. Academics sometimes think of peer review as a relatively specific activity undertaken by other academics before academic papers or journal articles are published. This process of peer review has been much studied over the years (often in peer-reviewed articles, naturally), and scholars have come to different conclusions about how effective it is in avoiding various types of errors in published research.

I’m not necessarily opposed to this type of peer review. But I think it defines peer review too narrowly and confines it too much to the academy. Peer review, to my mind, should be thought of as a continuous process: It starts from the moment a researcher first describes her result to a colleague over coffee and it never ends, even after her work has been published in a peer-reviewed journal (or a best-selling book). Many findings are contradicted or even retracted years after being published, and replication rates for peer-reviewed academic studies across a variety of disciplines are disturbingly low.

I have a dog in this fight, obviously. I think journalistic organizations from the Financial Times to FiveThirtyEight should be thought of as prospective participants in the peer-review process, meaning both that we provide peer review and that our work is subject to peer review.

I can’t speak for the FT, but I know that FiveThirtyEight gets some things badly wrong from time to time. It’s helpful to have readers who hold us to a very high standard. (A terrific question is whether FiveThirtyEight and other news organizations are transparent enough about their research to be full-fledged participants in the peer-review process. That’s something I should probably address more completely in a separate post, but see the footnotes for some discussion about it.)

Piketty’s errors would not have been detected so soon had he not published his data in detail. That’s not to say that transparency is an absolute defense. But one should also assume that there are as many problems (probably more) with unpublished data, or poorly explained methods.

The peer-review process ideally involves both exactly replicating a research finding and replicating it in principle. It would be problematic if other researchers couldn’t duplicate Piketty’s data. But it would be at least as problematic — I’d argue more so — if they could replicate it but found that Piketty’s conclusions were not very robust to changes in assumptions or data sources.

Some of Giles’s critique of Piketty gets at this problem. For instance, he calls into question Piketty’s finding that wealth inequality is rising throughout Western Europe, a result which he says depends on a particular series of assumptions and choices that Piketty made.

Of course, Giles’s methodological choices can be scrutinized, too. Perhaps there’s some reasonable set of assumptions under which wealth inequality is not rising at all in Western Europe, another under which it’s increasing modestly, and a third under which it’s increasing substantially.

In the medium term, the better test might be one of research that’s built up from scratch and largely independently of both Piketty and Giles. How robust are their findings to reasonable changes in data and assumptions?

And in the long run, the best test might be whether Piketty’s hypothesis makes a good prediction about wealth inequality, i.e. whether wealth inequality continues to rise. The prediction won’t be as easy to evaluate as election forecasts are. Still, Piketty’s book comes closer to making a testable prediction than much other macroeconomic work.

Science is messy, and the social sciences are messier than the hard sciences. Research findings based on relatively new and novel data sets (like Piketty’s) are subject to one set of problems — the data itself will have been less well scrutinized and is more likely to contain errors, small and large. Research on well-worn datasets are subject to another. Such data is probably in better shape, but if researchers are coming to some new and novel conclusions from it, that may reflect some flaw in their interpretation or analysis.

The closest thing to a solution is to remain appropriately skeptical, perhaps especially when the research finding is agreeable to you. A lot of apparently damning critiques prove to be less so when you assume from the start that data analysis and empirical research, like other forms of intellectual endeavor, are not free from human error. Nonetheless, once the dust settles, it seems likely that both Piketty and Giles will have moved us toward an improved understanding of wealth inequality and its implications.

Let Them Eat Carbon

By Michael T. Klare, TomDispatch

27 May 14

n the 1980s, encountering regulatory restrictions and public resistance to smoking in the United States, the giant tobacco companies came up with a particularly effective strategy for sustaining their profit levels: sell more cigarettes in the developing world, where demand was strong and anti-tobacco regulation weak or nonexistent. Now, the giant energy companies are taking a page from Big Tobacco’s playbook. As concern over climate change begins to lower the demand for fossil fuels in the United States and Europe, they are accelerating their sales to developing nations, where demand is strong and climate-control measures weak or nonexistent. That this will produce a colossal increase in climate-altering carbon emissions troubles them no more than the global spurt in smoking-related illnesses troubled the tobacco companies.

The tobacco industry’s shift from rich, developed nations to low- and middle-income countries has been well documented. “With tobacco use declining in wealthier countries, tobacco companies are spending tens of billions of dollars a year on advertising, marketing, and sponsorship, much of it to increase sales in... developing countries,” the New York Times noted in a 2008 editorial. To boost their sales, outfits like Philip Morris International and British American Tobacco also brought their legal and financial clout to bear to block the implementation of anti-smoking regulations in such places. “They’re using litigation to threaten low- and middle-income countries,” Dr. Douglas Bettcher, head of the Tobacco Free Initiative of the World Health Organization (WHO), told the Times.

The fossil fuel companies -- producers of oil, coal, and natural gas -- are similarly expanding their operations in low- and middle-income countries where ensuring the growth of energy supplies is considered more critical than preventing climate catastrophe. “There is a clear long-run shift in energy growth from the OECD [Organization for Economic Cooperation and Development, the club of rich nations] to the non-OECD,” oil giant BP noted in its Energy Outlook report for 2014. “Virtually all (95%) of the projected growth [in energy consumption] is in the non-OECD,” it added, using the polite new term for what used to be called the Third World.

As in the case of cigarette sales, the stepped-up delivery of fossil fuels to developing countries is doubly harmful. Their targeting by Big Tobacco has produced a sharp rise in smoking-related illnesses among the poor in places where health systems are particularly ill equipped for those in need. “If current trends continue,” the WHO reported in 2011, “by 2030 tobacco will kill more than 8 million people worldwide each year, with 80% of these premature deaths among people living in low- and middle-income countries.” In a similar fashion, an increase in carbon sales to such nations will help produce more intense storms and longer, more devastating droughts in places that are least prepared to withstand or cope with climate change’s perils.

The energy industry’s growing emphasis on sales to these particularly vulnerable lands is evident in the strategic planning of ExxonMobil, the largest privately owned oil company. “By 2040, the world’s population is projected to grow to approximately 8.8 billion people,” Exxon noted in its 2013 financial report to stockholders. “As economies and populations grow, and living standards improve for billions of people, the need for energy will continue to rise... This demand increase is expected to be concentrated in developing countries.”

This assessment, explained Exxon CEO Rex Tillerson, will govern the company’s marketing plans in the years ahead. “The global business environment continues to provide a mix of challenges and opportunities,” he told financial analysts at the New York Stock Exchange in March 2013. While the demand for energy in the developed economies “remains relatively flat,” he noted, “energy demand for the economies of the non-OECD countries is expected to grow about 65% to support anticipated growth.”

In recognition of this trend, Exxon has undertaken a wide variety of initiatives intended to boost its sales capacity in China, Southeast Asia, and other rapidly developing areas. In Singapore, for example, the company is expanding a refinery and petrochemical facility that make up its “largest integrated manufacturing site in the world.” The refinery is being modified to produce more diesel, so as to better service the growing fleets of trucks, buses, and other heavy vehicles in the region. Meanwhile, the hydrocarbon processing facility at the chemical plant is being doubled to meet the rising demand for petrochemicals used in making plastics and other consumer goods, especially in China. (“China alone is expected to represent over half of global demand growth” for these products, Tillerson observed last year.)

To promote its products in China, Exxon has established a “strategic alliance” with the China Petroleum and Chemical Corporation (Sinopec), one of China’s state-owned energy giants. A key goal of the alliance is the establishment of an “integrated world-scale refinery and petrochemical complex” in eastern China which, Exxon officials noted, is to “become a major marketer of petrochemicals throughout China and petroleum products throughout Fujian Province.” A major component of this joint effort, the Fujian Refining and Ethylene Integrated Project, came on line in September 2009.

Exxon is also expanding its capacity to supply liquefied natural gas (LNG) to Asia. In partnership with Qatar Petroleum, it has built the world’s largest LNG export facility at Ras Laffan in Qatar and is building a mammoth LNG operation in Papua New Guinea. This $19 billion project, which began operation in April, includes a 430-mile pipeline to deliver gas from the island’s interior highlands to an export terminal near Port Moresby, the capital. “The project is optimally located to serve growing Asia markets where LNG demand is expected to rise by approximately 165% between 2010 and 2025,” said Neil W. Duffin, president of ExxonMobil Development Company.

Next on the company’s agenda is a plan to draw on the natural gas being extracted in ever greater quantities from domestic shale formations in the United States via hydro-fracking and convert it into LNG for export to Asia. Although various American politicians have been pushing the strategic export of such supplies to Europe to “rescue” that continent from its reliance on Russian gas, Exxon has other ideas. It sees Asia, where gas prices are higher, as the natural market for its LNG -- and U.S. foreign policy be damned. “By exporting natural gas,” Tillerson told the Asia Society in June 2013, “the United States could shore up the energy security of Asian allies and trading partners and stimulate investment in American domestic production.”

Big Energy’s “Humanitarian” Mission

In promoting such policies, Exxon’s executives are careful to acknowledge that growing concerns over climate change are generating increased resistance to fossil fuel consumption in Europe and other First World areas. When it comes to the rest of the planet, however, such concerns, they claim, should be outweighed by a “humanitarian” impulse to provide cheap fossil energy to poor people. Drawing on the arguments of Danish environmental renegade Bjørn Lomborg, author of The Skeptical Environmentalist, they argue that tending to the needs of the poor constitutes a greater priority than curbing global warming. “We must also recognize that there is a humanitarian imperative to meeting these growing global energy needs,” Tillerson typically asserted in 2013.

Asked why global warming shouldn’t be of greater concern, the Exxon CEO parroted Lomberg’s anti-environmental perspective. “I think there are much more pressing priorities that we... need to deal with,” Tillerson told the Council on Foreign Relations in June 2012. “There are still hundreds of millions, billions of people living in abject poverty around the world. They need electricity... They need fuel to cook their food on that's not animal dung... They'd love to burn fossil fuels because their quality of life would rise immeasurably, and their quality of health and the health of their children and their future would rise immeasurably. You'd save millions upon millions of lives by making fossil fuels more available to a lot of the part of the world that doesn't have it.”

Although the leaders of the other giant energy firms, including BP, Chevron, and Royal Dutch Shell, are less outspoken than Tillerson, they are pursuing a similar marketing strategy. “Demand growth [for petroleum products] comes exclusively from rapidly growing non-OECD economies,” BP noted in its recent report on the global energy outlook. “China, India, and the Middle East account for nearly all of the net global increase.” Like ExxonMobil, BP and the others are hard at work expanding their capacity to sell fossil fuels in these growing markets.

Nor are only the oil and gas companies pursuing this strategy. So is Big Coal. With coal demand declining in the U.S., thanks to the growing availability of low-cost natural gas generated by fracking, the coal firms are shipping ever more of their American output to Asia, which will contribute significantly to increasingly the carbon emissions there. According to the Energy Information Administration (EIA) of the Department of Energy, U.S. coal exports to China rose from essentially zero in 2007 to 10 million tons in 2012. Exports to India increased from 1.5 million to seven million tons and to South Korea from virtually nothing to nine million. Exports to just these three countries jumped by more than 1,000% during these years.

The EIA summarized the situation this way: “Companies in key parts of the U.S. coal supply chain -- both producers and railways -- have increased sales to Asia because of rising Asian coal demand, overall strong export prices, and lower U.S. consumption of coal to produce electric power.” Looked at from another perspective, diminished carbon emissions from coal in the United States -- much touted by President Obama in his embrace of natural gas -- has no significance when it comes to climate change, because of the greeenhouse gases being produced when all that coal is consumed in Asia.

To increase sales yet more, the giant coal companies are promoting the construction of new shipping terminals on the West Coast, including two each in Oregon and Washington State. The largest of these, the Gateway Pacific Terminal near Bellingham, Washington, will handle up to 48 million metric tons of coal a year, most of it destined for China and other Asian countries.

Although the terminals are often promoted by local officials as sources of new jobs, they are sparking fierce opposition from community activists and Native Americans who view them as posing a severe threat to the environment. Claiming that coal dust and spills from trains and loading facilities will harm fishing sites they deem vital, members of the Lumni tribe are citing longstanding treaty rights in their efforts to block the Cherry Point Terminal, one of the planned Washington State facilities.

In the Pacific Northwest, opposition to the coal terminals and the rail lines that will be so crucial to their operation -- some of which will traverse Indian reservations and pass through green-minded cities like Seattle -- is gaining strength. The process has been similar to the way climate activists mobilized against the Keystone XL pipeline that, if built, is slated to bring carbon-dense tar sands from Canada to the U.S. Gulf Coast. But the coal companies and their allies are pushing back, insisting that their exports are essential to the country’s economic vitality. “Unless the ports are built on the West Coast,” said Jason Hayes, a spokesman for the American Coal Council, U.S. suppliers won’t be viewed as “reliable business partners” in Asia.

Although community and tribal opposition may succeed in blocking or delaying a terminal or two, most analysts believe that, in the end, several will be built. “There are two billion people in Asia who need more power, so eventually more U.S. coal will get onto global markets,” says Matt Preston, an analyst for the energy consultancy firm of Wood Mackenzie.

Perpetuating the Fossil Fuel Era

In the end, all these efforts to boost fossil fuel sales in Asia and other developing areas will have one unmistakable result: a sharp rise in global carbon emissions, with most of the growth in non-OECD countries. According to the EIA, between 2010 and 2040 world carbon dioxide emissions from energy use -- the main source of greenhouse gases -- will rise by 46%, from 31.2 billion metric tons to 45.5 billion. Little of this increase will officially be generated by the planet’s wealthiest countries, where energy demand is stagnant and tougher rules on carbon emissions are being put in place. Instead, almost all of the growth of CO2 in the atmosphere -- 94% of it -- will be sloughed off on the developing world, even if a significant part of those emissions will come from the combustion of U.S. fossil fuel exports.

In the view of most scientists, an increase of carbon emissions on this scale will almost certainly lead to a global temperature rise of at least four degrees centigrade and possibly more by the end of this century. That’s enough to ensure that the changes we are already seeing, including severe droughts, stronger storms, raging wildfires, and rising sea levels, will be eclipsed by exponentially greater perils in the future.

Everyone will share in the pain from such warming-induced catastrophes. But people in developing lands -- especially the poorest among them -- will suffer more, because the societies they live in are least prepared to cope with severe catastrophes. “Climate-related hazards exacerbate other [socioeconomic] stressors, often with negative outcomes for livelihoods, especially for people living in poverty,” the UN’s Intergovernmental Panel on Climate Change observed in its most recent assessment of what global warming will mean for planet Earth. “Climate-related hazards affect poor people’s lives directly, through impacts on livelihoods, reduction in crop yields, or destruction of homes, and indirectly through, for example, increased food prices and food insecurity.”

Certainly, the giant fossil fuel companies bear a moral, if not as yet in our society a legal, responsibility for the intensification of climate change and the lack of serious response to it. Beyond this, their carefully planned strategy of selling carbon products to those most at risk can only be viewed as outright immorality. Just as health officials now condemn Big Tobacco’s emphasis on cigarette sales to poor people in countries with inadequate health systems, so someday Big Energy’s new “smoking” habit will be deemed a massive threat to human survival.

Above all, Big Energy is insuring that one small ray of good news when it comes to climate change -- the contracting use of coal, oil, and gas across the developed world -- will prove meaningless. The economic incentive to sell fossil fuels to developing countries is undeniably powerful. The need for increased energy in developing countries is no less indisputable. In the long run, the only way to meet these needs without endangering our global future would be through a mammoth drive to expand renewable energy options there, not by shoving carbon products down their throats. Rex Tillerson and his cohorts will continue to claim that they are performing a “humanitarian” service with their new “tobacco” strategy. Instead, they are actually perpetuating the fossil fuel era and helping to create a future humanitarian catastrophe of apocalyptic dimensions.