“Not Physiologically Plausible” — Says a Study of 39 Years of National Health and Nutrition Examination Survey (NHANES) Data — and Why Has this Arguably Newsworthy Allegation Received No Coverage from the American Media?

22 October 2013

Citation

Edward Archer, Gregory A. Hand, and Steven N. Blair, Validity of U.S. Nutritional Surveillance: National Health and Nutrition Examination Survey Caloric Energy Intake Data, 1971–2010, PLoS ONE 8(10): e76632, DOI:10.1371/journal.pone.0076632 (09 October 2013)

Just because survey information is cheap and easy to collect, doesn’t make it accurate

I had to smile (sadly), when the above cited study found that 39 years’ worth of federally collected nutrition data is probably bogus. Enough so to distort some of the major conclusions and recommendations drawn from it.

The study of the long-running federal NHANES survey found that people were reporting less food intake than would have been necessary just to maintain their weight.

What is NHANES?

The Centers for Disease Control and Prevention explain that:

The National Health and Nutrition Examination Survey (NHANES) is a program of studies designed to assess the health and nutritional status of adults and children in the United States.

The survey is unique in that it combines interviews and physical examinations.

The NHANES program began in the early 1960s and has been conducted as a series of surveys focusing on different population groups or health topics.

The survey examines a nationally representative sample of about 5,000 persons each year. These persons are located in counties across the country, 15 of which are visited each year.

The NHANES interview includes demographic, socioeconomic, dietary, and health-related questions.

The examination component consists of medical, dental, and physiological measurements, as well as laboratory tests administered by highly trained medical personnel.

The booby trap lies in NHANES’ method:

The primary method used in NHANES to approximate EI [energy intake] is the 24-hour dietary recall interview.

The data collected are based on the subject’s self-reported, retrospective perceptions of food and beverage consumption in the recent past.

To calculate EI [energy intake] estimates, these subjective data are translated into nutrient food codes and then assigned numeric energy (i.e., caloric) values from food and nutrient databases.

© 2013 Edward Archer, Gregory A. Hand, and Steven N. Blair, Validity of U.S. Nutritional Surveillance: National Health and Nutrition Examination Survey Caloric Energy Intake Data, 1971–2010, PLoS ONE 8(10): e76632, DOI:10.1371/journal.pone.0076632 (09 October 2013) (at second paragraph under Introduction)

The methodological clinker — people are inaccurate reporters

From previous studies, we know that people have sloppy recollections, or they lie:

Given the indirect, pseudo-quantitative nature of the method (i.e., assigning numeric values to subjective data without objective corroboration), nutrition surveys frequently report a range of energy intakes that are not representative of the respondents’ habitual intakes, and estimates of EI that are physiologically implausible (i.e., incompatible with survival) have been demonstrated to be widespread.

For example, in a group of “highly educated” participants, Subar . . . demonstrated that when total energy expenditure (TEE) via doubly labeled water (DLW) was compared to reported energy intake (rEI), the raw correlations between TEE and rEI were 0.39 for men and 0.24 for women.

Men and women underreported energy intake by 12–14% and 16–20%, respectively.

The level of underreporting increased significantly after correcting for the weight gain of the sample over the study period, and underreporting was greater for fat than for protein, thereby providing additional support for the well-documented occurrence of the selective misreporting of specific macronutrients (e.g., fat and sugars).

The study of the NHANES data set out to find out whether people’s “forgetfulness” had affected accuracy

The research team compared:

(a) energy intake (contained in what people reported they ate) to their basal metabolic rate

and then

(b) calculated whether there was a disparity between energy intake and energy expenditure.

In other words, was what people said they were eating enough to keep them going?

The answer was — no, you can’t do what you do, based on what you say you’re eating

Not surprisingly, previous findings of recollection inaccuracy affected NHANES, as well:

Across the 39-year history of the NHANES, EI [energy intake] data on the majority of respondents (67.3% of women and 58.7% of men) were not physiologically plausible.

Does this discrepancy in reporting matter?

Yes. Because of what NHANES data is used for:

Findings from this [NHANES] survey will be used to determine the prevalence of major diseases and risk factors for diseases.

Information will be used to assess nutritional status and its association with health promotion and disease prevention.

NHANES findings are also the basis for national standards for such measurements as height, weight, and blood pressure.

Data from this survey will be used in epidemiological studies and health sciences research, which help develop sound public health policy, direct and design health programs and services, and expand the health knowledge for the Nation.

The study’s authors concluded that NHANES is so methodologically inconsistent and its survey data so inaccurate, that its conclusions cannot be trusted

From the paper’s conclusion:

Throughout its history, NHANES dietary measurement protocols have failed to provide accurate estimates of the habitual caloric consumption of the U.S. population.

Furthermore, successive changes to the nutrient databases used for the 24HR data-to-energy conversations and improvements in measurement protocols make it exceedingly difficult to discern temporal patterns in caloric intake that can be related to changes in population rates of obesity.

As such, there are no valid population-level data to support speculations regarding trends in caloric consumption and the etiology of the obesity epidemic.

Because under-reporting and physiologically implausible rEI values are a predominant feature of U.S. nutritional surveillance, the ability to generate empirically supported public policy and dietary guidelines relevant to the obesity epidemic based on these data is extremely limited.

Given this bombshell allegation — why didn’t the American media pick up on it?

Anti-scientific bias?

Complacence of institutionalized stupidity?

A simple inability to reason beyond the obvious?

The moral? — There are three

These are:

(i) Self-reported survey data cannot be trusted, when it comes to matters that may have corollary social stigmas attached to them — like obesity, sweets addiction, and “I’m a dietary fat-sucker”.

(ii) In science, inconsistent baseline methodology destroys the possibility of drawing quantifiable trend conclusions.

(iii) Do not depend on the American media to tell you anything potentially important, or in ways that will encourage you to see the big picture.

PeteFree.com

“Not Physiologically Plausible” — Says a Study of 39 Years of National Health and Nutrition Examination Survey (NHANES) Data — and Why Has this Arguably Newsworthy Allegation Received No Coverage from the American Media?