Biased with Biases

I came across two articles recently that caught my attention, both of which were published by Peter Attia and his team.

It is worth repeating something I have said before. For the vast majority of health and longevity topics I am in alignment with Peter Attia. I have taken the perspective that instead of repeating what I agree with, it is more useful to dive into the nuances of the field where I disagree with Dr. Attia's interpretation. As I do this I want to make clear that I think what Dr. Attia is doing for this area of medicine in building awareness and making dependable knowledge available to the common person is admirable. He appears trustworthy, dependable, and accurate. 

That being said there seemed to be some inconsistencies with how he and his team analyzed the following nutrition and exercise studies. Let's take a look. 

Recall that Peter Attia is of the opinion that 1) exercise is good for longevity, and 2) that animal proteins in your diet are safe. As well look at the two following articles it is important to keep those opinions in mind. My read of the reviews posted appear to show a biased application of bias. That is, there is a claim that there is a bias in the adjustment analysis used in one paper to discredit the paper's conclusion, while citing the same sort of adjustment analysis in the other paper not only without the claim of bias, but indeed citing the adjustment analysis as justified strong support for the papers conclusion. 

Into the first review post.

In the first review posted on September 14th 2024 titled "A study suggest that exercise doe not impact mortality offers little more than click-bait" Peter Attia and his co author Kathryn Birkenbach review a paper by Kankaanpaa et al published in June of 2023 purporting to show that there was not a relationship between leisure time physical activity and risk of all cause mortality. Note again that this study's conclusions are very much against the core of Peter Attia's approach to medicine.

One major critique used on the Kankaapaa et al paper in this review is that a statistically significant finding (15% risk reduction in all-cause mortality in the moderately active group, an 18% risk reduction in the active group, and a 23% risk reduction in the highly active group compared to the sedentary group) was washed away when the investigators adjusted for health related variables. That is, when the researchers controlled for other health variables between groups, the data from the study no longer showed significant differences in the risk of all-cause mortality between leisure time physical activity groups. The researches concluded that the initial differences they saw between groups were due to exercise being a proxy for other healthy behaviors and better states of health in the subjects, rather than exercise being the driver of the decreased all-cause mortality itself.

Let's look at what the adjustment analysis actually did to the outcomes by comparing pre-adjustment analysis risk reductions with post-adjustment analysis risk reductions.

     -13% risk reduction pre adjustment fell to 7% reduction in all-cause mortality in the moderately active group compared to the sedentary group

     -18% risk reduction pre adjustment fell to 7% reduction in all-cause mortality in the active group compared to the sedentary group

     - 23% risk reduction pre adjustment fell to 6% reduction in all-cause mortality in the highly active group compared to the sedentary group

To this Attia and Birkenbach called shenanigans. They argued that Kankaapaa was committing a form of 'informative censoring bias', specifically that in adjusting for the health indicators they chose to adjust for, they removed the factors that were directly related to the outcome measure of the study. 

Let's put this into english. Essentially, the bias Peter and Kathryn are claiming occurred in the original paper is focused on the fact that the researchers used an adjustment technique that eliminated all participants that 1) had a heart attack, 2) had chest pain with exertion, or 3) had diabetes at the end of the study. The claim is that in doing this the authors of the study were removing the sick people from the sedentary group and artificially taking away the very mortality factors that the amount of activity was affecting. 

An example of this same bias type of happening is like saying 'I am going to do a study to see if always open windows vs always closed windows on a house affect how much maintenance cost there is for that house... but I am going to eliminate any houses that have water damage to their floors during the period of the study from being in the analysis.' This adjustment of removing houses with water damage to their floors from the analysis would almost certainly affect the study outcomes. Having your window open during rain storms will likely increase the maintenance and repair costs of a house compared to if they were not closed. But if you adjust for that by eliminating all water damage to floors in the maintenance cost analysis, then the difference between open window houses and closed window houses becomes artificially less significant, possibly resulting in the false conclusion that the variable of having a window always open or always closed has no affect on house maintenance costs.

Fair enough. I agree with this take on the paper.

Figure from Kankaapaa et al 2023 showing activity level in MET/h between groups

Figure 5. Within-twin-pair differences in all-cause mortality between the long-term leisuretime physical activity classes for A) all twin pairs, B) monozygotic (MZ) pairs and C) dizygotic (DZ) pairs after excluding twin pairs who reported specific diseases. The sedentary class was treated as the reference. Only twin pairs with information on LTPA and alive in 1990 were included in the analysis. Model 1 was adjusted for sex (female) and age at the between-twin-pair level. Model 2 was additionally adjusted for education, body mass index, smoking and alcohol use at the within-twin-pair level.

The figure from Kankaapaa et al 2023 showing how the hazard odds ratio (hOR) of all-cause mortality went from significant in model 1 (red box) to not significant in model 2 (yellow box). You can tell this because the little tails on either side of the dots cross 1, indicating that they do not have statical significance. 

Now let's look at the second study. 

This one is reviewing an article about nutrition. Remember, Peter and his team have concluded before that they believe animal proteins are safe in human diets. 

This review posted on August 24th 2024 titled "Back on the merry-go-round of bad science regarding meat consumption." Peter Attia and his again co-author Kathryn Birkenbach review a paper by Li et al published in September 2024 (interestingly the review came out several days before the original paper was officially published) purporting to show that there is an increased incidence of type 2 diabetes in individual that consume meat.

As in the previous study reviewed, these authors took the data and did adjustment analysis. This caused the same thing to happen. A statistically significant finding reduced in power on the first analysis and then became non-statistically significant on the second adjustment analysis. 

To summarize, with the adjustment for BMI, the risk of Type2 diabetes dropped in every category.

     -Unprocessed red meat from 18% increase to 10% increase in Type 2 Diabetes

     -Processed meat from 23% increase to 15% increase in Type 2 Diabetes

     -Poultry from 21% increase to 8% increase in Type 2 Diabetes

To quote directly from the review article, it says: "However, they do share that hazard ratios associated with meat intake were substantially higher in models that excluded adjustments for BMI (HR: 1.18; 95% CI 1·07-1·29 for unprocessed red meat; HR: 1.23; 95% CI: 1.14-1.34 for processed meat; and HR: 1.21; 95% CI: 1.12-1.31 for poultry). This observation would agree with the alternative explanation that factors related to general health – rather than meat intake per se – are responsible for the apparent association between meat consumption and diabetes risk."

To quote further: "As we’ve already seen, adding adjustment for BMI cuts the excess risk associated with meat consumption approximately in half, leaving us with a 10% excess risk for every 100 g/day of unprocessed red meat, 15% for every 50 g/day of processed meat, and 8% for every 100 g/day of poultry. Given that no study can account for every possible covariate, these numbers are small enough on their own to cause plenty of doubt about these correlations."

Intersting... is not the same thing happening in both papers? Is not this same 'informative censoring bias' removing factors that are directly related to the  outcome measure of the study by performing their adjustment analysis?

Note that the drop when applying analysis is astonishingly similar between papers. To see it again let's see what adjustment analysis does to the findings in both papers:

Paper 1

     -13% risk reduction pre adjustment fell to 7% reduction in all-cause mortality in the moderately active group compared to the sedentary group

     -18% risk reduction pre adjustment fell to 7% reduction in all-cause mortality in the active group compared to the sedentary group

     - 23% risk reduction pre adjustment fell to 6% reduction in all-cause mortality in the highly active group compared to the sedentary group

Paper 2

     -18% increase pre adjustment fell to 10% increase in Type 2 Diabetes with unprocessed red meat 

     -23% increase pre adjustment fell to 15% increase in Type 2 Diabetes with processed meat

     - 21% increase pre adjustment fell to 8% increase in Type 2 Diabetes with poultry.

These look like similar direction and magnitude of changes to me.

However according to the review in Paper 1 it is a bias, in Paper 2 it is not a bias. 

Calling bias in the first paper strengthen's the assertion that exercise is good for longevity. 

Not calling bias in the second paper strengthens the assertion that animal proteins in your diet are safe. 

It seems there is a bias with the use of biases. 

To me this sets off some red flags. The little voice in my head is saying, "This sounds like selective use of statistical biases to confirm one's own opinions."

Kankaanpää, A., Tolvanen, A., Joensuu, L., Waller, K., Heikkinen, A., Kaprio, J., ... & Sillanpää, E. (2023). The associations of long-term physical activity in adulthood with later biological ageing and all-cause mortality–a prospective twin study. medRxiv.

Li, C., Bishop, T. R., Imamura, F., Sharp, S. J., Pearce, M., Brage, S., ... & Wareham, N. J. (2024). Meat consumption and incident type 2 diabetes: an individual-participant federated meta-analysis of 1· 97 million adults with 100 000 incident cases from 31 cohorts in 20 countries. The Lancet Diabetes & Endocrinology, 12(9), 619-630.