Breaking Out of the 'Seems Good for My Health' Illusion. A Framework and Check Template to Instantly Spot the 'Evidence Strength' of Medical Research

The internet is overflowing with health information like “Just chewing well makes you lose weight!” or “Eating ___ extends your lifespan!” Even if you are enticed by claims like “proven by the latest research,” the quality of those papers varies widely, and in reality, it’s not uncommon for them to be nothing more than a simple questionnaire survey.

Experts at medical schools and research institutions grade the countless papers by fitting them into a pyramid called “strength of evidence (scientific basis).” In this article, we explain a powerful evaluation template that allows anyone to easily judge “how credible this research is.” If you master this, you should no longer be swayed by the truth or falsehood of health information.

The Evidence Pyramid: Differences in Strength Depending on Study Design

When evaluating the quality of a paper, what is emphasized most is “what kind of design was used to collect the data (study design).” Let’s look at them in order from lowest strength to highest.

1. Animal/Cell Experiments (Weakest Evidence): Research like “we fed mice hard food and their brain cells increased.” Important for elucidating mechanisms, but there is no guarantee it applies directly to humans.
2. Expert Opinions / Case Reports: Personal rules of thumb like, “I have seen patients who chew well for many years, and they are healthy.” This falls into a very weak category as scientific evidence.
3. Cross-Sectional Studies (Observational Studies): Measuring both things at “a certain point in time.” On this site, E04 (the Suita Study) and E07 are of this type. Even when the result “people who chew well are less obese” comes out, there is no guarantee which is cause and which is effect (reverse causality). This item previously gave E05 as the example. E05 is a within-subject crossover trial in 50 adults, not a questionnaire survey.
4. Cohort Studies (Longitudinal Studies): Studies tracking thousands of people over years to decades. The strength is higher because you can follow the flow of time. This site does not currently cite a chewing cohort study. This item previously named E06, which is a systematic review, not a cohort.
5. Randomized Controlled Trials (RCT): An experiment where subjects are divided by lottery into a “gum-chewing group” and a “non-chewing group,” and changes are observed after a certain period. Because other factors (confounding factors) can be eliminated, it is considered strong evidence.
6. Systematic Reviews and Meta-Analyses (the heaviest tier here): Gathering dozens of past high-quality RCT papers, recalculating them together, and drawing a “conclusion.” But if the papers gathered are weak, the conclusion is weak too. Sitting high in the pyramid does not exempt a review from being read.

”Three Checkpoints” to Spot Chewing Evidence

In addition to the pyramid, here is a dedicated evaluation template with three points that should be applied specifically when reading papers related to “chewing.”

1. Measurement Bias: Self-Reported or Objective Indicator?

How was the data “chewing well” obtained?

• Weak: Questionnaire to subjects: “Do you usually chew well? (Yes/No)”
• Strong: Measured the degree of color change in color-changing gum (color tester), or accurately counted the number of masseter muscle activities with a wearable electromyography sensor.

2. Adjustment for Confounding Factors: Is it Really Thanks to “Chewing”?

If the result “people who chew well live longer” comes out, confirm if there are any overlooked factors.

• Weak: Simple comparison of average values.
• Strong: Is the figure the result of statistically adjusting (eliminating) all other elements related to lifespan, such as the subjects’ “age,” “income level,” “exercise habits,” and “smoking history”?

3. Confirmation of Sponsor Bias (COI)

• Is the funder of an RCT touting the health benefits of gum a major gum manufacturer? This in itself is not evil, but it is necessary to hold a certain reservation in interpretation (confirmation of “Conflict of Interest: COI”).

Summary: Upgrading from an Information Consumer to an “Evaluator”

Next time you see news like “The Latest Chewing Diet Method!”, instead of jumping on it immediately, try asking, “What tier of the pyramid is the paper acting as the source for that?” and “Is the frequency self-reported?” By possessing this evidence evaluation template (grading perspective), you will not be misled by media hype, and it becomes possible to extract and practice only truly valuable health habits.