Adverse Covid-19 Vaccines Booster Reactions Case Series, From November 2021 To September 2022, In a General Medicine Office in Toledo, Spain

  • Case Studies
  • Jose Luis Turabian1,*
  • 1 Specialist in Family and Community Medicine Health Center Santa Maria de Benquerencia. Regional Health Service of Castilla la Mancha (SESCAM), Toledo, Spain.
  • *Corresponding author: Jose Luis Turabian, Health Center Santa Maria de Benquerencia Toledo, Spain, E-mail: jturabianf@hotmail.com.
  • Received: 02-10-2022; Accepted: 05-11-2022; Published: 21-11-2022.

Abstract

Background: There is lack of clinical outcome data on the safety of covid-19 vaccines booster.

Objective: To study, in general medicine, clinical-epidemiological characteristics of patients who consulted for self-reported adverse reactions to covid-19 vaccines booster.

Methodology: An observational, longitudinal and prospective case series study of patients with adverse reactions to covid-19 vaccines booster, based on a cohort of patients in a family medicine office in Toledo (Spain) was carried out from November 1, 2021 to September 1, 2022.

Results: Twenty-one cases of adverse reactions to covid-19 vaccines booster were included. The mean age of the patients was 45 years. 67% were women. According to the criteria of causality, 42% were probable. According to the time of appearance, the majority (76%) occurred between 1-72 hours after shot. 52% were moderate (with interference with normal activities); there was no severe adverse reaction. 81% of adverse reactions to covid-19 vaccines booster occurred with Moderna mRNA-1273. Only 1 case had also presented an adverse reaction with a previous dose of the vaccine. 71% had chronic diseases, being 21% from the genitourinary group and 17% mental. 64% of the symptoms were from the group of symptoms, signs not elsewhere classified (Injection site pain, throat pain, fever, chills, dizziness, headache, asthenia, lymphadenopathy).

Conclusion: In the context of general medicine in Toledo (Spain), during the 10 months following the start of the covid-19 vaccination booster, no serious adverse effects were found, which were mostly symptoms and signs not otherwise specified in middle aged women with chronic diseases of genitourinary and mental system.

Key Words: COVID-19; Adverse Drug Events; Post-vaccination Reactions; Booster; COVID-19 vaccine; General Practice; Case Series

Introduction

The development and widespread use of vaccination in recent centuries has been the most impactful intervention in public health effectively preventing morbidity and mortality from infectious diseases. Vaccination is generally well tolerated in the vast majority of the population, and their benefits greatly outweigh the risk of serious adverse events in most patients (1), although vaccination, like any other medical intervention, can have adverse effects (2).

All health care providers play an important role in maintaining public confidence in vaccines because their attitude and knowledge are often critical to facilitating acceptance of a vaccine. Providers should have a thorough understanding of common and rare adverse effects of vaccination. The reason is that healthcare and vaccine providers play a central and critical role in vaccine pharmacovigilance through post-licensure surveillance. The safety of the vaccine can only be inferred from the absence of adverse reactions to the vaccine (1). Adverse drug reaction is a broad term that refers to unwanted, uncomfortable, or dangerous effects that a drug can have (3).

Common adverse reactions to the vaccine are characterized during clinical trials (Phase I-III), but as the vaccinated cohort is limited in number (often <20,000), reactions that are rare (frequency <1 in 1,000), late or occurring in subpopulations are unknown at the time of the vaccine authorization. Post-authorization surveillance plays a critical role in determining previously unrecognized reactions, but is also important in detecting an increased frequency of a known reaction. In most countries post-authorization surveillance rely on passive or spontaneous reporting by health professionals or consumers (1).

Although vaccine efficacy and effectiveness is of concern to public health authorities because it determines disease control, community and health care provider perceptions of vaccine safety is an important determinant of vaccine acceptability. This is more evident for newer vaccines (1). Available covid-19 vaccines are effective in protecting people from becoming seriously ill, being hospitalized, and dying. As with other illnesses, you're best protected against covid-19 when you stay up-to-date on recommended vaccinations, including any recommended boosters (4, 5).

Mass vaccination is associated with a lower spread of the pandemic and, to a greater extent, with a decrease in mortality in infected people (6-9). Inter-individual variability regarding incidence, severity and mortality rate of covid-19 was recently recorded and it was found that both Angiotensin Converting Enzyme 2 and Trans-membrane protease serine type 2 are playing a crucial role for virus entry into host cells, where, over gene-expression in males, may raise the hypothesis of male predominance in covid-19 pandemic, along with the higher severity and worse outcome (10). Additionally, immune dysregulation is the main leading cause for exacerbation of covid-19 symptoms and increased mortalities. Different therapeutic choices showed to be effective in restoring immune coordination, and improving clinical signs and symptoms (11).

A progressive decline in vaccine effectiveness against SARS-CoV-2 infection of any severity has been shown in all subgroups, but the rate of decline differed by vaccine type.

Different studies showed an advantage for covid-19 vaccine third “booster” dose regarding decreased infection, hospitalization, and morality rates. Absolute vaccine effectiveness for BioNTech/Pfizer (BNT162b2) or Moderna (mRNA-1273) booster ranged from 94% to 97% and 10-fold decrease in infection rate for those boosted with BNT162b2 (12).

Moreover, it was reported that the chances of vaccine infection breakthrough were much higher in delta variant infections compared to alpha strain cases, suggesting that messenger RNA (mRNA) vaccines were less effective in preventing infection with the delta type variation and raises the absolute risk of death, particularly in the elderly, reduces immunization efficacy, and increases the likelihood of hospitalization (13)

With regard to severe covid-19, the efficacy of the vaccine seemed to be better maintained, although some decline was evident after a few months. The results strengthen the evidence-based rationale for administering a third dose of vaccine as a booster (14-16). Genetic alterations of SARS-CoV-2 genome had imparted functional differences in infectivity, and sub-strains might arise as a result of acquired immunity which by turn  will diminish over time and become less effective against increasingly aggressive strains and thus a possible decrease in vaccination efficacy and emergence of treatment-resistant strains (17). To acquire immunity in a short amount of time, vulnerable persons should be mass vaccinated and thus a suggested virtual vaccination system might assist in providing immunization services while protecting people’s safety (18)

The risk and adverse event profiles associated with additional doses of vaccines are currently poorly understood (15). Studies have suggested short-term incremental adverse events after repeated vaccination against SARS-CoV-2 (19).

In short, the mass application of vaccines, as covid-19 vaccination, requires robust pharmacovigilance systems and global coordination of post-licensing surveillance, both for governments to make the right decisions and to maintain or gain public confidence in vaccines. On the other hand, general medicine consultation is a good observatory of adverse drug reactions, including vaccines. When GPs consider that there is an effect possibly related to the vaccine, they must report it, to rule it out or maintain it (20). In this context, we present a study whose objective is to provide a clinical-epidemiological overview of adverse events in patients who consult in general medicine for self-reported adverse reactions after vaccination with the third dose (booster) against covid-19.

Material and Methods

Design and Emplacement

An observational, longitudinal and prospective case series study of patients with adverse covid-19 vaccines reactions (ACVRs), based on a cohort of patients was carried out from November 1, 2021 to September 1, 2022, in a family medicine office, in the Health Center Santa Maria de Benquerencia, Toledo (Spain), which has a list of 2,000 patients > 14 years of age (in Spain, the general practitioners [GPs] care for people > 14 years of age, except for exceptions requested by the child's family and accepted by the GP).

Events of Interest

The events of interest in this study were the spontaneous communications of the adverse reactions to the third dose of covid-19 vaccine, of the patients who consulted spontaneously for this reason, with the GP in general medicine.

Booster Doses

As of November 23, 2021, in Castilla La Mancha, the region where the study was carried out, booster doses against Covid-19 with mRNA vaccines were started 6 months after the end of the vaccination calendar and 3 months in case of having received a dose of the Ad26.COV2. S vaccine (Janssen vaccine; Johnson & Johnson vaccine). Recruitment was actively carried out by descending age cohorts, starting with those over 80 years of age and people hospitalized in centers for the elderly and in other socio-health centers (including day centers and occupational centers) regardless of the age, the people who received a dose of the Ad26.COV2.S vaccine (Janssen vaccine; Johnson & Johnson vaccine) as a primary vaccination and those with a homologous schedule of Vaxzevria as a primary vaccination (first and second dose of Vaxzevria, from AstraZeneca), followed by people aged between 79 to 70 years old, 69 to 65 years old, 64 to 60 years old, 59 to 50 years old and 49 to 40 years old, etc. The booster dose was given with mRNA vaccines (0.3 ml Comirnaty or 0.25 ml Spikevax – half the usual primary dose) (21-23).

Definition of Adverse Reaction to the Vaccine

Adverse reaction was any unintended and noxious response to a drug that occurred at doses normally applied to humans for the prophylaxis, diagnosis, or treatment of disease, or for the restoration, correction, or modification of disease (24, 25).

Collected Variables

Data were extracted from the medical records of the general medicine practice under study. The following variables were collected:

  1. Age and sex.
  2. Symptoms of adverse reactions to covid-19 vaccines booster and chronic diseases (26), both classified according to the International Statistical Classification of Diseases and Health-Related Problems, CD-10 Version: 2019 (27).
  3. Complex family based on the genogram (28-29).
  4. Ethnic minority (30).
  5. Socio-sanitary workers.
  6. Symptomatic / asymptomatic prior covid-19 (the diagnosis was performed with reverse transcriptase polymerase chain reaction (PCR) oropharyngeal swab tests or antigen testing or antibody test (31).
  7. Vaccine type: Only ARNm vaccines: Comirnaty (Pfizer-BioNTech-BNT162b2 mRNA) or Spikevax (mRNA-1273 vaccine Moderna) (4).
  8. Criteria for the causality of ACVR, classified as Definitive (Certain), Probable (Likely), Possible, Unlikely, Conditional / Unclassified and Not evaluable / Unclassifiable (32-34).
  9. Severity or intensity of ACVR, classified as Mild, Moderate and Severe (35).
  10. Time of appearance of adverse reactions to covid-19 vaccines booster classified as Immediate, Expedited and Late (36).
  11. Date of the adverse reaction of the booster.

Results

Twenty-one cases of adverse reactions to covid-19 vaccines booster were included

Table 1: Adverse Covid-19 Vaccine Booster Reactions in General Medicine for The Period November 1, 2021 to September 1, 2022.

VARIABLES

ADVERSE COVID-19 VACCINES BOOSTER REACTIONS
N=21 (%)

Age in years (arithmetic mean+-standard deviation; range)

45.57 +- 13.38 (18-71 years)

=< 14 years

0

> = 65 years

1 (5)

14-65 years

20 (95)

49-65 years

8 (38)

14-49 years

12 (57)

14-18 years

1 (5)

Women

14 (67)

Date of the adverse reaction of the booster in 2022

15 (71)

Adverse reaction with first or second dose

1 (5) (with first dose)

Symptomatic and asymptomatic prior covid-19

2 (9)

Socio-sanitary workers

4 (19)

Complex family (Potential problems familiar context of the patient)

2 (9)

Ethnic minority

2 (9)

Cases with chronic diseases

15 (71)

Table 2: Criteria of Causality, Time of Appearance, Gravity and Type of Vaccine Involved in the Adverse Covid-19 Vaccines Reaction in General Medicine for the period November 1, 2021 to September 1, 2022.

VARIABLE

ADVERSE COVID-19 VACCINES BOOSTER REACTIONS
N=21 (%)

CRITERIA OF CAUSALITY

 

1.-Certain

0

2.-Probable

9 (42)

3.-Possible

6 (29)

4.-Unlikely

6 (29)

5.-Conditional/ Unclassified)

0

6.-Unassessable/Unclassifiable

0

TIME OF APPEARANCE OF THE ADVERSE COVID-19 VACCINES REACTION

 

1.-Immediate

0

2.-Accelerated

16 (76)

3.-Late

5 (24)

GRAVITY OF THE ADVERSE COVID-19 VACCINES REACTION

 

1.-Mild

10 (48)

2.-Moderate

11 (52)

3.- Severe

0

TYPE OF VACCINE INVOLVED IN THE ADVERSE COVID-19 VACCINES REACTION

 

Pfizer-BioNTech-BNT162b2 (Pfizer / BioNTech) mRNA

4 (19)

Moderna mRNA-1273

17 (81)

Table 3: Chronic Diseases of Patients with Adverse Covid-19 Vaccines Booster Reactions in General Medicine for the period November 1, 2021 TO September 1, 2022.

CHRONIC DISEASES*
ACCORDING TO WHO,
ICD-10 GROUPS

ADVERSE COVID-19 VACCINES BOOSTER REACTIONS
N=21 (%)

-I Infectious

1 (2)

-II Neoplasms

1 (2)

-III Diseases of the blood

0

-IV Endocrine

6 (10)

-V Mental

11 (17)

-VI-VIII Nervous and Senses

9 (14)

-IX Circulatory system

4 (6)

-X Respiratory system

4 (6)

-XI Digestive system

8 (13)

-XII Diseases of the skin

0

-XIII Musculo-skeletal

6 (9)

-XIV Genitourinary

13 (21)

TOTAL chronic diseases*

100)

* Patients could have more than one chronic disease. The percentages are over the total of chronic diseases

Table 4: Symptoms of Adverse Covid-19 Vaccines Reaction in General Medicine for the period November 1, 2021 to September 1, 2022.

SYMPTOMS *
ACCORDING TO WHO,
ICD-10 GROUPS

ADVERSE COVID-19 VACCINES BOOSTER REACTIONS
N=21 (%)

-VI-VIII Nervous and Senses
(earache, conjunctivitis)

2 (5)

-X Respiratory system
(cough)

3 (7)

-XI Digestive system
(diarrhea, nausea, vomiting)

2 (5)

-XII Diseases of the skin
(urticaria)

1 (2)

-XIII Musculo-skeletal
(myalgia)

7 (17)

XVIII Symptoms, signs and abnormal clinical and laboratory findings, not elsewhere classified
(Injection site pain, arm pain, throat pain, fever, chills, dizziness, headache, asthenia, lymphadenopathy)

27 (64)

TOTAL SYMPTOMS*

42 (100)

* Patients could have more than one symptom. The percentages are over the total of symptoms

  1. Age and Sex: The mean age of the patients was 45 years. Only 5% were >= 65 years old, and another 5% were between 14-18 years old. 67% were women.
  2. According to the Criteria of Causality, Time of Appearance and Gravity of the Adverse Covid-19 Vaccines Reaction: 42% were probable and 29% possible. The majority (76%) occurred between 1-72 hours after the shot. 52% were moderate (with interference with normal activities); there was no severe adverse reaction.
  3. Vaccine Type: 81% of adverse reactions to covid-19 vaccines booster occurred with Moderna mRNA-1273.
  4. Date of the adverse reaction of the booster: 71% of the total occurred during 2022.
  5. Covid-19 before the booster: 9% had previously presented covid-19 before the booster.
  6. Adverse Reaction with other Doses of Vaccine: 1 case had also presented an adverse reaction with the first dose of the vaccine.
  7. Psychosocial Factors: 9% presented complex family (Potential problems familiar context of the patient); Also 9% were ethnic minority. 19% were social health personnel.
  8. Chronic Diseases: Regarding the chronic diseases of the adverse reactions to covid-19 vaccines booster, 71% had chronic diseases, being 21% from the genitourinary group, 17% mental, 14% nervous and senses, and 13% digestive system. 64% of symptoms were signs not elsewhere classified (Injection site pain, arm pain, throat pain, fever, chills, dizziness, headache, asthenia, lymphadenopathy).

Discussions

Main Findings

In our series of self-reported cases of adverse reaction to the covid-19 booster, they occurred in middle-aged women, with mild, symptoms not elsewhere classified (Injection site pain, throat pain, fever, chills, dizziness, headache, asthenia, lymphadenopathy, etc.), less than half were considered probable, the vast majority occurred between 1-72 hours after the shot, most occurred with Moderna mRNA-1273 (which was the most used vaccine), and 71% had chronic diseases with 21% of chronic diseases of Genitourinary group and 17% Mental.

Comparison with other Studies

Adverse reactions to the covid-19 vaccine booster occur more frequently in women and middle-aged adults. Women report more and worse Adverse reactions. This may be due to a combination of factors, including hormones, genes, and the dose of the injections, but it also occurs in vaccines other than covid-19 (37). Likewise, in Spain it has been published that the majority of adverse reactions with the booster occurred in women (69%) and in people between 18 and 65 years of age (87%) (38); in our case series, this data is repeated, with 67% of women.

Adverse reactions after getting a booster shot have been reported to be mild to moderate (39, 40). The most commonly reported side effects were fever, headache, fatigue and pain at the injection site (39). At the end of March 2022 with the third dose of the covid-19 vaccine in Spain (40) (where more than 24 million booster doses had been administered: 44% corresponding to the Comirnaty (Pfizer-BioNTech-BNT162b2 mRNA) and 56% to Spikevax (mRNA-1273 vaccine Moderna), and 37% of the adverse reactions corresponded to the administration of Comirnaty and 63% to Spikevax), the vast majority were mild reactions (although 35% were considered serious) and transient that occurred in the first days after administration of the vaccine. Until March 2022, in Spain, after receiving the booster dose of Comirnaty (Pfizer-BioNTech-BNT162b2 mRNA), already known symptoms were reported, except for axillary pain, all adverse reactions are known for this vaccine (21% lymphadenopathy, 20% fever, 10% headache, 6% myalgia, 7% malaise, 6% fatigue, 4% injection site pain, 4% chills, 3% arthralgia). In the same way, the ten most repeated adverse effects with the third dose with Spikevax (mRNA-1273 vaccine Moderna) have been identified: 34% fever, 18% headache, 16% lymphadenopathy, 12% myalgia, 9% malaise, 9% pain in the area injection, 8% nausea, 8% fatigue, 7% arthralgia, and 6% chills (38, 41).

On the other hand, both homologous and heterologous boosters have been shown to be safe and immunogenic (42). But, those who received the same type of vaccine for all of their shots (homologous boosters) tended to have fewer reactions after the booster dose than after the second dose (16).

Limitations and Strengths of the Study

  1. It must be taken into account that the results presented are minimal data, since adverse reactions to the covid-19 vaccine booster being mostly mild or moderate in nature, and so they could not generate always a consultation with the GP.
  2. It must be taken into account that in our study the notification by patients of side effects was considered. Thus, non-specific adverse effects of taking medications, which are not a direct result of the pharmacological action of the drug, may be included. Several factors appear to be related: the patient's expectations of adverse effects; a conditioning process in which the patient learns from previous experiences and associates taking medication with somatic symptoms; certain psychological characteristics such as anxiety, depression and the tendency to somatise; and circumstantial and contextual factors (43-45). In our study, the patients who reported adverse reactions to the covid-19 vaccine booster presented 17% of chronic diseases of mental group.
  3. The follow-up period for reporting adverse events is important to interpret event appropriately. Our prospective study based on continued GP care allowed a long follow-up time.

Conclusion

In the context of general medicine in Toledo (Spain), during the first 10 months of booster covid-19 vaccination, no serious adverse effects were found. They were mostly symptoms and signs not otherwise specified and transient, which occurred preferentially in middle-aged women with chronic diseases.

References

  1. Pillai RK. Clinical epidemiology: Let us join hands to make it more popular. Clin Epidemiol Glob Health. 2015;3(1): 1–2.
  2. Gómez Viera N. Clinical epidemiology. [Article in Spanish]. Rev cubana med. 1997;36(2).
  3. Fowkes FG, Dobson AJ, Hensley MJ, Leeder SR. The role of clinical epidemiology in medical practice. Eff Health Care. 1984;1(5):259–265.
  4. Rodríguez Artalejo F, del Rey Calero J. Clinical epidemiology versus public health epidemiology. Rev San Hig Púb. 1992;66(2): 111–113.
  5. Levy PS. Clinical Epidemiology: The Essentials. JAMA; 1983;250(11):1469.
  6. Steineck G, Kass PH, Ahlbom A. A Comprehensive Clinical Epidemiological Theory Based On the Concept of the Source Person–Time and Four Distinct Study Stages. Acta Oncol; 1998;37:(1): 15–23.
  7. Díaz–Vélez C, Soto–Cáceres V, Peña–Sánchez RE, Apolaya Segura MA, Galán–Rodas E. Clinical Epidemiology and Its Relevance for Public Health in Developing Countries. In: Rodriguez–Morales AJ. Current Topics in Public Health: Intech Open. 2013.
  8. Turabian JL. [Notebooks of Family and Community Medicine. An introduction to the principles of Family Medicine]. Madrid: Díaz de Santos. 1995.
  9. Frérot M, Lefebvre A, Aho S, Callier P, Astruc K, Glélé LSA . What is epidemiology? Changing definitions of epidemiology 1978–2017. Plos One. 2018 : 0208442.
  10. Morabia A. Pandemics and methodological developments in epidemiology history. J Clin Epidemiol. 2020;125: 164–169.
  11. Olsman E, Veneberg B, Alfen C van, Touwen D. The value of metaphorical reasoning in bioethics: An empirical–ethical study. Nursing Ethics. 2019;26(1): 50–60.
  12. Sontag S. Illness as metaphor. Aids and its metaphors. London: Penguin Books. 1991.
  13. Reisfield GM, Wilson GR. Use of Metaphor in the Discourse on Cancer. J Clin Oncol. 2004;22(19): 4024–4027.
  14. Skelton JR, Wearn AM, Hobbs FD. A concordance–based study of metaphoric expressions used by general practitioners and patients in consultation. Br J Gen Pract. 2002;52: 114–118.
  15. Colman W. Theory as metaphor: clinical knowledge and its communication. J Anal Psychol. 2009;54(2): 199–215.
  16. Scherer AM, Scherer LD, Fagerlin A. Getting Ahead of Illness: Using Metaphors to Influence Medical Decision Making. Med Decis Making. 2015;35(1): 37–45.
  17. Rosenman S. Watch that metaphor! Aust N Z J Psychiatry; 2016;50(6): 507–508.
  18. Creed A, McIlveen P. Metaphor identification as a research method for the study of career. Int J Educ Vocat Guidance. 2018;18: 27–44.
  19. Martínez Calvo S, Ramis Andalia RM. The clinical method, the epidemiological method and the clinical epidemiology]. [Article in Spanish]. Rev Cub Salud Publica. 2012;38(4): 615–621.
  20. Rodriguez Rivera L. The clinic and its method. Reflections on two epochs. Madrid: Díaz de Santos. 1999.
  21. Thompson S, Schick–Makaroff K. Qualitative Research in Clinical Epidemiology. In: Parfrey, P.S., Barrett, B.J. (eds) Clinical Epidemiology. Methods in Molecular Biology, 2021. vol 2249. Humana, New York, NY.
  22. Gordon D. Therapeutic Metaphors: Helping Others through the Looking Glass. Capitolo, CA: Meta Publications. 2017.
  23. Kaptein AA, Meulenberg F, Smyth JM. A breath of fresh air: Images of respiratory illness in novels, poems, films, music, and paintings. J Health Psychol. 2015;20(3): 246–58.
  24. Turabian JL, Perez–Franco B. The Family Doctors: Images and Metaphors of the Family Doctor to Learn Family Medicine. New York. Nova Publishers. 2016.
  25. Turabian JL. Fables of Family Medicine. A collection of fables that teach the Principles of Family Medicine. Saarbrücken, Deutschland/Germany: Editorial Académica Española. 2017.
  26. Huyck PH, Kremenak NW. Design and memory. Computer programing in the 20th Century. New York: McGraw–Hill Book Company. 1980.
  27. Hoggan C. Transformative Learning Through Conceptual Metaphors: Simile, Metaphor, and Analogy as Levers for Learning. Adult Learning. 2014;25(4): 134–141.
  28. Turabian JL, Pérez Franco B. A way to make clinical pragmatism operative: sistematization of the actuation of competent physicians]. [Article in Spanish]. Med Clin (Bar). 2005;124(12): 476.
  29. Marín R, De la Torre S (Coordinadores). Manual de la Creatividad. Aplicaciones educativas. Barcelona: Vicens Vives. 1991.
  30. De Bono E (1996) Textbook of wisdow. London: Penguin Books.
  31. Shaw B. The Doctor’s dilemma (1958) Edinburgh: Penguin Books.
  32. Confucio (1974) [The four classic books]. Barcelona: Editorial S.A.
  33. Carreño Serra A. [Frequentist statistics and inferential statistics. The Bayes Theorem]. [Article in Spanish]. CAP [Homepage in Internet]. 2006.
  34. Hamm RM, Beasley WH. The Balance Beam Metaphor: A Perspective on Clinical Diagnosis. Med Decis Making. 2014;34(7): 841–53.
  35. Turabian Fernández JL, Pérez Franco B. [Variablility is an Indicator of Good Clinical Management in Family Medicine]. [Article in Spanish]. Aten Primaria. 2006;37(3):160–166.
  36. Leeflang MMG, Rutjes AWS, Reitsma JB, Hooft L, Bossuyt PM. Variation of a test’s sensitivity and specificity with disease prevalence. CMAJ. 2013;185(11) E537– E544.
  37. Wasserstein RL, Schirm AL, Lazar NA. Moving to a World Beyond “p < 0.05”. Am Stat. 2019;73(sup1): 1–19.
  38. Barker DJP, Rose G. Epidemiology in medical practice. New York: Churchill Livingstone. 1984.
  39. Fletcher RH, Fletcher SW, Wagner EH. Clinical epidemiology. The esssentials. Baltimore: Williams & Wilkins. 1982.
  40. Lilienfeld AM, Lilienfeld DE. Fundamentals of epidemiology. México: Addison–Wesley Iberoamericana. 1986.
  41. Last JM. The iceberg: “completing the clinical picture” in general practice. The Lancet. 1963;282(7297): 28–31.
  42. Greenland S, Senn SJ, Rothman KJ, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31(4): 337–350.
  43. Heckman MG, Davis JM, Crowson CS. Post Hoc Power Calculations: An Inappropriate Method for Interpreting the Findings of a Research Study. J Rheumatol. 2022;49(8): 867–870.