INTRODUCTION
Both retrospective and prospective research has shown that most adult mental disorders begin in childhood and adolescence (Freţian et al., 2021; Radez et al., 2021; Whitney & Peterson, 2019; Medina-Mora et al., 2003). Untreated mental health problems at these stages of life are strong predictors of higher rates of low occupational attainment, difficulties in interpersonal and family relationships, as well as a reduction in life expectancy and quality of life due to associated medical conditions (Kessler et al., 1995; Singh & Winsper, 2017). Despite this evidence, research shows that between 40% and 80% of adolescents and young people worldwide fail to receive mental health care (WHO, 2006; Thornicroft, 2007; Zwaanswijk et al., 2005). In Mexico, the percentage is above 86%, which is alarming (Kohn et al., 2018).
One of the main barriers to receiving care faced by children and adolescents is stigma (Radez et al., 2021; Thornicroft et al., 2022; Hatzenbuehler et al., 2013; Gulliver et al., 2010; Pescosolido et al., 2007). Stigma is the labeling, stereotyping, rejection, or social discrimination of people with a characteristic regarded as undesirable (Corrigan & Wassel, 2008). It has been conceptualized in terms of knowledge (ignorance or misinformation), attitude (prejudice or stereotyped beliefs that predispose people to express a certain emotional response) and behavior (discrimination or rejection and social actions involving the marginalization and/or exclusion of those with mental health problems) (Thornicroft, 2007; Graham et al., 2003; WHO, 2002).
These stereotypes have a range of adverse effects, compromising care quality and creating care gaps (Corrigan & Wassel, 2008; Henderson et al., 2014). A systematic review revealed stigmatizing beliefs, discriminatory attitudes, and behaviors among health professionals in both primary and specialized care, and in high- as well as low- and middle-income countries (Henderson et al., 2014). Even if people with mental illnesses overcome barriers and seek help, they are likely to encounter stigmatizing beliefs, negative attitudes, and discrimination, which hinder their care. Understanding stigma towards mental illness from the perspective of healthcare professionals is therefore a critical step towards reducing its impact on care gaps.
However, stigma towards mental health is poorly understood and research on stigma towards children’s mental health by health personnel is scant, partly due to the lack of valid, reliable instruments. A systematic review showed that although a number of instruments have been developed to assess mental health-related stigma among health professionals, there has been a lack of evaluation of certain measurement properties, with most studies focusing on the mental health of adults (Cardoso et al., 2020; Őri et al., 2023; Sastre-Rus et al., 2019). Likewise, although instruments have been developed and validated to measure stigma towards mental health in children, they have mainly focused on self-stigma (Kaushik et al., 2017), stigma in parents (Williams & Polaha, 2014), and peers (Nearchou et al., 2021), overlooking stigma in health personnel, with few exceptions (Davis et al., 2007).
Given the above, it is essential to have a valid, reliable instrument to identify the knowledge of doctors and health workers and their attitudes and discriminatory intent when they encounter these problems. The aim of this article is as follows:
Evaluate the reliability and construct validity of the Health Personnel Questionnaire
Determine the efficiency indicators of clinical vignettes of the Health Personnel Questionnaire in child psychiatrists and other groups of health professionals
METHOD
Study design and population
A cross-sectional, analytical study was conducted with purposive sampling. The sample comprised 2,477 health professionals, attached to 144 primary-level care centers in the 16 boroughs of Mexico City.
Instruments
Questions were included on demographic data such as sex, age, marital status, number of children, occupation, years of practicing the profession, education, main activity at the health center, and mental health training.
To measure stigma, the self-administered instrument developed by Berenice Pescosolido for the National Stigma Study–Children (NSS-C) was used. This study was part of an ongoing series of stigma-related modules within the 2002 General Social Survey (GSS) in the United States (Pescosolido, 2007; Pescosolido et al., 2008). The instrument was further refined, and in 2006, the GSS incorporated the Stigma in Global Context–Mental Health Study (SGC-MHS) component in a subsequent sample (Pescosolido et al., 2008).
The NSS-C is divided into three sections. The first section evaluates knowledge regarding mental illness in children and adolescents through a vignette technique originally developed in stigma research by the GSS. This technique includes vignettes depicting diagnoses such as depression and attention deficit hyperactivity disorder (ADHD), as well as control vignettes illustrating other conditions such as asthma and routine, subclinical issues (everyday discomfort). These vignettes were created by the team’s psychiatrist and subsequently reviewed by the team’s survey methodologist to ensure their suitability for survey use. The purpose of the vignettes was to determine whether respondents could identify cases of mental health disorders based on DSM-IV criteria.
The second and third sections are what we have called the Health Personnel Questionnaire. The second section examines attitudeproblems (prejudice) by exploring general opinions about children with mental health conditions, including perceptions of dangerousness, beliefs about the origins of these conditions, and views on available treatments, with particular attention being paid to the use of medication. The third section explores discrimination (discriminatory intent) by posing questions regarding the level of rejection respondents feel toward children with mental illnesses and their likely responses to hypothetical social interaction scenarios (Davis et al., 2007; Pescosolido, 2007; Pescosolido et al., 2007; 2008).
Based on a review of GSS studies and the extensive body of innovative and rigorous research derived from these findings (Pescosolido, Fettes et al., 2007; Pescosolido, Perry et al., 2007; McLeod et al., 2007; Perry et al., 2007), it was decided to adapt the vignette method and the Health Personnel Questionnaire (HPQ) for this study.
In accordance with the methodological framework of the NSS-C, 20 clinical vignettes were developed by a psychiatry professional, based on the results and experiences of a study conducted of a primary care population in Mexico City (Caraveo, 2016; Caraveo et al., 2011). Of the 20 vignettes, ten correspond to “cases” with sufficiently significant symptoms to indicate the presence of ADD, anxiety or depression and ten to “non-cases” with isolated symptoms, which were neither significant nor indicative of psychopathological problems (Caraveo, 2016).
The vignettes were subsequently reviewed by a psychiatrist and a child psychiatrist, who were asked to determine whether any elements or concepts should be added or eliminated to clarify the case presented in the vignette. The final vignettes were the result of agreement between the two specialists and a review by the research team comprising two psychologists and a psychiatrist. Finally, the vignettes underwent a blind review by seven expert child psychiatrists. Table 1 provides examples of vignettes categorized as “case” for mental health disorders and “non-case” vignettes. The full set of vignettes can be reviewed in Caraveo, (2016).
To measure prejudice and discriminatory intent, the HPQ was translated by an expert translator and reviewed by another three subject matter experts, who were asked to rate each item regarding its applicability to health providers, making suggestions if they thought any of the questions required modification. They reviewed the instructions for administering the adapted version, making the corresponding adjustments.
Finally, a group of 15 primary care physicians with similar characteristics to the final sample evaluated the clarity and understandability of the instructions, questions, and response options of the previously revised version. The questionnaire comprised 37 Likert-type items with five response options (from 0 not at all to 4 very much), distributed in two dimensions: prejudice (32 items) and discriminatory intent (five items). Examples of items include the following: “Medication for children with behavioral problems makes them act automatically (without thinking about what they are doing),” “How much of J/M’s situation is due to their lack of discipline at home?” “How likely do you think it is that J/M would commit a violent act toward themself?” “How likely would you be to spend an afternoon socializing with J/M’s family if they were your neighbors?”
Procedure
The study was conducted from August 1 to October 30, 2009. Thirty-one interviewers were trained in the objectives, structure and contents of the instrument, as well as the way to administer it. The authors worked with professionals who agreed to participate in the study. Each professional was given a questionnaire and a vignette, with care being taken to ensure that the 20 vignettes were uniformly distributed among the personnel surveyed. The questionnaire took 60 minutes to complete.
The research was conducted with the approval of the Research Ethics Committee of the Ramón de la Fuente Muñiz National Institute of Psychiatry, the directors of the participating primary health care centers and the informed consent of the professionals surveyed.
Statistical analysis
Frequency analyses and X2 and Mann-Whitney U tests were performed to describe the sample. Efficiency values (sensitivity, specificity, and positive and negative predictive value) were calculated for the clinical vignettes presented, with the assessment of the child psychiatrists being the gold standard.
An internal consistency analysis (Cronbach’s α) was conducted for construct validity, and the sample was randomly divided into two groups. Exploratory factor analysis (EFA) was conducted with the first group (n = 1250) and confirmatory factor analysis (CFA) with the second (n = 1227). The principal components method with oblique rotation method was used (Schermelleh-Engel et al., 2003) for exploratory factor analysis (EFA), showing that the Kaiser-Meyer-Olkin (KMO) sampling adequacy index was > .80 while Bartlett’s tests of sphericity had p < .05 (Hair et al., 2014).
The following indicators of goodness of fit were considered (Hu et al., 1999) for confirmatory factor analysis (CFA): 1) chi-square divided by degrees of freedom (quotients < 2 indicated good fit); 2) χ2/df: good fit is indicated by values of less than 2; 3) Comparative fit index (CFI): acceptable fit is given by values ≥ .90; 4) Root mean square error of approximation (RMSEA): acceptable fit is indicated by values ≤ .08 (90% CI ≤ .10) and good fit by values ≤ .05 (90% CI ≤ .08).
Analyses were conducted using SPSS Statistics 25 and Amos 24.0.
RESULTS
Sample characteristics
2,477 health professionals, the majority of whom were women (72.1). A total of 50.5% were ages 21 to 40 (X̄=39.8 SD=12.2), half were partnered, most were doctors or nurses (70.2%), 54.3 % had had 15 years or fewer of service, the majority had children (62.5%), and nearly a third reported having received mental health training. No significant differences were found between the groups used for EFA and CFA.
Efficiency of clinical vignettes
A high degree of agreement (97%) was found among child psychiatrists, who showed the highest sensitivity and the lowest rate of false negatives. All other groups of professionals were characterized by having more specificity than sensitivity. However, the highest sensitivity was found among mental health specialists, which was also the group with the highest positive predictive value (PPV). Total agreement of 34% was obtained overall (Table 3).
Exploratory Factor Analysis (EFA)
Five dimensions related to prejudice and one for discriminatory intent were identified. Together, all the dimensions explained 55.047% of variance.
The first factor, associated with prejudice, and reflecting subjects’ view of psychiatric medications, which were mostly adverse (seven items), was called Supply and Effect of Psychoactive Drugs (SEP).
The second factor, comprising opinions on the way children and adolescents are raised, including lack of discipline, poor education, exposure to violence in the media and games, as well as having a bad temper (five items), was called Type of Upbringing (TU).
The third factor, consisting of serious behaviors with the potential to harm, such as violence towards oneself and others (three items), was called Disruptive Behavior (DB).
Opinions on mental health problems such as social rejection and parental burden were grouped into the fourth factor (four items), called Treatment for Mental Health Problems (TMHP).
The fifth factor comprises opinions on the origin of mental illness, such as hereditary factors, chemical alterations, and their management through drugs (three items), in other words, Biological Causes (BC).
Finally, the sixth factor, consisting of statements about rejection and avoidance related to intent to discriminate (four items), was called Avoidance (A).
Alphas per factor ranged from α=.636 to α=.856 while that of the total scale was α=.783, as can be seen from Table 4.
Total-Element Statistics
Confirmatory Factor Analysis (CFA)
CFA was conducted for the model obtained in the EFA in half the sample (n = 1227). The six-factor structure identified through the EFA (one factor for discriminatory intent and five for prejudice) was corroborated by the CFA, with similar item factor loadings and alphas. No items were eliminated. Figure 1 shows the standardized factor coefficients, and the fit indices of the model with 26 items. All factor loadings were significant, ranging from .44 to .88. The model fit was adequate, with values within the recommended standards (X2 = 860.555, df = 284, p < = .000, CFI = .919, RMSEA = .041, 90% CI = .038-.044).
DISCUSSION AND CONCLUSION
This research sought to evaluate the reliability and construct validity of the health personnel questionnaire (HPQ) in a sample of primary health care professionals in Mexico City, and to determine the efficiency indicators of the clinical vignettes it contains to obtain a useful tool for examining problems of knowledge, attitude and behavior as the constituent features of stigma towards mental health problems in children and adolescents (Caraveo, 2016).
A PPV of 35% was obtained in first-contact doctors, above the rate found in other studies (WHO, 2006; Zwaanswijk et al., 2005). The efficiency of the clinical vignettes, based on the assessment of the group of child psychiatrist experts, was highly satisfactory, validating the contents presented. Assessments were more specific than sensitive for the study population in general as well as for the groups of professionals. This result is expected in diagnostic tests (Shreffler & Huecker, 2023).
The results of the validity and reliability analysis applied to the Health Personnel Questionnaire showed that the scale proved effective in exploring problems of attitude (prejudice) and discriminatory intent as elements of stigma towards mental illness in children and adolescents. The internal consistency of the global scale obtained a Cronbach’s alpha of .8 and factor/dimension values between .63 and .85, considered adequate. Regarding the dimensions obtained, attitudes towards the supply and effect of psychotropic drugs, type of upbringing, disruptive behavior, psychiatric treatment, biological causes, and discriminatory intent are all indicators commonly used in research on stigma towards mental illness in minors (Thornicroft, 2007; Graham et al., 2003; WHO, 2002). The results obtained are consistent with the stigma literature on prevailing attitudes towards mental health problems (Thornicroft et al., 2016).
Identifying these dimensions is valuable, since they have the potential to be key discriminating elements in stigma towards mental illness in children, adolescents, and adults. Significant differences exist between adults and children that probably contribute to the differing perceptions of their mental health among health personnel. These include the importance placed on child-raising styles in the configuration of mental illness in children, as well as disruptive behavior. Since mental illness stigma has not been studied as extensively in children and adolescents as it has been in adults (Cardoso et al., 2020), having a valid instrument will facilitate its evaluation.
Primary care health personnel are often the first point of contact for families seeking help with children’s mental health problems. If these professionals hold stigmatizing perceptions, they may underestimate or dismiss symptoms, potentially delaying diagnosis and access to appropriate, quality treatment (Mkubwa et al., 2024). Having this measurement tool meets the need for a clearer understanding of the stigmatization processes children and young people are subjected to by health personnel. It makes it possible to propose specific, effective interventions targeting this population, since stigma can be modified and there are reports of its reduction and management through training programs for health personnel (Vila-Badia et al., 2016; Villamil-Salcedo et al., 2017).
Scope and limitations. It was not possible to evaluate the criterion validity of the HPQ due to the lack of a gold standard to assess mental health-related stigma in this population. However, it was possible to calculate the efficiency values for the clinical vignettes presented, through the evaluations of child psychiatrists serving as the gold standard. Nevertheless, more psychometric studies on measurement error and responsiveness parameters are still required. Another limitation of this study is that it did not include vignettes covering all the possible mental disorders children may experience, and the fact that the scale does not directly measure stigmatizing behaviors. Moreover, stigma is a complex, evolving concept, with new terms and strategies emerging for its measurement (Porfyri et al., 2022; Mora-Ríos et al., 2013; Mkubwa et al., 2024; Thornicroft et al., 2016). Nevertheless, the HPQ represents an important step towards advancing the evaluation of this construct and enhancing its administration across diverse contexts and populations.
Implications for mental health programs and actions. Stigma associated with mental health disorders in children and adolescents cannot be directly extrapolated from studies on adults, nor inferred from research on the stigma related to physical illnesses and developmental disabilities in children. Having a validated instrument to address this issue is a crucial step toward enhancing the understanding of mental health stigma in this population, particularly in Spanish-speaking countries such as Mexico. Moreover, it enables the evaluation of the way stigma among health professionals influences the facilitation or hindrance of access to care and recovery for children with mental disorders, as well as the design of targeted actions for their care. In future work, we will present an analysis of the interrelation among the three components of perceived stigma across groups of health professionals, which will provide further insights into their perceptions of mental illness.
In conclusion, this questionnaire has the potential to assess stigma towards child and adolescent mental health among healthcare professionals, constituting a promising tool for evaluating the resulting barriers to care.
STATEMENTS