Original Article

WHO survey tool for behavioral insights on COVID-19: A psychometric study for translation, adaptation and content validity in Persian

Introduction

Healthy behaviors are absolutely critical in the fight against the COVID-19 pandemic.¹ They are essential in supporting the immune system, preventing underlying health conditions, reducing transmission, and promoting overall well-being.² It is crucial to understand that wearing a mask, practicing good hand hygiene, maintaining physical distance, keeping isolation/quarantine, respiratory hygiene/etiquette, cleaning surfaces, and avoiding touching the T-zone are healthy behaviors that can prevent the transmission of COVID-19 to others.³ Individuals with underlying health conditions are at an extremely high risk of severe illness from COVID-19.⁴ Therefore, it is imperative that everyone adopt healthy behaviors to protect themselves and their communities from infection.⁵

The COVID-19 pandemic needs strenuous efforts to adapt individual behaviors to achieve population-level prevention.⁶ Understanding people’s experiences and intentions is thus essential for managing this unprecedented public health crisis.¹ The WHO has encouraged its member states to conduct surveys to monitor the knowledge and behavior of the population about pandemic response measures. Following this issue, divers surveys and investigations were started and developed globally.⁷

Monitoring population behavior comprising their risk perceptions, knowledge, attitude, self-efficacy, trust in institutions, and trust in/use of information sources are crucial in evolving policies to control COVID-19 transmission.^8,9 The World Health Organization Survey Tool for Behavioral Insights on COVID-19 (WHO-STBIC) has been validated through the six rounds of data collection in Germany, which led to adjustments of the questionnaire. WHO has recommended translating, adapting, and validating this tool by expert translators familiar with the terminology of COVID-19 and behavioral science in various countries to be used at national levels.¹⁰

WHO-developed tools can provide reliable information, data, and evidence in various settings.⁸ As the tool developed by WHO experts through scientific approaches, WHO-STBIC can be used by countries at the national level to achieve reliable and comprehensive data on public insights on COVID-19.⁸ Moreover, modification, validation, and use of this tool in various settings may lead to origination tool optimization.⁸

Scientists have employed both indirect and direct measurement techniques to study the factors that influence preventive behaviors. Indirect methods have typically relied on morbidity/mortality as a substitute for preventive behaviors, assuming that lower morbidity/mortality suggests higher levels of preventive behaviors. Conversely, direct methods have involved collecting self-reports of preventive behaviors from individuals or using objective measures like mobility patterns tracked by mobile phones.¹¹

Various tools have been used to measure public behavior and insight on COVID-19.^12-16 Population behavioral insights on COVID-19 are the foundation of public health response within a given society, and measuring population behaviors is an important area of research. The WHO-STBIC is an attractive tool to measure population behavioral insights, it is a flexible tool, which makes it more feasible for use. However, a validated tool remains unavailable in Persian.

The lack of a survey tool for behavioral insights on COVID-19 raises concerns due to language barriers, potential for inaccurate data collection, and exclusion of certain populations. These concerns could lead to unreliable information and limit the comprehensiveness and representation of insights gathered. It is crucial to address these concerns in order to develop a comprehensive and inclusive survey tool for capturing behavioral insights related to COVID-19. Therefore, the current study aimed to translation, adaptation and validating WHO-STBIC⁸ among Iranian adult population.

Methods

This is part of a national research project on developing a monitoring package for COVID-19 in Iran. The original English version of WHO-STBIC⁸ collects information about insights and behaviors on COVID-19 across 21 dimensions, consisting of a total of 134 items. Figure 1 describes steps of conducting of the present study.

Close

Figure 1.

Flow chart for translation, content validity and adaptation of WHO-STBIC in Iran

Figure 1.

Flow chart for translation, content validity and adaptation of WHO-STBIC in Iran

Forward and back-translate the English text to Persian and then back to English of the WHO-STBIC

It carried out an iterative, strict, and precise translation procedure in accordance with the five-step translation standard advised by the WHO including forward translation, expert panel, back translation, pre-testing, cognitive interviewing, and final version.¹⁷ The WHO-STBIC was independently translated from English into Persian by two translators, leading to two versions that were integrated after resolving disagreements through discussion. A preliminary Persian version emerged and was back-translated by two other experts in English and the field (epidemiologist). The original and produced English versions were compared by four translators and two research team members. Ultimately, the Persian version of the tool was established.

Adaptation

To consider Iran’s cultural, social, and religious characteristics and promote linguistic aspects of the items content, an expert panel session was held and required editions were made based on the experts’ opinions. Moreover, through the adaptation process, some questions were removed/modified/added to the questionnaire due to either feasibility or cultural adaptation reasons, so the original version was shortened through adaptation.

Content validity

To determine the content validity of the questionnaire, two methods were used, including qualitative (experts’ opinions and suggestions) and quantitative (content validity ratio [CVR] and modified kappa [mK]) methods. At this stage, the content validity form was provided to the experts through e-mail, and they were asked to evaluate each item based on the two criteria of “relevance” and “necessity”. The scoring method for relevancy was like 1 (not relevant), 2 (item needs some revision), 3 (relevant but needs minor revision), 4 (very relevant), and for Necessity were like 1 (not necessary), 2 (useful but necessary), 3 (very necessary). According to the experts’ opinions, CVR and mK were calculated for each dimension of the questionnaire and the whole questionnaire. To calculate the CVR, the criterion of necessity in the form of a 3-part Likert scale was used and calculated based on the following formula.

nE: Number of experts who have responded to the “necessary” option

N = Total number of experts

Multi-rater kappa coefficient was introduced by literature, which adjusts for chance agreement. mK coefficient was calculated using the formula reported by Sim et al. 2005.¹⁸

Kappa = P₀−P_c/1−P_c

Where P_o is the proportion of observed agreements and P_c is the proportion of agreements expected by chance. Experts’ opinions and recommendations on the item’s content were reviewed by the research team and required revisions were made.

Pilot study

A pilot study was conducted to find out the tool feasibility in Tabriz, East Azerbaijan, Iran. Considering diverse socio-economic conditions, 100 samples have participated in study with sampling based on previous studies’ sampling framework. Questionnaires were completed through interviews with educated interviewers. In addition to the completion of items, challenges, and unpleasant senses during the questionnaire filling were reported by interviewers.

Developing a shortened through adaptation of Persian WHO-STBIC

The WHO has recommended adapting the WHO-STBIC tool to customize the questionnaire for particular circumstances in different countries.⁸ A shortened through adaptation version of the Persian WHO-STBIC was developed to address the long completion time of the original questionnaire. Experts used criteria such as item similarity, merging possibility, and simplification to shorten the tool. The shortened version consists of 90 items in 17 subscales. The internal consistency was evaluated using Cronbach’s alpha. Expert panel members’ inclusion criteria included individuals with related academic fields of study and identified experiences in tool development and also COVID-19 management in Iran encompassing epidemiology, infectious disease, health services management, health policy, psychology, health education, occupational health, and public health (7 males and 2 females).

Softwares

Data were analyzed using STATA version 15 (STATA Corp, College Station, TX, USA).

Ethics approval and consent to participate

Participants were informed of the study’s purpose and guaranteed anonymity by the researcher. An informed consent form was provided for participants to affirm their assent. Their consent was shown by completing the form and the study. Participation in the study was voluntary, as explicitly stated in the questionnaire’s introduction. The study results would be kept anonymous and confidential and only used for research purposes. All methods followed the guidelines of the Declaration of Helsinki (DoH).

Results

The original Persian version of WHO-STBIC

The original Persian version included 124 items in 20 subscales. Some items were merged based on experts’ opinion and their content similarity, and some were omitted. In addition, six items related to restriction protocols were added.

The shortened through adaptation Persian version of WHO-STBIC

This version consists of 17 subscales with 90 items. In this version of the WHO-STBIC, 34 items were removed including two items in the probability and severity subscale, four items in health literacy, two items in the prevention own behavior’s subscale, two items in the affect subscale, one item in subscale of trust in sources of information, two items in perceptions, one item in resilience, one item in the fairness, one item in the institutional trust, two items in COVID-19 vaccine. Moreover, the use of sources and frequency of information subscales which contains 11 items and wellbeing subscale with five items were removed completely.

Content validity index and internal consistency for shortened through adaptation of Persian version of WHO-STBIC

In terms of content validity, the mK index was calculated as excellent for 86 percent (n = 78) of items (mK > 0.9). The 12 items in the socio-demography, probability and severity, policies, interventions, and fairness subscales were not taken into account for determining content validity. The mean CVI of the whole scale was 0.87 ranging from 0.46 to 0.86 for 17 subscales, indicating acceptable content validity. The internal consistency of the whole scale was calculated to be 0.86. Content validity and internal consistency for whole scale and subscales of WHO-STBIC values are tabulated in Table 1.

According to the pilot study, the average time to fill the original Persian version of WHO-STBIC and shortened through adaptation Persian version was 35 and 15 minutes, respectively. Supplementary file 1 reports the items in each version and also items that were modified or added. Items omission and merging to develop shortened through adaptation Persian version was done based on the similarity of their contents, the necessity to be assessed in rapid assessment, and the richness of gathered data for decision making to reach a time-saving and simple version. Moreover, Items were added to the restriction subscale based on Iran’s particular protocols during COVID-19.

Discussion

The develped shortened through adaptation Persian version of WHO-STBIC with 90 items still have to many items which need to be considered for minimizing in future research. The mean CVI of the total scale was 0.87 for the 17 subscales in Persian version, indicating acceptable and satisfactory content validity of the tool. Moreover, this version of the tool was developed to provid a time-saving and more simple instrument to be used as assessment tool.

Continuous monitoring of these behavioral and psychosocial factors throughout the pandemic can assist in recognizing evolving needs and changes, especially crucial for individuals with chronic diseases.¹⁹ Monitoring and assessment of public insights on COVID-19 and employing proper strategies to respond and manage public behavior is vital during the epidemic and is recommended by WHO.^20-22 These investigations not only reveal the challenges of public adherence to non-pharmaceutical measures but also result in the development of potentially effective vaccine acceptance and uptake mechanisms.²⁰

It is crucial to understand community behavior to effectively implement preventive measures such as mask-wearing, gatherings limitations, etc.^23,24 Public health strategies development should benefit from community behavior knowledge to be effective and successful, nevertheless, these strategies will be disobeyed by the public which will decrease the mutual trust.^9,25

Regarding the mentioned issues, previous studies have introduced some tools to monitor public behavior during COVID-19 in diverse views including knowledge, attitude, practice, insight, perception etc, are related to public health measures. Creating cross-context synthesis analyses, offering essential insights for the ongoing response effort as well as post-outbreak evaluation, sharing of lessons learned, and the ongoing effort to better understand effective crisis response processes were raised as the added value of published tools.²⁶ WHO as an international health authority has published WHO-STBIC instrument to study public insights and perceptions on COVID-19, which was adapted and content validated in the Persian language in the present study. This tool provides a comprehensive understanding of various components regarding public behavior during the epidemic. Compared with previously published tools in this field,^13-15,27-29 WHO-STBIC was more complete and inclusive. However, to study each behavior in detail, it is required to use its especial tool while WHO-STBIC could be used as a preliminary screening tool, too.

Even though WHO-STBIC has been introduced as a rapid and simple tool,⁸ results of this study were not confirmed. The long time needed to fill out the questionnaire which led to an unpleasant sense in participants was a challenge during the feasibility pilot study. Although the original version is more suitable for profound assessment, it is costly and time-consuming due to its length. So, the shortened through adaptation Persian version of the tool was developed. This version focuses more on shifting behaviors that change in a short period, and does not include items such as the COVID-19 stigma, which is less variable in a short period.

Although shortened through adaptation Persian version takes less time to respond, and the information provided is not as detailed as the original version. The data presented here appear to support the findings that an original version as a parallel tool with other tools is difficult to implement. To facilitate data collection, we decided to reduce the number of items according to the recommendations of the WHO for quick evaluations. Considering that this tool is still long, it seems still that the Persian version needs to extract the short version. Rapid data gathering and sharing could help authorities, health workers, journalists, and the public to communicate effectively to inspire appropriate behavior change, manage the crisis, and protect public trust.²⁶

The tool’s simplicity was acceptable but, it is hard to say it is simple to use for all situations. To assess simplicity, it is necessary to investigate in future studies. We need to have a comprehensive and full package that has original, adapted, and, short Persian version tools to respond to various situations. Countries in pandemic situations require instruments that can be quickly implemented; used on a regular schedule; simple and adaptable enough to adapt to the situation; and cost-effective, especially for low- and middle-income countries.²⁶ This tool is flexible to adjust to the diverse situations. This tool can also be used online or on other platforms in cross-cultural surveys. Human behaviors differ in populations, inter-population, between persons, and inter-persons.³⁰ This tool, as a flexible tool, can examine people’s behavior about COVID-19 in different situations.

It seems that in this tool less attention has been paid to some aspects of behaviors related to COVID-19, such as the quality of masks and traveling in the main tool. Specific tools related to behaviors affecting COVID-19, such as masks, travel, etc, are required,³¹ so it is better to generate special tools as well. Change in public insight is a time-consuming process, while behavior could be changed in a short period.³² Considering the issue, measuring interventions’ effects on behavior change is easier than the effect on insights while the tool is more focused on insights. Most of the items in this tool measure people’s behavior and insights, and these measurements are less focused on government and macro actions.³⁰

It is recommended to determine factor structure of the WHOSTBIC tool which is not currently reported neither for the original tool or translated versions. These tools can be utilized by researchers to guarantee that data collection is culturally suitable, pertinent, valid, and dependable.

Limitations

Due to the COVID-19 pandemic’s unexpected and severe impact, public participation in order to conduct main study was low. The tool also gathers information on self-reported behavior and insights about COVID-19. These are known to vary from actual behavior at times, due to the social desirability effect. As a result, this limitation must be considered while evaluating the results. It is recommended to investigate the complete psychometric properties of the tool including its factor structure both for the original and translated versions. Considering the high number of items and complexity of the structure of the tool this needs to be reported as a separate original publication through future research.

Conclusion

WHO survey tool for assessing public insight on COVID-19 Persian version was adapted and content validated with minimum changes to be used in Iran. In comparison with previous instruments developed in Iran in this field, the WHO-STBIC Persian was more inclusive and compatible. However, to in-depth measure of the components such as mask usage behavior etc., special tools need to be developed while this tool could be used as a preliminary screening tool. On the contrary with declared characteristics of the tool, study results revealed that it was hard to say that this tool is a fast one. So, the shortened through adaptation Persian version was developed to save the time.

Competing Interests

The authors declare that they have no competing interests.

Ethical Approval

The current study was approved by the ethics committee of the National Institute for Medical Research Development NIMAD in Tehran, Iran; ethical code: IR.NIMAD.REC.1399.120.

Supplementary Files

References

1. Soleimanpour H, Sarbazi E, Davtalab Esmaeili E, Mehri A, Ghaffari Fam S, Nikbakht HA. Predictors of receiving COVID-19 vaccine among adult population in Iran: an observational study. BMC Public Health 2023; 23(1):490. doi: 10.1186/s12889-023-15409-0 [Crossref] [ Google Scholar]
2. Koulierakis G, Dermatis A, Zavras D, Pavi E. Protective behaviors during COVID-19 confinement measures in Greece: the role of anxiety, perceived risk and risky-choice framing. AIMS Public Health 2023; 10(2):281-96. doi: 10.3934/publichealth.2023021 [Crossref] [ Google Scholar]
3. Vasli P, Shekarian-Asl Z, Zarmehrparirouy M, Hosseini M. The predictors of COVID-19 preventive health behaviors among adolescents: the role of health belief model and health literacy. Z Gesundh Wiss 2024; 32(1):157-66. doi: 10.1007/s10389-022-01808-x [Crossref] [ Google Scholar]
4. Davtalab Esmaeili E, Azizi H, Sarbazi E, Khodamoradi F. The global case fatality rate due to COVID-19 in hospitalized elderly patients by sex, year, gross domestic product, and continent: a systematic review, meta-analysis, and meta-regression. New Microbes New Infect 2023; 51:101079. doi: 10.1016/j.nmni.2022.101079 [Crossref] [ Google Scholar]
5. Firouzbakht M, Omidvar S, Firouzbakht S, Asadi-Amoli A. COVID-19 preventive behaviors and influencing factors in the Iranian population; a web-based survey. BMC Public Health 2021; 21(1):143. doi: 10.1186/s12889-021-10201-4 [Crossref] [ Google Scholar]
6. Van den Broucke S. Why health promotion matters to the COVID-19 pandemic, and vice versa. Health Promot Int 2020; 35(2):181-6. doi: 10.1093/heapro/daaa042 [Crossref] [ Google Scholar]
7. Glasziou PP, Sanders S, Hoffmann T. Waste in COVID-19 research. BMJ 2020; 369:m1847. doi: 10.1136/bmj.m1847 [Crossref] [ Google Scholar]
8. World Health Organization (WHO). Survey Tool and Guidance: Rapid, Simple, Flexible Behavioural Insights on COVID-19: 29 July 2020. WHO; 2020.
9. Sarbazi E, Sadeghi-Bazargani H, Farahbakhsh M, Ala A, Soleimanpour H. Psychometric properties of trust in trauma care in an emergency department tool. Eur J Trauma Emerg Surg 2023; 49(6):2615-22. doi: 10.1007/s00068-023-02348-z [Crossref] [ Google Scholar]
10. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006; 6:44. doi: 10.1186/1472-6963-6-44 [Crossref] [ Google Scholar]
11. Dang J, Jia L. Validity issues in measures of COVID-19 preventive behaviours. J Glob Health 2023; 13:03026. doi: 10.7189/jogh.13.03026 [Crossref] [ Google Scholar]
12. Bagi HM, Soleimanpour M, Abdollahi F, Soleimanpour H. Evaluation of clinical outcomes of patients with mild symptoms of coronavirus disease 2019 (COVID-19) discharged from the emergency department. PLoS One 2021; 16(10):e0258697. doi: 10.1371/journal.pone.0258697 [Crossref] [ Google Scholar]
13. Kumari A, Ranjan P, Vikram NK, Kaur D, Sahu A, Dwivedi SN. A short questionnaire to assess changes in lifestyle-related behaviour during COVID 19 pandemic. Diabetes Metab Syndr 2020; 14(6):1697-701. doi: 10.1016/j.dsx.2020.08.020 [Crossref] [ Google Scholar]
14. Chopra S, Ranjan P, Malhotra A, Sahu A, Dwivedi SN, Baitha U. Development and validation of a questionnaire to evaluate the impact of COVID-19 on lifestyle-related behaviours: eating habits, activity and sleep behaviour. Public Health Nutr 2021; 24(6):1275-90. doi: 10.1017/s1368980020004656 [Crossref] [ Google Scholar]
15. WHO/Europe. Coronavirus Disease (COVID-19) Outbreak - Survey Tool and Guidance: Behavioural Insights on COVID-19. 2020. Available from: https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/publications-and-technical-guidance/risk-communication-and-community-engagement/who-tool-for-behavioural-insights-on-covid-19/survey-tool-and-guidance-behavioural-insights-on-covid-19-produced-by-the-who-european-region.
16. Rezaei M, Sadeghi-Bazargani H, Razzaghi A, Jafari-Khounigh A, Heydari ST, Entezarmahdi R. Validity and reliability of short-form travel behavior questionnaire during COVID-19 pandemic (TBQ-Cov19). J Transp Health 2023; 29:101586. doi: 10.1016/j.jth.2023.101586 [Crossref] [ Google Scholar]
17. World Health Organization (WHO). Process of Translation and Adaptation of Instruments. 2021. Available from: http://Www.Who.Int/Substance_Abuse/Research_Tools/Translation/En. Accessed May 17, 2021.
18. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005; 85(3):257-68. [ Google Scholar]
19. Hasenpusch C, Matterne U, Tischer C, Hrudey I, Apfelbacher C. Development and content validation of a comprehensive health literacy survey instrument for use in individuals with asthma during the COVID-19 pandemic. Int J Environ Res Public Health 2022; 19(4):1923. doi: 10.3390/ijerph19041923 [Crossref] [ Google Scholar]
20. World Health Organization (WHO). Behavioural Considerations for Acceptance and Uptake of COVID-19 Vaccines: WHO Technical Advisory Group on Behavioural Insights and Sciences for Health, Meeting Report, 15 October 2020. WHO; 2020.
21. World Health Organization (WHO). Social and Behavioural Insights COVID-19 Data Collection Tool for Africa. WHO Regional Office for Africa; 2021.
22. Jafari-Khounigh A, Haghighi M, Razzaghi A, Habibzadeh S, Heydari ST, Entezarmahdi R. Development and validation of a specific questionnaire for measuring the attitude and behavior of mask use in pandemics with the dominant airborne pattern: the experience of COVID-19. Iran J Epidemiol 2023; 19(3):177-92. [ Google Scholar]
23. Tam VC, Tam SY, Khaw ML, Law HK, Chan CP, Lee SW. Behavioural insights and attitudes on community masking during the initial spread of COVID-19 in Hong Kong. Hong Kong Med J 2021; 27(2):106-12. doi: 10.12809/hkmj209015 [Crossref] [ Google Scholar]
24. Rostami Z, Abbasian M, Sarbazi E, Soleimanpour H, Mostafaei H, Ghasemi M. Features influencing older adults’ quality of life and their association with loneliness and activities of daily living: a cross-sectional study in Iran. Int J Aging 2023; 1(1):e24. doi: 10.34172/ija.2023.e24 [Crossref] [ Google Scholar]
25. Cecchi-Dimeglio P, Luu D, Cabon Y, Kitson-Pantano R. Comparative Analysis of the Application of Behavioural Insights of 33 Worldwide Governments on the Landing Pages of their COVID-19 Official Websites and their Impact on the Growth Scale of the Pandemic. 2020. Available from: https://ssrn.com/abstract=3601976.
26. Betsch C, Wieler LH, Habersaat K. Monitoring behavioural insights related to COVID-19. Lancet 2020; 395(10232):1255-6. doi: 10.1016/s0140-6736(20)30729-7 [Crossref] [ Google Scholar]
27. Ranandeh Kalankesh R, Mohammadian E, Ghalandari M, Delpasand A, Aghayari H. Health information seeking behavior (HISB) among the university students. Front Health Inform 2019; 8(1):e13. doi: 10.30699/fhi.v8i1.189 [Crossref] [ Google Scholar]
28. Liu PL. COVID-19 information seeking on digital media and preventive behaviors: the mediation role of worry. Cyberpsychol Behav Soc Netw 2020; 23(10):677-82. doi: 10.1089/cyber.2020.0250 [Crossref] [ Google Scholar]
29. Chopra S, Ranjan P, Singh V, Kumar S, Arora M, Hasan MS. Impact of COVID-19 on lifestyle-related behaviours- a cross-sectional audit of responses from nine hundred and ninety-five participants from India. Diabetes Metab Syndr 2020; 14(6):2021-30. doi: 10.1016/j.dsx.2020.09.034 [Crossref] [ Google Scholar]
30. Lunn P, Belton C, Lavin C, McGowan F, Timmons S, Robertson D. Using Behavioural Science to Help Fight the Coronavirus. ESRI Working Paper No. 656. Dublin: The Economic and Social Research Institute (ESRI); 2020.
31. Karaivanov A, Lu SE, Shigeoka H, Chen C, Pamplona S. Face masks, public policies and slowing the spread of COVID-19: evidence from Canada. J Health Econ 2021; 78:102475. doi: 10.1016/j.jhealeco.2021.102475 [Crossref] [ Google Scholar]
32. Asiamah N, Opuni FF, Mends-Brew E, Mensah SW, Mensah HK, Quansah F. Short-term changes in behaviors resulting from COVID-19-related social isolation and their influences on mental health in Ghana. Community Ment Health J 2021; 57(1):79-92. doi: 10.1007/s10597-020-00722-4 [Crossref] [ Google Scholar]