Validation of questionnaire in health research and Quality Criteria Assessment

Introduction:

Research is a careful investigation or inquiry, especially through scientific method to search for new facts or to verify the established facts. Surveys are one of the most widely used tools in research.¹ usually, in a survey the data is collected using a questionnaire/ proforma. Surveys are carried out to achieve specific objectives. A survey carried out systematically provides a broad overview of the health status (example morbidity and mortality status of a population) and supply comparableeasyto-analyze data. However, while designing the survey one needs to be vigilant as wrong wording and poor question construction decreases the reliability of resulting data obtained from it. The number of health status questionnaires has increased dramatically over past decades. Though many questionnaires are being used for collecting data in research studies, the metric properties of research evaluation instruments have received little attention.² In this regard the present article is designed to discuss the important criteria that are essential to determine the methodological quality for development and evaluation of questionnaires in medical research.

(i) How quality assessment criteria’s of questionnaire was evolved

The US National Council on Measurement in Education (NCME), refers to psychometrics as the field in psychology and education that is devoted to testing, measurement, assessment, and related activities.³ A measurement in health research means a systematic, replicable process by which objects or events are quantified and/or classified with respect to a particular dimension usually achieved by the assignment of numerical values. For an instrument the validation process of its measurement properties is relatively simpler as compared to validation of a questionnaire. Therefore some criteria for good measurement properties are needed to legitimize what best the questionnaire is. The medical outcome trust is a non- profit organization dedicated to improving health care by promoting the science of outcomes measurement. The trust was incorporated in Massachusetts in 1992 as a public service organization.⁴ The mission of the Medical Outcomes Trust (MOT) is to promote the universal adoption of health outcome assessment in health care. Early in 1994, the Board of Trustees (BOT) of the Medical Outcomes Trust (MOT) appointed a seven member Scientific Advisory Committee (SAC) to establish measurement standards for health outcomes measures and to serve a “certifying” function for outcomes measuring instruments submitted to the Trust for consideration. SAC has established eight criteria to evaluate instruments which includes These include (a) conceptual and measurement model, (b) validity, (c) reliability, (d) responsiveness, (e) interpretability, (f) respondent and administrative burden, (g) alternative forms, and (h) cultural and language adaptations (translations)⁵ . However, Caroline B. Terwee et al in 2007 further refined the available quality criteria for studies on the development and evaluation of health status questionnaires, including explicit criteria for the following measurement properties: (1) content validity, (2) internal consistency, (3) criterion validity, (4) construct validity, (5) reproducibility, (6) responsiveness, (7) floor and ceiling effects, and (8) interpretability^.6 In this article, each of these above mentioned criteria by Caroline B. Terwee is described, In addition to that (9) linguistic and cultural validity is also discussed. These principles are fundamental cornerstones of the scientific method for tool validation.

(ii) Refined criteria for Measurement Properties of a questionnaire

Validity means how well you are measuring what you intend to measure. In other words validity refers to getting results that accurately reflect the concept being measured.⁷ Eg: Does a high Quality of Life score (QOL) for a person really means that the person is in a good state of well being? So validity is the degree of confidence we can place on the QOL score for achieving our intention. In a survey, we often use the term construct which refers to an underlying theme, abstract idea,that one wishes to measure using survey questions.⁸ The criteria for validation as given by per Caroline B. Terwee et al is been described below with few modifications

1) Content validity - How well the items in a questionnaire is able to represent a certain specified area of content or the extent to which the items are fairly representative of the entire domain the test seeks to measure.⁹ Face validity is different from content validity Face validity may be more superficial, however it helps to improve the objectively determined validity – wording, structure of the test contents, its appropriateness, sensibility, or relevance. It's the degree to which test respondents view the content and the relevance of the context in the questionnaire.¹⁰

2) Reproducibility and Reliability - The variation in measurements made on a subject under changing conditions is known as Reproducibility.^11,12 Whereas,the degree to which the results obtained by a measurement and procedure can be replicated is known as Reliability. It’s the measure of extent to which items in a questionnaire (sub) scale are correlated. It can be assessed using statistical methods by measuring - internal consistency, Test retest reliability, Intra class correlation coefficient, Crombach’s alpha. Inter rater reliability can be calculated using Cohens kappa or weighted kappa for qualitative or categorical data. Agreement between two quantitative methods can be assessed using BladAltmans plot.

3) Criterion validity- criterion validity of a measurement technique is said to be present if its results are closely related to those given by some other, definitive technique(or a gold standard).It reflects the well established measurement procedure to create the new measurement procedure.¹³ There are two subtypes for criterion validity they are

3.1)Predictive validity- The extent to which a score on a scale or test predicts scores on some criterion measure. Example the validity of a IQ test for performance is the correlation between subject test scores and, for example, teachers ratings based on academic performance of the student. A high correlation would provide the evidence of predictive validity.¹²

3.2)Concurrent validity- The ability of the test to distinguish between groups. How clearly the test can distinguish between the groups that are very similar. Example - Covid 19 Vs pneumonia and SARI, for assessing manic-depression vs diagnosed manicdepression and those diagnosed paranoid schizophrenic.^14,15

4) Construct validity- Construct validity is related to generalizability of the measures.It is computed only when both content and criterion are not available. Convergent and discriminant validity are two necessary components of construct validity.

4.1)Convergent validity takes into consideration two measures that are supposed to be measuring the same construct and shows that they are related in reality also.

4.2)Discriminant validity, if the sets of measures seem to be related to two different constructs and you need to show that measures should not be related to each other are in reality also. Example Anxiety and depression and Selfworth and depression.¹⁶

5) Responsiveness- The ability of the instrument to detect clinically important differences or changes in the assessed construct over time is known as responsiveness. ^17,18,19

6) Floor and Ceiling effect- Floor effect can be present certain situations when large proportion of the study participants may perform very poorly on a task making it difficult to differentiate among the many individuals at that low level. A test results are too difficult for those taking it would show a floor effect because most people would obtain or be close to the lowest possible score of 0. On the other hand when the participants score towards the best possible score of the instrument ceiling effect is present.^20,21

7) Interpretability- Degree to which one can assign the values obtained through the application of the instrument and produce information relevant to the individual i.e; degree to which qualitative meaning can be assigned to quantitative scores in relation to the measured construct is Interpretability in psychometric research.^22,23

8) Linguistic and crosscultural validity - linguistic validation is to produce a translated version of a questionnaire in a foreign language,clear and easy to understand and which is conceptually equivalent to the original version. The translated instrument should maintain a reading and comprehension level that will be accessible by the respondents despite their education level. This validation is not applicable for all questionnaires .The process of linguistic validity involves the following steps 1.Forward translation 2. Reconciliation and 3. Backward translation.²⁴ Cross cultural validation is usually used to determine whether the items that were originally generated in a foreign language in a different culture are applicable , meaningful and equivalent in another culture.²⁵

Conclusion:

In this paper some important aspects for validation are discussed. Questionnaires are widely used in health research. The questionnaire needs to be rigorously tested so as to ensure that the data collected is meaningful. While planning any research study itself one has to ensure that the tool is validated. The quality of the data generated depends on the quality of the questionnaire hence validation is essential.