Validation of health screening questionnaire used for screening gastrointestinal disorder in worker's special health examination for night time work

Article information

Abstract

BACKGROUND

Night time work is related to several health and safety issues. In addition to causing work accidents [1], night time work also causes breast cancer [2], sleep disorders [3], cardiovascular disease [4], and type 2 diabetes [5]. Night time work may also cause disorders of digestive enzyme secretion [6], a disorder of acid-base balance, and eventually gastrointestinal (GI) disorder such as peptic ulcer and reflux disease [67]. Previous epidemiological studies also consistently report the association of night time work and GI disorder [8]. Moreover, functional GI disorders have also been reported to be associated with night time work [9].

Those adverse effects of night time work let the Korean government has added ‘night time work’ as a harmful factor of worker's special health examination (WSHE) since 2014 as a part of the policy to protect workers [10]. The purpose of WSHE is to prevent and make an early diagnose the occurrence of nervous, cardiovascular, GI, and endocrine disorder in workers who work at night. The WSHE consists of 2 steps. First, screening is done through the questionnaire items, and the second test is conducted when an abnormality is found in this item. In the case of GI disorder, the same method is used, and the second test of GI disorder is upper GI endoscopy. Although the first screening test of the WSHE is based on the questionnaire, it has not been tested whether the items in the questionnaire correctly plays a role in screening the target diseases since WSHE for night time work introduced in 2014. Therefore, in this study, we focus on the GI disorder and obtain the validity of the screening questionnaire item of the GI disorder (health screening questionnaire GI disorder [HSQGI]).

METHODS

Study population

The results of HSQGI and basic information among WSHE for 151,301 night shift workers performed in 7 centers (Jongno, Yeouido, Gangnam, Suwon, Gwangju, Daegu, and Busan) from 2014 to 2016 were collected. The results of each year's screening were considered to be independent because of the relatively short duration of treatment and due to the nature of GI disorder that respond well to treatment. Each study was considered as an individual study. The study was conducted for examinees who received opportunistic screening and WSHE simultaneously. Subjects who did not have adequate upper GI endoscopy due to residual food in stomach or overreacting by sedatives were excluded from the study. In addition, we also excluded cases in which the answer to the questionnaire was missing value. Finally, 5,057 examinees in 2014, 8,352 examinees in 2015, 10,587 examinees in 2016 were included in the main study. The selection of the final subject according to exclusion criteria is shown in Fig. 1.

Fig. 1

Flow chart of subject selection.

WSHE: worker's special health examination; GI: gastrointestinal; HSQGI: health screening questionnaire gastrointestinal disorder.

Endoscopic findings

Upper GI endoscopy results were classified as ‘non-specific’ and ‘GI disorder’ as 2 variables. The classification of upper GI endoscopy results was based on the criteria of Table 1. The criterion table was prepared by reflecting opinions of 4 physicians who are well-experienced in the judgment of WSHE for night time workers. In the case of gastritis, simple superficial gastritis, or simple erythematous gastritis, which does not cause any specific symptoms, is classified as non-specific when there is no other abnormal findings.

Table 1

Classification of endoscopic findings

Statistical analysis

Sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) of the WSHE items were obtained to evaluate whether the HSQGI could be used as a screening tool for GI disorder. Sensitivity was determined by the percentage of those who had GI disorder findings on the endoscopic examination and those who answered “dyspepsia” to the HSQGI. The specificity was calculated as the ratio of the subjects who showed non-specific findings on the endoscopic examination and those who responded normally to the HSQGI. The accuracy was determined by the percentage of subjects (true positive and true negative) who matched the results of the endoscopic examination and the HSQGI. PPV and NPV were obtained using the following formula:

PPV = Sensitivity × Prevalence(Sensitivity × Prevalence) + (1 − Specificity) × (1 − Prevalence)

NPV = Sensitivity × (1 − Prevalence)Prevalence × (1 − Specificity) + Specificity × (1 − Prevalence)

The prevalence was calculated as the ratio of the subjects who showed ‘GI disorder’ findings on upper GI endoscopy. All analyses were conducted using SAS statistical software, version 9.4 (SAS Institute Inc., Cary, NC, USA).

RESULTS

The general characteristics of the study population are as follows Table 2. The average age of the examinees in each year was 40.3 years in 2014, 41.5 years in 2015, 42.9 years in 2016. Among the target population, males accounted for 80%, and about one third of the respondents said that they are currently smoking. The proportion of people who drank over a week was 34.0% in 2014, 34.2% in 2015 and 29.5% in 2016. The average working time at the current job was about 9 years, and the percentage of people who worked for 12 years or more was the highest.

Table 2

General characteristics of the examinees of WSHE performed in specific health screening clusters

Table 3 shows the distribution of upper GI endoscopy abnormalities according to HSQGI responses. In all 3-year screenings, the number of people diagnosed with GI disorder by endoscopy was higher than those with Non-specific diagnoses. Nonetheless, the number of people confirmed with HSQGI dyspepsia was significantly less than that of normal people. Among the people who were confirmed with dyspepsia in HSQGI, the percentage of GI disorder was higher than those who were confirmed as non-specific in all 3-year screenings. The validity of the HSQGI based on the endoscopy is shown in Table 4.

Table 3

Distribution of GI disorder identified by upper GI endoscopy according to HSQGI results, 2014–2016

Table 4

Validity of HSQGI in WSHE

The sensitivities of ‘HSQGI’ for GI disorder confirmed by Upper GI endoscopy were 12.3% (95% confidence interval [CI], 11.1–13.4), 5.9% (95% CI, 5.2–6.5) and 6.0% (95% CI, 5.4–6.5), in 2014, 2015 and 2016, respectively. On the other hand, the specificity is relatively high at almost 90%, which is 88.6% (95% CI, 87.2–90.1), 93.6% (95% CI, 92.7–94.4), and 94.2% (95%:CI, 93.4–95.0), in 2014, 2015 and 2016, respectively.

Accuracy is reported to be 41.1% (95% CI, 39.8–42.5) in 2014, 38.6% (95% CI, 37.6–39.6) in 2015, and 34.2% (95% CI, 33.3–35.1) in 2016. Despite the high specificity due to the relatively low sensitivity, the accuracy was about 40%.

The usefulness, i.e., PPV and NPV of the HSQGI were 64.0% (95% CI, 60.2–67.8) and 38.0% (95% CI, 36.6–39.5) in 2014, 60.4% (95% CI, 56.2–64.7) and 37.2% (95% CI, 36.1–38.2) in 2015, and 68.7% (95% CI, 65.1–72.3), and 32.1% (95% CI, 31.1–33.0) in 2016, respectively. PPV and NPV were in the almost 60% and 40%, respectively.

DISCUSSION

The validity test for the HSQGI used in WSHE was performed with the medical records and questionnaire from 5,057, 8,352, and 10,587 subjects who received both upper GI endoscopy in opportunistic screening and WSHE simultaneously. The HSQGI in WSHE showed a sensitivity of 10%, a specificity of 90%, an accuracy of 40%, a PPV of 60% and a NPV of 30% for predicting the target diseases in upper GI system.

Among the validity of HSQGI, the specificity was high value of 90%, but the sensitivity was low as 10% so as not to meet the purpose of screening. A previous study for the validity of questionnaires through endoscopic results reported that the symptom questionnaire had a 92% sensitivity and 19% specificity for Gastroesophageal reflux disease [11], in addition, the other study reported the sensitivity and specificity of the symptom questionnaire (GerdQ) for Gastroesophageal reflux disease was 71.7% and 72,2% respectively [12]. The sensitivity and specificity of the symptom questionnaire, which targets peptic ulcer in the same way, has been reported to be 88% and 61%, respectively [13]. There might be several reasons for those low sensitivity of questionnaire in WSHE in the present study. We can first consider the high prevalence of chronic gastritis in Korea. According to a study of the subjects who visited for the purpose of health checkup, 85.9% of them had more than one endoscopic ‘gastritis’ finding [14]. High-prevalence of gastritis among Koreans is consistently reported in previous studies, which is attributed to high rate of Helicobacter pylori infection, high salt intake and excessive drinking. Many of these endoscopic gastritis is asymptomatic, and it is sufficient to observe without further treatment. However, this characteristic of asymptomatic gastritis confirmed by upper GI endoscopy was not considered in this study and it increased false negative by being classified as abnormal. If a definition of gastritis as a GI disorder was limited to gastritis requiring immediate treatment, or H. pylori-associated gastritis, sensitivity may have increased. However, the data used in this study did not contain such additional information in endoscopic findings. Second, a functional dyspepsia (FD) can be considered for increasing numbers of negative results. The design of this study used endoscopic findings as a gold standard in defining the GI disorder, which means that although FD is a GI disorder that WSHE wanted to screen, many FD cases misclassified to false positives because of their non-specific endoscopic findings. If the clinical procedure used to diagnose FD, which excludes the possibility of organic causes at first, as a gold standard in this study, the results may have changed. The cases that were classified as false positive will be reclassified as true positive, thus the sensitivity may increase. However, since this method is a clinical diagnostic process for symptomatic patients used in practical field, using that in WSHE does not satisfy the condition that screening should be considered cost-effective. Third, actual clinical process may be different from the guidelines. In this study, the guidelines of the Occupational Safety and Health Research Institute, were applied as HSQGI decision logic, as mentioned in the method above [10]. However, this is only a guideline, and the final judgment at the actual clinical field is left entirely to the physician in charge. Therefore, even if the subject responds at the HSQGI that he or she has no symptoms, which do not comply with the positive criteria in the guideline, an endoscopy may be implemented if the physician who has had the examination thinks that an endoscopy is necessary for the subject. Besides, in such cases, it is more likely be positive findings than the endoscopic cases conducted only with HSQGI. Thus, if these cases had been considered in study, the sensitivity might have increased. Nevertheless, despite these limitations, in this study, to reduce variability among physicians who examine subject in actual clinical fields as much as possible, a standard guideline was selected to HSQGI decision logic.

Nevertheless, it is obvious that the sensitivity of HSQGI is too low and the item needs to be improved. If so, the question is how much sensitivity should be increased. Generally, in order to increase the sensitivity, the specificity should be sacrificed, so the appropriate level is determined considering the severity and curability of the target disease [15]. The target disease of WSHE is GI disorder which includes gastric cancer to functional dyspepsia, which coverage is very wide and vague. For example, to screen for gastric cancer, sensitivity should be significantly increased even if the specificity is lowered because the false negatives should be reduced. However, sensitivity is not required for other target diseases, gastritis or functional dyspepsia as much as gastric cancer. Therefore, it is important to improve the questionnaire, but it should be preceded by clearly selecting the target disease of WSHE. Furthermore, these target diseases should include diseases such as ulcer, reflux, and functional dyspepsia that proven to be associated with night time work. However, of these 3 diseases, the 2 preceding diseases, i.e., the organic dyspepsia, are clearly diagnosed and the treatment method is reliable, so there is no problem in inclusion in the target disease. However, in the case of FD, it is necessary to reconsider whether it is included as target disease. FD does not meet the conditions for successful screening because pathophysiology is not well established and no treatment has been established [16]. In addition, the psychosocial factors such as stress are more likely to cause the disease [17], so rather than treating it as GI disease, it may be better to treat as a nervous system disease such as sleep disorder or a separate category. If this separation is made, the target disease of the GI disorder of WSHE can be clearly identified as an organic disease such as ulcer or reflux, and the sensitivity target and the item can be improved accordingly. In this study, we evaluated again the validity of HSQGI with reflux disease and ulcer as target disease. which was consistent low with the sensitivity of 10% (Appendixes 2 and 3). The improvement of the question seems to be certainly needed, and it can be based on many questionnaires already developed [111213]. For example, in order to rule out reflux disease, a GerdQ questionnaire that has already been validated through many studies may be added to the current HSQGI. While current HSQGI asked about a sore symptom vaguely thorugh only one question, the addition of the GerdQ questionnaire may provide a more specific approach to reflux symptoms by asking about sleep discomfort caused by symptoms, whether they are injected by symptoms, and how often they are reversed [12]. It may also be considered for the addition of specific question about family history related to ulcers disease or whether the symptoms are relieved by the ordinary meal as in the GLADYS questionnaire [13].

Some endoscopic findings of chronic gastritis such like simple superficial gastritis and simple erythematous gastritis were excluded. As mentioned above, we expected that false negatives would be overestimated due to the high prevalence of chronic gastritis in Koreans. Simple superficial gastritis alone does not cause any significant symptoms and is sufficient for follow-up observation without special treatment. These diseases are classified as non-specific because they are not considered to be within the GI disorder range that WSHE wants to find.

This study was conducted for those who receive opportunistic screening and WSHE at the same time. Unlike WSHE, opportunistic screening is not a mandatory test prescribed by law, and is a type of 'welfare within the company' provided by employers for the health of workers. Therefore, companies offering opportunistic screening are more likely to be workers-friendly than companies that do not, and more likely to be with well-established systems for worker health. Considering these features of opportunistic screening, bias may have occurred in the selection process of the subject. To determine whether this bias was present, we conducted a subgroup analysis. The best method for subgroup analysis is to compare the validation obtained by dividing into 2 groups: opportunistic screening group and only-WSHE group. However, as described above, validation can not be obtained in only-WSHE group because GI endoscopy is not performed unless an abnormality is found in HSQGI. Therefore, we analyzed the size of the business by considering it as a proxy for the degree of worker health friendliness of the company. In previous study [18], it was found that the health status of workers varied according to the size of the workplace. The analysis conducted by dividing workplaces with less than 50 employees, workplaces with more than 50 employees and less than 300 employees, workplaces with more than 300 employees, showed 10% lower sensitivity and 90% higher specificity as like the main result of this study (Appendix 4). It may reflect that the welfare-friendly aspects of opportunistic screening do not bias the research results.

The HSQGI consists of 6 individual items, however, we did not receive the data for each of the 6 individual items of HSQGI, and only data on dyspepsia according to the HSQGI logic mentioned above were received due to data registry of health screening cluster. Therefore, the accuracy of the subtypes of the diseases each item was seeking could not be determined.

This study was also not performed on all subjects who performed WSHE in Korea. Nonetheless, the tests were performed in 7 multi-center health examination clusters distributed across nationwide and might be a sufficient to show representative with sufficient subject numbers.

The endoscopic findings as the gold standard in this study were not a histopathological findings but a descriptive findings of the practitioner. Despite the high utility of upper GI endoscopy, there are differences among practitioners for unclear lesions [19]. Because of these limitations, biopsy is performed for definitive diagnosis [20]. However, biopsy was not performed for every procedure in the upper GI endoscopy performed in this study.

The WSHE for night time workers was conducted for 3 years after the implementation. It is the first study to obtain the validity of the questionnaire screening for GI symptoms in WSHE. In addition, this study did not have to integrate the 3-year test because the number of subjects per year was sufficient enough to consider each year test as a separate test.

CONCLUSIONS

Since 2014, WSHE has been implemented for night time workers, but there was no validity test for the screening tools used in WSHE. In this study, the validity of the diagnostic tool as a target disease for GI disorder was determined and low sensitivity was obtained that did not fit the purpose of screening. In order to achieve the purpose of screening, it is necessary to develop a more sensitive item with a clear selection of target disease.

Article information Continued