Developing a short standard questionnaire for assessing work organization hazards: the Healthy Work Survey (HWS)

Article information

Ann Occup Environ Med. 2023;35.e7

Publication date (electronic) : 2023 April 27

doi : https://doi.org/10.35371/aoem.2023.35.e7

BongKyoo Choi ¹^,²

, Youngju Seo ¹

¹Center for Work and Health Research, Irvine, CA, USA.

²Center for Occupational and Environmental Health, Department of Medicine, University of California Irvine, Irvine, CA, USA.

Correspondence: BongKyoo Choi. Center for Work and Health Research, 57 Middlebury Lane, Irvine, CA 92620, USA. b.choi@uci.edu

Received 2023 January 30; Revised 2023 March 01; Accepted 2023 March 13.

Abstract

Background

At present, no short standard questionnaire exists for assessing and comparing major work organization hazards in the workplaces of the United States.

Methods

We conducted a series of psychometric tests (content validity, factor analysis, differential-item functioning analysis, reliability, and concurrent validity) to validate and identify core items and scales for major work organization hazards using the data from the 2002–2014 General Social Surveys (GSSs), including the Quality of Worklife (QWL) questionnaire. In addition, an extensive literature review was undertaken to find other major work organization hazards which were not addressed in the GSS.

Results

Although the overall validity of the GSS-QWL questionnaire was satisfactory in the psychometric tests, some GSS-QWL items of work-family conflict, psychological job demands, job insecurity, use of skills on the job, and safety climate scales appeared to be weak. In the end, 33 questions (31 GSS-QWL and 2 GSS) were chosen as the least, but best validated core questions and included in a new short standard questionnaire (called the Healthy Work Survey [HWS]). And their national norms were established for comparisons. Furthermore, based on the literature review, fifteen more questions for assessing other significant work organization hazards (e.g., lack of scheduling control, emotional demands, electronic surveillance, wage theft) were included in the new questionnaire. Thus, the HWS includes 48 questions in total for assessing traditional and emerging work organization hazards, which covers seven theoretical domains: work schedule/arrangement, control, support, reward, demands, safety, and justice.

Conclusions

The HWS is a short standard questionnaire for assessing work organization hazards which can be used as a first step toward the risk management of major work organization hazards in the workplaces of the US.

Keywords: General Social Survey; Quality of Worklife Questionnaire; Psychometric; United States

BACKGROUND

Work organization hazards1 (e.g., low job control, high job demands, low social support at work, low rewards, long work hours, low safety climate, and high work and family conflicts) have negative impacts on workers’ health and safety2 3 4 5 6 7 8 and productivity.9 10 11 As a first step toward the risk management of work organization hazards,12 13 it is essential to have a short standard instrument for assessing the hazards (e.g., the Health & Safety Executive [HSE] Management Standards Indicator Tool of the United Kingdom14).

In the United States, there is not yet such standard instrument. In 2001, the US National Institute for Occupational Safety and Health (NIOSH) developed the Quality of Worklife (QWL) questionnaire for assessing the contemporary working life of US workers.15 The questionnaire includes about 60 items about major work organization hazards. The QWL questionnaire has been used as part of a nationwide sociological survey called the General Social Survey (GSS) since 2002.16 Thus, the QWL questionnaire has great potential to be used as a standard questionnaire for identifying and comparing the levels of major work organization hazards in the US workforce.17 However, the questionnaire is too long for routine use at the workplace. In addition, the psychometric validity of QWL items and scales of working conditions should be further tested and validated.18

On the other hand, several questionnaires based on contemporary influential work stress models (e.g., the Job Content Questionnaire19 based on the Karasek’s demand-control model20) have been widely used for workplace research projects in the US. However, the questionnaires do not allow national comparisons of the levels of work organization hazards due to a lack of recent national survey data including the questionnaires in the US. Also, both the NIOSH QWL questionnaire and several work stress model-based questionnaires are limited in assessing other important work organization hazards in a rapidly changing global economy. For example, the NIOSH QWL questionnaire and the JCQ (version 1.7) do not include items for emotional labor/demands,21 electronic surveillance,22 23 and wage theft (e.g., paid less than the minimum wage).24

The purpose of the current study is to describe a whole process of developing a short standard questionnaire (called the Healthy Work Survey [HWS]) for identifying and comparing major work organization hazards in the workplaces of the US.

The HWS was designed to be utilized by employers, worker organizations, individuals, and researchers who are interested in evaluating work organization hazards in terms of workers’ health and safety, and productivity. It addresses the following needs: 1) a short questionnaire of validated items and scales that assesses major traditional and emerging work organization hazards that are applicable to a wide range of occupations and industries in the US; and 2) identifying high-risk work organization hazards at a particular company or organization by comparing aggregate scores with national norms of work organization hazards in the US workforce.

METHODS

The HWS project

It was a two-year research collaboration project among researchers at the Center for Occupational and Environmental Health, University of California Irvine, Center for Work and Health Research, Center for Social Epidemiology, and State University of New York Downstate Health Sciences University. The goal of the project was to develop a short standard questionnaire for assessing work organization hazards in the US workforce. The project started in January 2018 with a research fund from Center for Social Epidemiology (Marina Del Rey, California, USA). Dr. Choi, principal investigator of the project, led this project during 2018–2019 (for details, see below “Acknowledgements”) and as planned, the HWS was developed in December 2019 25. Although some additional work was done for getting feedback of a group of external occupational health and safety experts (n = 9) on the developed HWS and creating an online version of the HWS during 2020–2021, the current study is restricted to the work done during 2018–2019. Since the analysis of the de-identified, publicly available GSS-QWL data did not constitute human subjects research, the current study did not require an Institutional Review Board’s review.

The HWS project during 2018–2019 consisted of two main phases17 25 26 27 28: Phase I – Identifying the least, but best core GSS-QWL or GSS items and scales of major work organizational hazards for the HWS and calculating their national norms; and Phase II – Adding or creating questions for other traditional or emerging major work organization hazards that are not part of the GSS-QWL or GSS, but need to be included in the HWS.

Phase I: The GSS-QWL items and scales were first examined to select the core items and scales that had been used in all five waves of the GSS in 2002, 2006, 2010, 2014, and 2018. Two non-QWL GSS items (one for labor force status and one for work hours) were also selected for analyses due to their significance.4 29 The core GSS-QWL and GSS items and scales were then reviewed in terms of clarity of item wording, content validity, and importance of work organization hazards according to contemporary work stress models such as the demand-control model,20 the effort-reward imbalance model,30 or organizational justice model.31 In addition, the following psychometric tests of the core items and scales were conducted in 2018 with the 2002–2014 GSS-QWL data (n = 5,796 workers; Supplementary Table 1): factor analysis, differential item functioning (DIF) analysis,32 scale reliability, and concurrent validity.

Exploratory factor analyses (extraction method: principal component analysis with varimax rotation) were undertaken with the core GSS-QWL items. In addition, we replicated the factor analyses in the following subgroups by age (up to 44 year and 45+ years), sex (men and women), race (white, black, and other), time (2002–2006 vs. 2010–2014), and occupation to test any sub-group differential construct validity of the scales.33 The following five occupational groups were tested: management, business, science, and arts occupations; service occupations; sales and office occupations; natural resources, construction, and maintenance occupations; and production, transportation, and material moving occupations.27

An item is considered to be DIF if “all respondents at a given level of the attribute measured (at a given index score) do not have equal probability of scoring positively on the item regardless of subgroup membership.”34 DIF analyses for item-level measurement equivalence of the core multi-item scales between the aforementioned subgroups were also examined with the partial gamma coefficient method.32 35 Category C (moderate to large) DIF items between comparison groups was defined as items with partial gamma outside the interval (−0.31 to 0.31) and its 95% confidence interval significantly outside the interval (−0.21 to 0.21). Category A (no or negligible) and Category B (slight to moderate) DIF items were also examined.

Reliability (internal consistency) of the core multi-item scales was examined with Cronbach’s alpha statistics. Concurrent validity of the core GSS-QWL and GSS items and scales was investigated with fourteen health outcomes included in the 2002-2014 GSS-QWL data: the healthy work days measures of the US Center for Disease Control and Prevention (CDC)36 (4 items: each for general health, physical health, mental health; and unhealthy days), perceived stress at work, exhaustion, back pain, injuries, sleep problem, hypertension, diabetes, depression, obesity, and absenteeism (missing work days during the past 30 days: 1 or more vs. never). The health outcomes were all dichotomized for analyses. The concurrent validity was initially checked with Pearson and Spearman correlation coefficients and later confirmed with logistic regression analyses after controlling for age and sex.

The least, but best core GSS-QWL and GSS items for the HWS were determined with all considerations of the results of the aforementioned validation tests. National norms of the best core items/scales for the HWS were calculated in 2019 with the 2002–2018 GSS-QWL data (n = 7,189 workers) after taking into account the nature of a social complex survey of the GSS. For convenient comparisons with national statistics, scores of each multi-item scale of work organization hazards were further divided into three (low, medium, and high) risk groups using their tertiles.

Phase II: Several traditional or emerging important work organization hazards are missing in the NIOSH-QWL questionnaire and several work stress model-based questionnaires. Thus, we conducted an extensive literature review on major work organization hazards, including their assessment instruments, and if necessary, contacted external subject matter experts by e-mail for more information. In the end, for the final version of the HWS, we adopted some items from existing instruments or created some new items for the missing, but important work organization hazards.

All statistical analyses were conducted with SPSS software version 26.0 (IBM, Armonk, NY, USA).

RESULTS

Factor analyses with core GSS-QWL items

Exploratory factor analyses were initially conducted with the core 34 GSS-QWL items (Table 1) and more items (up to 50 GSS-QWL items in total) were later added and tested in subsequent factor analyses. Overall, the factor analysis of the core 34 GSS-QWL items confirmed the following nine underlying constructs of the 34 items: safety climate (4 items), psychological job demands and resource adequacy (7 items), supervisor and coworker support (4 items), job control (6 item), physical job demands (3 items), work and family conflict (3 items), discrimination (2 items), harassment (2 items), and pay/fairness (2 items) (Table 1). As expected, when more items (up to 50 items in total) of other work organization hazards (e.g., job insecurity, promotion, and respect) were added in subsequent factor analyses, more underlying constructs were identified. At the same time, in subsequent factor analyses, some general constructs (e.g., supervisor and coworker support) identified in the factor analyses with the core 34 GSS-QWL items were separated into more specific distinct constructs (e.g., supervisor support and coworker support).

Table 1

Exploratory factor analysis of 34 GSS-QWL items in the 2002–2014 GSS-QWL data (n = 5,796 workers)

However, one item of work and family conflict (“How hard is it to take time off during your work to take care of personal or family matters?”), one item of psychological job demands (“My job requires that I work very fast”), and one item of job insecurity (“How easy would it be for you to find a job with another employer with approximately the same income and fringe benefits as you have now?) were not loaded (factor loadings < 0.30) on the supposed underlying constructs in the subsequent expanded factor analyses with more numbers of items. In addition, the items of work and family conflict and psychological job demands scales also appeared to be problematic in the subgroup factor analyses, particularly, in the five occupational groups.

DIF analyses with core GSS-QWL multi-item scales

There were no moderate to large DIF items of the following multi-item GSS-QWL scales: decision making opportunities (a subscale of job control), resource adequacy, work and family conflict, supervisor and coworker support, and promotion. However, as in the factor analyses, some items of the psychological job demands and job insecurity were identified as moderate to large DIF items, particularly between occupational groups. In addition, one item of use of skills on the job (a subscale of job control: “My job requires that I keep learning new things”) and two items of safety climate (“Where I work, employees and management work together to ensure the safest possible working conditions” and “The safety and health conditions where I work are good”) turned out to be moderate to large DIF items between occupational or racial groups (Table 2).

Table 2

Differential item functioning analyses for each item of safety climate scale (4 items in total; scores ranged from 4 to 16) between various comparison groups in the 2002–2014 GSS-QWL data (n = 5,796 workers)

Reliability of core GSS-QWL multi-item scales

Table 2 shows Cronbach’s alpha values of some GSS-QWL multi-item scales. The alpha values of use of skills on the job, supervisor support, coworker support, physical demands, safety climate, promotion, and reward scales were satisfactory, given their relatively smaller numbers of items. However, the alpha values of job insecurity and pay/fairness was very low (close to 0.30). In addition, when an item was removed, the alpha values in the following scales increased moderately (≥ 0.05) (Table 3): work and family conflict (“How hard is it to take time off during your work to take care of personal or family matters?”), resource adequacy (“How often are there not enough people or staff to get all the work done?”), and job insecurity (“How easy would it be for you to find a job with another employer with approximately the same income and fringe benefits as you have now?”).

Table 3

Cronbach’s alphas of GSS-QWL multi-item scales in the 2002–2014 GSS-QWL data (n = 5,796 workers)

Concurrent validity of core GSS-QWL and GSS items

The core QWL-QWL and GSS items and scales were first examined with both Pearson and Spearman correlation coefficients in relations to the fourteen health outcomes included in the 2002-2014 GSS-QWL data. The directions of the correlations were consistent with our expectations. For example, low job control, high psychological job demands, high strain (a ratio of psychological job demands to job control), job insecurity, and low rewards were all positively correlated with depression, hypertension, and absenteeism. The results remained to be very similar in complex survey design based multivariate logistic regression analyses after controlling for age and sex. In addition, comparisons of the correlations between some full-scales with and without psychometrically weak item(s) were made in relations to the health outcomes. For instance, the full scale of work and family conflict (3 items) and the reduced scale of work and family conflict (2 items; without the following psychometrically weak item: “How hard is it to take time off during your work to take care of personal or family matters?”) were compared in terms of correlations with the four healthy work days measures (Table 4). The odds ratios of the full and reduced scales of work and family conflict for each of the four healthy work days measures after controlling for age and sex were very similar to each other.

Table 4

Complex-design based multivariate odds ratios (their 95% confidence intervals) of the full (3 items) and reduced (2 items) scales of work and family conflict scales with each of the four healthy work days measures after controlling for age and sex in the 2002–2014 GSS-QWL data (n = 5,796 workers)

National norms of 33 GSS-QWL and GSS items and scales included in the HWS

With all considerations of the results of the psychometric tests and their theoretical relevance and importance in work stress and health research, thirty-three (31 GSS-QWL and 2 GSS) items of work organization hazards (Supplementary Data 1) were selected to be included in the HWS. For national norms of the thirty-three items, the response distribution table of each item was created using the 2002–2018 GSS-QWL data (not shown here, available on request to the authors). Table 5 shows the low-, intermediate-, and high-risk score ranges of GSS-QWL multiple-item scales included in the HWS, based on the 2002-2018 GSS-QWL data.

Table 5

Three-group risk classification of multi-item scales of work organization hazards using the data (n = 7,189 workers) from the 2002–2018 General Social Surveys

New fifteen items of traditional or emerging work organization hazards included in the HWS

Based on an extensive literature review on major work organization hazards and their measurement instruments, new fifteen non-GSS items of traditional or emerging work organization hazards were included in the HWS (Supplementary Data 1): scheduling control (2 items), emotional demands (2 items), electronic surveillance (1 item), exposure to toxic chemicals (1 items), safety hazard (1 item), workplace physical violence (1 item), bullying (1 item), low wage (1 item), and wage theft (1 item), organizational justice (1 item), union (1 item), paid sick leave (1 item), and medical insurance (1 item). Out of 15 non-GSS items, 11 items came from existing validated instruments, 3 items were newly created, but simple in content for asking about existence of union and workers’ benefits, and 1 item was suggested by an external expert (see below).

Two items for scheduling control, work time arrangement (flextime)37 and schedule changes/advanced notice, were selected from the 2015 European working condition survey questionnaire.38 The items were also used in the 2015 American working condition survey.39 Two items for emotional demands came from the Copenhagen Psychosocial Questionnaire (COPSOQ) II.40 A single item for electronic surveillance22 23 was suggested by an expert at the NIOSH with information that the item could be included in a future national survey. Exposure to chemical and safety hazards cannot be ignored as workplace stressors.12 Thus, two items for those hazards were adopted from the 2010 National Health Interview Survey-Occupational Health Supplement41 and the 1977 Quality of Employee Survey.42 Items about workplace physical violence and bullying were selected from the COPSOQ II and a 2017 report of the Workplace Bullying Institute,43 respectively. The items for low wage (two-thirds of the median hourly wage rate in US workers),44 wage theft,24 and organizational justice (procedural justice)45 were added based on the literature review. Items for union,46 paid sick leave,47 and medical insurance48 were created due to their particular importance in the US where union density is very low and neither federal legal requirements for paid sick leave nor universal healthcare systems exist.

Structures of the HWS

In summary, the HWS includes 48 questions for assessing important traditional and emerging work organization hazards: 31 questions from the GSS-QWL and 2 questions from the GSS and new 15 non-GSS questions mostly from existing validated instruments (Table 6, Supplementary Data 1). The 48 HWS items are conceptually summarized into seven theoretical domains of work organization hazards (Fig. 1): Work schedule/Arrangement, 8 items (W1–W8); Control, 6 items (W9–W14); Support, 4 items (W15–W18); Reward, 5 items (W19–W23); Demands, 11 items (W24–W34); Safety, 5 items (W35–W39); and Justice, 9 items (W40–W48). In addition, in supplementary sections, the HWS also includes 17 questions for health outcomes (H1–H17: 14 questions from the GSS, 2 questions from the Work Limitations Questionnaire,49 and 1 item for stress at home) and 10 questions for sociodemographic characteristics (S1–S10) (for details, see Supplementary Data 1).

Table 6

Forty-eight questions of major work organization hazards in the Healthy Work Survey

Fig. 1

Seven theoretical domains of the forty-eight questions of work organization hazards in the Healthy Work Survey.

DISCUSSION

In this paper, we presented a new short, standard questionnaire for identifying and comparing major work organization hazards in the workplace of the US. The HWS was developed through a series of rigorous psychometric tests of core GSS-QWL and GSS items of major work organization hazards with consideration of their theoretical relevance and importance in work and health research, and an extensive literature review of other important work organization hazards and their instruments.

We think that the HWS has two major merits over other instruments for assessing work organization hazards particularly in the US workforce. First, as intended, the HWS is relatively short (48 items in total) in length while addressing various major traditional and emerging work organization hazards of seven theoretical domains. For the HWS, we were able to select only almost half out of 60 items in the original NIOSH QWL questionnaire through rigorous systematic validation tests. This significant reduction in the number of items created a room for 15 items about other important work organization hazards in the HWS, which are not assessed with the QWL questionnaire. In addition, we expect that the HWS would be completed within 10 minutes in most cases. Second, it enables the users to compare their aggregated scores (at least, for the 33 GSS-QWL/GSS items and scales) at a particular company or organization with national norms of work organization hazards in the US workforce. Thus, the users will be able to not only asses work organization hazards, but also identify high-risk work organization hazards at their organizations based on the comparison with national norms. Such comparison will help all workplace stakeholders better understand the levels of their work organization hazards from a national perspective, which may facilitate their concerted efforts to address their high-risk work organization hazards. The HSE Management Standards Indicator Tool of the United Kingdom (35 items of six theoretical domains in total) is shorter in length than the HWS; however, in contrast with the HWS, it neither has national norms for comparison, nor includes questions about electronic surveillance, safety climate, chemical and physical hazards, low income, and wage theft.

On the other hand, this study is one of the few comprehensive psychometric studies of the NIOSH QWL questionnaire.18 Although the overall validity of the GSS-QWL questionnaire was good, this study indicates that some GSS-QWL items of work-family conflict, psychological job demands, job insecurity, use of skills on the job, and safety climate scales are psychometrically weak. Although the weak items were not selected for the HWS, their validity needs to be further tested and confirmed in future studies. Also, the current study has a strength of providing a detailed methodology of creating a short, standard questionnaire for assessing major work organization hazards. Furthermore, our literature review in the current study demonstrated that several important work organization hazards are not currently addressed in the NIOSH QWL questionnaire and influential work stress model-based questionnaires, for example, schedule control, emotional demands, electronic surveillance, and wage theft. We hope that the HWS facilitates more future studies on those relatively understudied, but increasingly important work organization hazards.

There are two limitations in the current study. First, the validity of the HWS as a whole, including the non-GSS items, should be further tested and evaluated in a wide range of occupations and industries. Nonetheless, it should be reminded that most of the 15 non-GSS items came from existing validated instruments and some items were created, but very simple in content. Furthermore, the developed HWS was additionally reviewed by a group of experts during 2020–2021 and no significant weaknesses were identified. Second, at present, the national norms of the HWS are established only for the 31 GSS-QWL and 2 GSS items using the 2002–2018 GSS-QWL data, but not for the 15 non-GSS items of work organization hazards. Thus, the national norms of the non-GSS items remains to be established in the future. Also, the current national norms of the 31 GSS-QWL and 2 GSS items needs to be continuously updated with future GSS-QWL data.

CONCLUSIONS

We developed a new short, standard questionnaire for identifying and comparing major work organization hazards in the workplace of the US. The developed questionnaire, the HWS includes 48 questions (31 GSS-QWL, 2 GSS, and 15 non-GSS items) in total for assessing traditional and emerging work organization hazards, which covers seven theoretical domains: work schedule/arrangement, control, support, reward, demands, safety, and justice. We believe that the HWS, if widely accepted and used, has great potential to be an essential tool for the risk management of traditional and emerging major work organization hazards in the workplaces of the US.

ACKNOWLEDGMENTS

We thank Dr. Naomi Swanson (National Institute for Occupational Safety and Health; NIOSH) for providing a copy of the Quality of Worklife (QWL) questionnaire included in the 2018 General Social Survey. We also thanks the following researchers for providing information on instruments of some major work organization hazards or health outcomes which are not addressed in the QWL questionnaire: Drs. Paul Landsbergis (State University of New York Downstate Health Sciences University), Marnie Dobson (University of California, Irvine), Peter Schnall (Center for Social Epidemiology), Pouran Faghri (University of Connecticut), Meredith Minkler (University of California, Berkeley), Isabel Garcia (University of California, Los Angeles), Toni Alterman (NIOSH), Gary Namie (Workplace Bullying Institute), and Hermann Burr (Federal Institute for Occupational Safety and Health, BAuA in Germany).

Notes

Funding: This study is funded by the Center for Social Epidemiology (Marina Del Rey, CA, USA) and the Center for Work and Health Research (Irvine, CA, USA).

Competing interests: The authors declare that they have no competing interests.

Author Contributions:

Conceptualization: Choi B.
Data curation: Choi B, Seo Y.
Formal analysis: Choi B, Seo Y.
Funding acquisition: Choi B.
Investigation: Choi B, Seo Y.
Methodology: Choi B.
Software: Choi B.
Writing - original draft: Choi B.
Writing - review & editing: Choi B, Seo Y.

Abbreviations

CDC

Center for Disease Control and Prevention

COPSOQ

Copenhagen Psychosocial Questionnaire

DIF

differential item functioning

GSS

General Social Survey

GSS-QWL

General Social Survey-Quality of Worklife

HSE

Health & Safety Executive

HWS

Healthy Work Survey

NIOSH

National Institute for Occupational Safety and Health

QWL

Quality of Worklife

References

1. National Institute for Occupational Safety and Health (NIOSH). The changing organization of work and the safety and health of working people. DHHS (NIOSH) publication No. 2002–116 Updated 2002. Accessed January 20, 2023. https://www.cdc.gov/niosh/docs/2002-116/pdfs/2002-116.pdf?id=10.26616/NIOSHPUB2002116 .

2. Dragano N, Siegrist J, Nyberg ST, Lunau T, Fransson EI, Alfredsson L, et al. Effort-reward imbalance at work and incident coronary heart disease: a multicohort study of 90,164 individuals. Epidemiology 2017;28(4):619–626. 28570388.

3. Kivimäki M, Nyberg ST, Batty GD, Fransson EI, Heikkilä K, Alfredsson L, et al. Job strain as a risk factor for coronary heart disease: a collaborative meta-analysis of individual participant data. Lancet 2012;380(9852):1491–1497. 22981903.

4. Virtanen M, Nyberg ST, Batty GD, Jokela M, Heikkilä K, Fransson EI, et al. Perceived job insecurity as a risk factor for incident coronary heart disease: systematic review and meta-analysis. BMJ 2013;347:f4746. 23929894.

5. Stansfeld S, Candy B. Psychosocial work environment and mental health--a meta-analytic review. Scand J Work Environ Health 2006;32(6):443–462. 17173201.

6. Theorell T, Hammarström A, Aronsson G, Träskman Bendz L, Grape T, Hogstedt C, et al. A systematic review including meta-analysis of work environment and depressive symptoms. BMC Public Health 2015;15(1):738. 26232123.

7. Clays E, De Bacquer D, Leynen F, Kornitzer M, Kittel F, De Backer G. The impact of psychosocial factors on low back pain: longitudinal results from the Belstress study. Spine (Phila Pa 1976) 2007;32(2):262–268. 17224824.

8. Ostergren PO, Hanson BS, Balogh I, Ektor-Andersen J, Isacsson A, Orbaek P, et al. Incidence of shoulder and neck pain in a working population: effect modification between mechanical and psychosocial exposures at work? Results from a one year follow up of the Malmö shoulder and neck study cohort. J Epidemiol Community Health 2005;59(9):721–728. 16100307.

9. Milner A, Butterworth P, Bentley R, Kavanagh AM, LaMontagne AD. Sickness absence and psychosocial job quality: an analysis from a longitudinal survey of working Australians, 2005-2012. Am J Epidemiol 2015;181(10):781–788. 25841868.

10. Svedberg P, Mather L, Bergström G, Lindfors P, Blom V. Work-home interference, perceived total workload, and the risk of future sickness absence due to stress-related mental diagnoses among women and men: a prospective twin study. Int J Behav Med 2018;25(1):103–111. 28639237.

11. Oshio T, Tsutsumi A, Inoue A, Suzuki T, Miyaki K. The reciprocal relationship between sickness presenteeism and psychological distress in response to job stressors: evidence from a three-wave cohort study. J Occup Health 2017;59(6):552–561. 28993575.

12. International Standards Organization (ISO). Occupational health and safety management - Psychological health and safety at work - Guidelines for managing psychosocial risks (ISO 45003:2021) Accessed January 20, 2023. https://www.iso.org/obp/ui/#iso:std:iso:45003:ed-1:v1:en .

13. International Labor Organization (ILO). Inspection actions to deal with psychosocial risks (Module 15), ILO Curriculum on Building Modern and Effective Labour Inspection Systems Accessed January 20, 2023. https://www.ilo.org/wcmsp5/groups/public/---ed_dialogue/---lab_admin/documents/genericdocument/wcms_856575.pdf .

14. Health and Safety Executive. The HSE Management Standards indicator tool Accessed January 20, 2023. https://www.hse.gov.uk/stress/standards/step2/surveys.htm .

15. National Institute for Occupational Safety and Health (NIOSH). Quality of Worklife questionnaire Accessed January 20, 2023. https://www.cdc.gov/niosh/topics/stress/qwlquest.html .

16. The General Social Survey Quality of Worklife Module Accessed January 20, 2023. https://gss.norc.org/Pages/quality-of-worklife.aspx .

17. Choi B, Garcia-Rivas J, Landsbergis P, Dobson M, Schnall P, Baker D, et al. International comparison of national work organizational risk assessment tools in seven countries In : Proceedings of the 2nd International Symposium to Advance Total Worker Health; 2018 May 8-11; Bethesda, MD, USA. St. Louis, MO, USA: Healthy Work Center; 2018.

18. Shan Y, Imran H, Lewis P, Zhai D. Investigating the latent factors of quality of work-life affecting construction craft worker job satisfaction. J Constr Eng Manage 2017;143(5)04016134.

19. Karasek RA, Gordon G, Pietrokovsky C, Frese M, Pieper C, Schwartz J, et al. Job Content Questionnaire and User’s Guide Los Angeles, CA, USA/Lowell, MA, USA: University of Southern California/University of Massachusetts; 1985.

20. Karasek RA. Job demands, job decision latitude, and mental strain: implications for job redesign. Adm Sci Q 1979;24(2):285–308.

21. Hochschild AR. The Managed Heart: Commercialization of Human Feeling Berkeley, CA, USA: University of California Press; 1983.

22. Siegel R, König CJ, Lazar V. The impact of electronic monitoring on employees’ job satisfaction, stress, performance, and counterproductive work behavior: a meta-analysis. Comput Hum Behav Rep 2022;8100227.

23. Ball K. Electronic monitoring and surveillance in the workplace Updated 2021. Accessed January 20, 2023. https://op.europa.eu/en/publication-detail/-/publication/e7e6f646-4694-11ec-89db-01aa75ed71a1/language-en .

24. Minkler M, Salvatore AL, Chang C, Gaydos M, Liu SS, Lee PT, et al. Wage theft as a neglected public health problem: an overview and case study from San Francisco’s Chinatown district. Am J Public Health 2014;104(6):1010–1020. 24825200.

25. Choi B, Landsbergis P, Seo Y, Dobson M, Schnall P. Creating a new instrument for work organization risk assessment in the United States: the Healthy Work Survey (HWS) project In : Proceedings of the 2019 American Public Health Association Conference; 2019 November 2-6; Philadelphia, PA, USA. Washington, D.C., USA: American Public Health Association; 2019.

26. Choi B. Developing a short, standard tool for work organization risk assessment based on the NIOSH Quality of Working Life (QWL) Survey In : Proceedings of the 2nd Creating Healthy Organizations Conference; 2018 October 12; Costa Mesa, CA, USA.

27. Choi B, Seo Y, Dobson M, Landsbergis P, Schnall P. Construct validity of the NIOSH Quality of Work-Life questionnaire: the Healthy Work Survey (HWS) project In : Proceedings of the 2019 APA/NIOSH Work, Stress and Health Conference; 2019 November 6-9; Philadelphia, PA, USA. Washington, D.C., USA: American Psychological Association; 2019.

28. Choi B, Dobson M, Landsbergis P, Schnall P. Developing a short on-line standard questionnaire for work organization risk assessment in the United States: The Healthy Work Survey (HWS) project In : Proceedings of the 2019 APA/NIOSH Work, Stress and Health Conference; 2019 November 6-9; Philadelphia, PA, USA. Washington, D.C., USA: American Psychological Association; 2019.

29. Choi B. Job strain, long work hours, and suicidal ideation in US workers: a longitudinal study. Int Arch Occup Environ Health 2018;91(7):865–875. 29959524.

30. Siegrist J. Adverse health effects of high-effort/low-reward conditions. J Occup Health Psychol 1996;1(1):27–41. 9547031.

31. Greenberg J. Organizational justice: yesterday, today, and tomorrow. J Manage 1990;16(2):399–432.

32. Choi B, Bjorner JB, Ostergren PO, Clays E, Houtman I, Punnett L, et al. Cross-language differential item functioning of the job content questionnaire among European countries: the JACE study. Int J Behav Med 2009;16(2):136–147. 19575297.

33. Choi B, Kurowski A, Bond M, Baker D, Clays E, De Bacquer D, et al. Occupation-differential construct validity of the Job Content Questionnaire (JCQ) psychological job demands scale with physical job demands items: a mixed methods research. Ergonomics 2012;55(4):425–439. 22423675.

34. Ørhede E, Kreiner S. Item bias in indices measuring psychosocial work environment and health. Scand J Work Environ Health 2000;26(3):263–272. 10901120.

35. Bjorner JB, Kreiner S, Ware JE, Damsgaard MT, Bech P. Differential item functioning in the Danish translation of the SF-36. J Clin Epidemiol 1998;51(11):1189–1202. 9817137.

36. Center for Disease Control and Prevention. Measuring healthy days Updated 2000. Accessed January 20, 2023. https://www.cdc.gov/hrqol/pdfs/mhd.pdf .

37. Joyce K, Pabayo R, Critchley JA, Bambra C. Flexible working conditions and their effects on employee health and wellbeing. Cochrane Database Syst Rev 2010;2010(2)CD008009. 20166100.

38. European Working Condition Survey 2015 Updated 2015. Accessed January 20, 2023. https://www.eurofound.europa.eu/surveys/european-working-conditions-surveys/sixth-european-working-conditions-survey-2015/ewcs-2015-questionnaire .

39. Maestas N, Mullen KJ, Powell D, Von Wachter T, Wenger JB. Working Conditions in the United States: Results of the 2015 American Working Conditions Survey Santa Monica, CA, USA: RAND Corporation; 2017.

40. Pejtersen JH, Kristensen TS, Borg V, Bjorner JB. The second version of the Copenhagen Psychosocial Questionnaire. Scand J Public Health 2010;38(3)Suppl. :8–24. 21172767.

41. National Institute for Occupational Safety and Health (NIOSH). Worker health information from the national health interview survey Accessed on January 20, 2023. https://www.cdc.gov/niosh/topics/nhis/default.html .

42. Quinn RP, Staines G. Quality of employment survey, 1977: cross-section. Interuniversity Consortium for Political and Social Research Ann Arbor, MI, USA: University of Michigan; 1992.

43. Namie G. 2017 Workplace bullying institute US workplace bullying survey Accessed on January 20, 2023. https://workplacebullying.org/download/2017-wbi/ .

44. Leigh JP, De Vogli R. Low wages as occupational health hazards. J Occup Environ Med 2016;58(5):444–447. 27158950.

45. Colquitt JA. On the dimensionality of organizational justice: a construct validation of a measure. J Appl Psychol 2001;86(3):386–400. 11419799.

46. Dollard MF, Neser DY. Worker health is good for the economy: union density and psychosocial safety climate as determinants of country differences in worker health and productivity in 31 European countries. Soc Sci Med 2013;92:114–123. 23849285.

47. Callison K, Pesko MF. The effect of paid sick leave mandates on coverage, work absences, and presenteeism. J Hum Resour 2022;57(4):1178–1208. 35812986.

48. Goh J, Pfeffer J, Zenios SA. The relationship between workplace stressors and mortality and health costs in the United States. Manage Sci 2016;62(2):608–628.

49. Lerner D, Amick BC 3rd, Lee JC, Rooney T, Rogers WH, Chang H, et al. Relationship of employee-reported work limitations to work productivity. Med Care 2003;41(5):649–659. 12719689.

SUPPLEMENTARY MATERIALS

Supplementary Table 1

Sociodemographic characteristics of 5,796 workers in the 2002–2014 GSS-QWL data

aoem-35-e7-s001.xls

Supplementary Data 1

The Healthy Work Survey

aoem-35-e7-s002.doc

Article information Continued

Korean Society of Occupational & Environmental Medicine

https://creativecommons.org/licenses/by-nc/4.0/

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Funded by : Center for Social Epidemiology

Funded by : Center for Work and Health Research

GSS-QWL item	Component^a
GSS-QWL item	1.000	2.000	3.000	4.000	5.000	6.000	7.000	8.000	9.000
HOW HARD TO TAKE TIME OFF	0.023	0.145	0.202	0.093	−0.184	−0.557	−0.124	0.106	0.035
HOW OFTEN JOB INTERFERES FAM LIFE	−0.022	−0.223	−0.022	0.114	0.033	0.794	0.037	0.055	0.045
HOW OFTEN FAM LIFE INTERFERE JOB	−0.012	−0.078	0.058	0.054	−0.078	0.734	−0.062	0.173	−0.095
JOB REQUIRES R TO WORK FAST	0.043	−0.336	−0.018	0.269	0.249	0.206	0.063	−0.017	0.047
R HAS TOO MUCH WORK TO DO WELL	−0.050	−0.564	0.018	0.060	0.008	0.250	0.061	−0.096	−0.049
R HAS ENOUGH TIME TO GET THE JOB DONE	0.095	0.680	0.255	−0.060	0.017	−0.143	−0.034	−0.072	0.007
R FREE FROM CONFLICTING DEMANDS	0.083	0.444	0.367	0.028	0.015	−0.157	−0.011	−0.178	−0.043
ENOUGH HELP AND EQUIP TO GE THE JOB DONE	0.204	0.610	0.308	0.142	−0.030	0.037	−0.045	−0.072	0.142
ENOUGH INFO TO GET THE JOB DONE	0.164	0.583	0.274	0.154	0.059	0.048	−0.017	−0.029	0.023
HOW OFTEN NOT ENOUGH STAFF	−0.060	−0.656	0.042	0.053	0.087	0.095	0.078	0.062	−0.035
JOB REQUIRES R TO LEARN NEW THINGS	0.088	−0.166	0.068	0.708	−0.112	0.095	0.048	−0.021	0.015
OPPORTUNITY TO DEVELOP MY ABILITIES	0.099	0.198	0.260	0.652	−0.065	0.053	−0.039	−0.055	0.119
JOB ALLOWS R USE OF SKILLS	0.257	0.145	0.063	0.671	0.010	−0.019	−0.050	−0.020	0.009
R DOES NUMEROUS THINGS ON JOB	0.136	−0.108	0.028	0.719	0.028	−0.006	0.007	0.063	−0.032
HOW OFTEN R TAKE PART IN DECISIONS	0.081	−0.198	0.362	0.387	−0.016	0.025	−0.021	0.080	0.140
A LOT OF FREEDOM TO DECIDE HOW TO DO JOB	0.054	0.322	0.284	0.438	−0.097	−0.058	−0.094	−0.005	0.047
SUPERVISOR CONCERNED ABOUT WELFARE	0.207	0.092	0.748	0.067	−0.058	−0.051	−0.085	−0.058	0.074
SUPERVISOR HELPFUL TO R IN GETTING JOB DONE	0.171	0.172	0.738	0.051	−0.003	−0.033	−0.073	−0.076	0.065
COWORKERS CAN BE RELIED ON WHEN R NEEDS HELP	0.110	0.273	0.567	0.132	−0.057	−0.013	−0.064	−0.086	0.067
COWORKERS TAKE A PERSONAL INTEREST IN R	0.135	0.112	0.651	0.226	−0.063	−0.013	−0.046	0.042	−0.025
HOW FAIR IS WHAT R EARN ON THE JOB	−0.067	−0.188	−0.141	0.016	0.060	0.006	0.112	−0.006	−0.688
INCOME ALONE IS ENOUGH	0.047	−0.044	0.017	0.128	−0.039	−0.068	0.030	−0.053	0.802
R DO REPEATED LIFTING	−0.015	0.018	−0.022	−0.080	0.836	−0.030	0.004	0.034	−0.026
R PERFORM FORCEFUL HAND MOVEMENTS	−0.038	−0.073	−0.043	−0.047	0.732	0.005	0.027	0.086	−0.013
RATE PHYSICAL EFFORT	−0.059	−0.014	−0.069	−0.014	0.829	0.106	0.016	−0.022	−0.072
SAFETY AND HEALTH CONDITION GOOD AT WORK	0.793	0.122	0.162	0.181	−0.094	−0.023	−0.067	−0.058	0.029
WORKER SAFETY PRIORITY AT WORK	0.826	0.098	0.181	0.156	0.008	−0.021	−0.059	−0.043	0.058
NO SHORTCUTS ON WORKER SAFETY	0.834	0.120	0.121	0.143	−0.034	−0.018	−0.049	−0.043	0.042
MGT AND EMPLOYEES WORK TOGETHER RE SAFETY	0.843	0.132	0.218	0.153	−0.019	−0.006	−0.060	−0.054	0.030
R FEELS DISCRIMINATED BECAUSE OF AGE	−0.058	−0.086	−0.052	−0.006	0.070	−0.005	0.678	−0.010	−0.021
R FEELS DISCRIMINATED BECAUSE OF RACE	−0.065	−0.015	−0.101	−0.022	0.023	0.050	0.723	0.018	−0.052
R FEELS DISCRIMINATED BECAUSE OF GENDER	−0.058	−0.115	−0.059	−0.022	−0.081	0.033	0.582	0.387	0.005
R SEXUALLY HARASSED ON THE JOB LAST 12 MONTHS	−0.040	−0.002	0.007	−0.024	0.004	0.017	0.038	0.799	−0.042
R THREATENED ON THE JOB LAST 12 MONTHS	−0.089	−0.097	−0.129	0.041	0.109	0.087	0.100	0.615	−0.004

Safety climate scale item^c	Partial gamma correlation coefficients
	Survey year	Age	Sex	Race	Race	Occupation^b
	2010/2014 vs. 2002/2006	≥ 45 vs. < 45 years	Women vs. Men	Black vs. White	Other vs. White	2 vs. 1	3 vs. 1	4 vs. 1	5 vs. 1
SAFEHLTH	−0.036	0.081	−0.132	0.160	−0.076	0.073	0.074	0.476^a	0.395^a
SAFETYWK	0.114	0.005	0.039	−0.011	0.151	−0.052	0.010	−0.309	−0.185
SAFEFRST	−0.034	−0.107	0.109	−0.054	0.179	0.072	0.031	−0.003	−0.127
TEAMSAFE	−0.057	0.042	−0.048	−0.146	−0.380^a	−0.169	−0.145	−0.150	−0.076

Scale		Number of items	Cronbach alpha	Cronbach alpha if an item is removed
Work and family conflict		3	0.568	0.633^a
Psychological job demands		4	0.579	0.591^b
Resource adequacy		3	0.617	0.665^c
Use of skills on the job		4	0.716
Decision-making opportunities		3	0.602	0.614^d
	Supervisor support	2	0.774
	Coworker support	2	0.578
	Physical demands	3	0.751
	Safety climate	4	0.895
Job insecurity		4	0.300	0.401^e
	Pay/fairness	2	0.336
	Promotion	2	0.625
	Reward	5	0.688

Work and family conflict	Level	General health (fair/poor vs. excellent/very good/good)^b	Physically unhealth days (≥ 14 vs. < 14 days)^c	Mentally unhealthy days (≥ 14 vs. < 14 days)^d	Activity limitations (≥ 14 vs. < 14 days)^e
Full scale (3 items)	Low	1	1	1	1
	Middle	1.15 (0.93–1.42)	1.18 (0.86–1.63)	1.35 (1.04–1.75)	1.68 (1.06–2.67)
	High	1.40 (1.12–1.75)	1.49 (1.08–2.07)	2.41 (1.85–3.15)	2.43 (1.58–3.74)
Reduced scale (2 items)^a	Low	1	1	1	1
	Middle	1.07 (0.82–1.41)	1.09 (0.78–1.51)	1.07 (0.82–1.41)	1.56 (0.99–2.47)
	High	1.21 (0.96–1.53)	1.47 (1.03–2.11)	1.95 (1.47–2.60)	2.50 (1.56–3.99)

Scale	Score range
Scale	Low risk	Intermediate risk	High risk
Low Use of Skills on the Job (3 items)	3–4 (34.1%)	5–6	7–12 (20.3%)
Low Decision Making Opportunities (2 items)	2 (25.9%)	3–4	5–8 (21.5%)
Low Control (5 items)	5–7 (33.8%)	8–9	10–20 (35.2%)
Low Supervisor Support (2 items)	2 (38.2%)	3–4	5–8 (19.1%)
Low Coworker Support (2 items)	2 (31.5%)	3–4	5–8 (15.0%)
Low Promotion Opportunities (2 items)	2–3 (29.9%)	4–5	6–8 (25.2%)
Low Reward (5 items)	5–9 (28.0%)	10–12	13–21 (28.1%)
High Workload (2 items)	2–3 (38.7%)	4	5–8 (29.6%)
High Psychological Demand (3 items)	3–5 (39.8%)	6–7	8–12 (19.7%)
Low Resource Adequacy (2 items)	2–3 (28.6%)	4–5	6–8 (21.6%)
High Workload Low Resource Adequacy (4 items)	4–7 (37.8%)	8–9	10–16 (30.5%)
High Work-Family Conflict (2 items)	2–3 (29.0%)	4–5	6–8 (25.8%)
Low Safety Climate (2 items)	2 (35.6%)	3–4	5–8 (11.4%)

Work schedule/Arrangement (8 items)	Control (6 items)	Support (4 items)	Reward (5 items)	Demands (11 items)	Safety (5 items)	Justice (9 items)
• Labor force status	• Use of skills on the job	• Coworker support	• Respect	• Workload	• Safety climate	• Discrimination
• Work hours/overtime	• Decision making opportunity	• Supervisor support	• Promotion	• Role conflict	• Chemical hazards	• Harassment
• Work time arrangement			• Fair earning	• Physical effort	• Safety hazards	• Bullying
• Work schedule change/notice			• Job security	• Resource adequacy	• Physical violence	• Low wage
• Shift work				• Work-family conflict		• Wage theft
				• Emotional demands		• Procedural justice
				• Electronic surveillance		• Union
						• Paid sick leave
						• Medical insurance