Improving inter-rater reliability of the enhancing assessment of common therapeutic factors (ENACT) measure through training of raters

Journal of Public Health in Africa

Field Value
Title Improving inter-rater reliability of the enhancing assessment of common therapeutic factors (ENACT) measure through training of raters
Creator Mwenge, Mwamba M. Figge, Caleb J. Metz, Kristina Kane, Jeremy C. Kohrt, Brandon A. Pedersen, Gloria A. Sikazwe, Izukanji Van Wyk, Stephanie Skavenski Mulemba, Saphira M. Murray, Laura K.
Subject — training; raters; inter-rater reliability; counselor competency
Description Background. As evidence supports task-shifting approaches to reduce the global mental health treatment gap, counselor competency evaluation measures are critical to ensure evidence-based therapies are administered with quality and fidelity. Objective. This article describes a training technique for evaluating lay counselors’ competency for mental health lay practitioners without rating scale experience.Methods: Mental health practitioners were trained to give the Enhancing Assessment of Common Therapeutic Factors (ENACT) test to assess counselor proficiency in delivering the Common Elements Treatment Approach (CETA) in-person and over the phone using standardized video and audio recordings. A two-day in-person training was followed by a one-day remote training session. Training includes a review of item scales through didactic instructions, active learning by witnessing and scoring role-plays, peer interactions, and trainer observation and feedback. The trainees rated video and audio recordings, and ICC values were calculated.Results:  The training technique presented in this research helped achieve high counselor competency scores among lay providers with no prior experience using rating scales. ICC rated both trainings satisfactory to exceptional (ICC: .71 - .89).Conclusions.  Raters with no past experience with rating scales can achieve high consistency when rating counselor competency through training. Effective rater training should include didactic learning, practical learning with trainer observation and feedback, and video and audio recordings to assess consistency.
Publisher AOSIS
Date 2022-09-07
Type info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion — —
Format application/pdf
Identifier 10.4081/jphia.2022.2201
Source Journal of Public Health in Africa; Vol 13, No 3 (2022); 12 2038-9930 2038-9922
Language eng
The following web links (URLs) may trigger a file download or direct you to an alternative webpage to gain access to a publication file format of the published article:
Coverage — — —
Rights Copyright (c) 2024 Mwamba M. Mwenge, Caleb J. Figge, Kristina Metz, Jeremy C. Kane, Brandon A. Kohrt, Gloria A. Pedersen, Izukanji Sikazwe, Stephanie Skavenski Van Wyk, Saphira M. Mulemba, Laura K. Murray