SETQ
What
SETQ (System of Evaluation of Teaching Qualities) measures faculty’s teaching qualities. In 2008, the SETQ was introduced in the Netherlands; the first internationally validated version was published in 2013. The SETQ has been updated several times given new research findings. The most recent version is named SETQ2.0.
The SETQ2.0 measures faculty’s teaching qualities using residents’ feedback on seven teaching performance aspects, such as creating a constructive learning climate, prioritizing the resident’s learning, providing feedback to the resident, practice management, and professionalism. SETQ questionnaires are specialty-specific.
The SETQ2.0 invites residents – in addition to numeric ratings – to describe faculty members’ strengths and areas for professional growth. Generally, teaching faculty very much appreciate these ‘free text’ comments, and research shows they use these improvement suggestions to reflect upon and adjust teaching practices.
From medical specialists in a teaching hospital, it is expected that they combine providing responsible patient care with high quality training and supervision of their future colleagues. Since 2015, a permanent quality management cycle (measuring, reflecting, and improving) is mandatory in postgraduate medical education in the Netherlands. The SETQ system can be part of such a cycle. For many supervisors receiving feedback from residents on their teaching performance works supportive and motivating for many faculty members in further improving their own performance.
Evaluation takes place using web-based questionnaires. There is a questionnaire for residents and a (self-evaluation) questionnaire for faculty members. Faculty members fill in one survey (the self-evaluation), residents may fill in several questionnaires (one for each faculty member). It takes approximately 5 minutes to complete each questionnaire. Formulating narrative feedback clear and specific is essential for a good understanding and, therefore, the acceptance of the feedback. Both filling in the questionnaires and receiving the (individual) evaluation reports, which summarizes the feedback per staff member, are automated within the Perito Professional Performance platform.
Faculty members assess themselves and are evaluated by residents. Thousands of faculty from all medical specialties have already used SETQ and received a feedback report.
SETQ participation is anonymous for residents. The feedback reports that faculty members receive only show the number of residents but no recognizable or traceable personal data, such as the year of training or gender. This data is only used as research information. SETQ is also anonymous for faculty members (unless otherwise agreed by the department).
After the pre-agreed measurement period, all faculty members receive an individual feedback report containing the results of all evaluations (residents’ evaluations + self-evaluation). In addition, an anonymous group report is created, summarizing the results of all faculty members within the team or department.
All SETQ2.0 questionnaires have been developed by the AMC in collaboration with faculty members and residents. The quality of the questionnaires is continuously investigated, as well as the use of feedback reports by faculty members (see publications). The questionnaires have been validated and found to be reliable. Renée van der Leeuw, PhD and Benjamin Boerebach, PhD both wrote their PhD dissertations (dissertation van der Leeuw; dissertation Boerebach) on the use of the SETQ system. Other researchers also studied the SETQ in relation to the quality of postgraduate medical education and faculty members’ supervision. (see publication list). Moreover, research on the SETQ questionnaires is being conducted internationally.