New tools for systematic evaluation of teaching qualities of medical faculty: results of an ongoing multi-center survey

Background

Tools for the evaluation, improvement and promotion of the teaching excellence of faculty remain elusive in residency settings. This study investigates (i) the reliability and validity of the data yielded by using two new instruments for evaluating the teaching qualities of medical faculty, (ii) the instruments'' potential for differentiating between faculty, and (iii) the number of residents'' evaluations needed per faculty to reliably use the instruments.

Methods and Materials

Multicenter cross-sectional survey among 546 residents and 629 medical faculty representing 29 medical (non-surgical) specialty training programs in the Netherlands. Two instruments—one completed by residents and one by faculty—for measuring teaching qualities of faculty were developed. Statistical analyses included factor analysis, reliability and validity exploration using standard psychometric methods, calculation of the numbers of residents'' evaluations needed per faculty to achieve reliable assessments and variance components and threshold analyses.

Results

A total of 403 (73.8%) residents completed 3575 evaluations of 570 medical faculty while 494 (78.5%) faculty self-evaluated. In both instruments five composite-scales of faculty teaching qualities were detected with high internal consistency and reliability: learning climate (Cronbach''s alpha of 0.85 for residents'' instrument, 0.71 for self-evaluation instrument, professional attitude and behavior (0.84/0.75), communication of goals (0.90/0.84), evaluation of residents (0.91/0.81), and feedback (0.91/0.85). Faculty tended to evaluate themselves higher than did the residents. Up to a third of the total variance in various teaching qualities can be attributed to between-faculty differences. Some seven residents'' evaluations per faculty are needed for assessments to attain a reliability level of 0.90.

Conclusions

The instruments for evaluating teaching qualities of medical faculty appear to yield reliable and valid data. They are feasible for use in medical residencies, can detect between-faculty differences and supply potentially useful information for improving graduate medical education.     

Attitudes toward the use of gender-inclusive language among residency trainees

OBJECTIVE: To explore postgraduate medical trainees'' attitudes toward the use of gender-inclusive language. DESIGN: Self-administered questionnaire. SETTING: Seven residency training programs at McMaster University, Hamilton, Ont., from July 1993 to June 1994. PARTICIPANTS: Of 225 residents in the programs, 186 responded to the survey, for a response rate of 82.7%. Men and women were equally represented among the respondents. OUTCOME MEASURES: Categorization of attitudes about the use of language as gender-inclusive or gender-exclusive; characteristics predicting a gender-inclusive attitude. RESULTS: Factor analysis and Cronbach''s alpha (0.90) supported the existence of a construct related to attitudes about language use, the poles of which were categorized as gender-inclusive and gender-exclusive. The authors classified residents with respect to their attitudes to language use from their responses to the questionnaire. In univariate analyses, sex, residency program and country of graduation significantly predicted a gender-inclusive attitude (p < 0.01). Only the first 2 variables were significant in a multivariate model; residency program explained 18% of the variance and sex 3%. Residents in obstetrics and gynecology and psychiatry had the most gender-inclusive attitudes, whereas residents in surgery and anesthesia had the most gender-exclusive attitudes. CONCLUSIONS: Residents'' values are reflected in the language they choose to use. Language use may provide an index of underlying attitudes that may create hostile environments for female trainees.     

Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: a critical appraisal

OBJECTIVE: To examine the evidence that the teaching of critical appraisal (evidence-based medicine) skills to undergraduate medical students or residents will result in significant gains in knowledge and increased use of the literature in clinical decision-making. DATA SOURCES: Articles published from 1966 to 1995, retrieved through a MEDLINE search supplemented by manual searches; review of bibliographies maintained by individuals involved in teaching critical appraisal skills; and a previous methodological review. STUDY SELECTION: Articles were selected if the study involved some form of control group, although strict randomization was not required, and a measure of performance followed the intervention. Articles were excluded if they simply reported the process of teaching critical appraisal skills or used some form of "happiness index." DATA SYNTHESIS: There were 10 studies of the impact of teaching critical appraisal skills, 6 involving medical students and 4 involving residents. Results from 3 of the studies were nearly uninterpretable and thus were excluded; the remaining 7 were methodologically acceptable. Analysis showed that interventions implemented in undergraduate programs resulted in significant gains in knowledge, as assessed by a written test (mean gain 17.0%; standard deviation [SD] 4.0%). Conversely, studies at the residency level consistently showed a small change in knowledge (mean gain 1.3%; SD 1.7%). Two studies that examined residents'' use of the literature were unable to demonstrate any positive changes. CONCLUSIONS: Studies of the effect of teaching critical appraisal skills on gains in knowledge at the undergraduate level showed consistent improvement. By contrast, changes in knowledge at the residency level were small. Several suggestions from the educational literature are offered to increase effectiveness of critical appraisal interventions.     

Systematic evaluation of the teaching qualities of Obstetrics and Gynecology faculty: reliability and validity of the SETQ tools

Background

The importance of effective clinical teaching for the quality of future patient care is globally understood. Due to recent changes in graduate medical education, new tools are needed to provide faculty with reliable and individualized feedback on their teaching qualities. This study validates two instruments underlying the System for Evaluation of Teaching Qualities (SETQ) aimed at measuring and improving the teaching qualities of obstetrics and gynecology faculty.

Methods and Findings

This cross-sectional multi-center questionnaire study was set in seven general teaching hospitals and two academic medical centers in the Netherlands. Seventy-seven residents and 114 faculty were invited to complete the SETQ instruments in the duration of one month from September 2008 to September 2009. To assess reliability and validity of the instruments, we used exploratory factor analysis, inter-item correlation, reliability coefficient alpha and inter-scale correlations. We also compared composite scales from factor analysis to global ratings. Finally, the number of residents'' evaluations needed per faculty for reliable assessments was calculated. A total of 613 evaluations were completed by 66 residents (85.7% response rate). 99 faculty (86.8% response rate) participated in self-evaluation. Factor analysis yielded five scales with high reliability (Cronbach''s alpha for residents'' and faculty): learning climate (0.86 and 0.75), professional attitude (0.89 and 0.81), communication of learning goals (0.89 and 0.82), evaluation of residents (0.87 and 0.79) and feedback (0.87 and 0.86). Item-total, inter-scale and scale-global rating correlation coefficients were significant (P<0.01). Four to six residents'' evaluations are needed per faculty (reliability coefficient 0.60–0.80).

Conclusions

Both SETQ instruments were found reliable and valid for evaluating teaching qualities of obstetrics and gynecology faculty. Future research should examine improvement of teaching qualities when using SETQ.     

Good clinical teachers likely to be specialist role models: results from a multicenter cross-sectional survey

Context

Medical educational reform includes enhancing role modelling of clinical teachers. This requires faculty being aware of their role model status and performance. We developed the System for Evaluation of Teaching Qualities (SETQ) to generate individualized feedback on previously defined teaching qualities and role model status for faculty in (non) academic hospitals.

Objectives

(i) To examine whether teaching qualities of faculty were associated with their being seen as a specialist role model by residents, and (ii) to investigate whether those associations differed across residency years and specialties.

Methods & Materials

Cross-sectional questionnaire survey amongst 549 Residents of 36 teaching programs in 15 hospitals in the Netherlands. The main outcome measure was faculty being seen as specialist role models by residents. Statistical analyses included (i) Pearson''s correlation coefficients and (ii) multivariable logistic generalized estimating equations to assess the (adjusted) associations between each of five teaching qualities and ‘being seen as a role model’.

Results

407 residents completed a total of 4123 evaluations of 662 faculty. All teaching qualities were positively correlated with ‘being seen as a role model’ with correlation coefficients ranging from 0.49 for ‘evaluation of residents’ to 0.64 for ‘learning climate’ (P<0.001). Faculty most likely to be seen as good role models were those rated highly on ‘feedback’ (odds ratio 2.91, 95% CI: 2.41–3.51), ‘a professional attitude towards residents’ (OR 2.70, 95% CI: 2.34–3.10) and ‘creating a positive learning climate’ (OR 2.45, 95% CI: 1.97–3.04). Results did not seem to vary much across residency years. The relative strength of associations between teaching qualities and being seen as a role model were more distinct when comparing specialties.

Conclusions

Good clinical educators are more likely to be seen as specialist role models for most residents.     

The teacher, the physician and the person: how faculty's teaching performance influences their role modelling

Objective

Previous studies identified different typologies of role models (as teacher/supervisor, physician and person) and explored which of faculty''s characteristics could distinguish good role models. The aim of this study was to explore how and to which extent clinical faculty''s teaching performance influences residents'' evaluations of faculty''s different role modelling statuses, especially across different specialties.

Methods

In a prospective multicenter multispecialty study of faculty''s teaching performance, we used web-based questionnaires to gather empirical data from residents. The main outcome measures were the different typologies of role modelling. The predictors were faculty''s overall teaching performance and faculty''s teaching performance on specific domains of teaching. The data were analyzed using multilevel regression equations.

Results

In total 219 (69% response rate) residents filled out 2111 questionnaires about 423 (96% response rate) faculty. Faculty''s overall teaching performance influenced all role model typologies (OR: from 8.0 to 166.2). For the specific domains of teaching, overall, all three role model typologies were strongly associated with “professional attitude towards residents” (OR: 3.28 for teacher/supervisor, 2.72 for physician and 7.20 for the person role). Further, the teacher/supervisor role was strongly associated with “feedback” and “learning climate” (OR: 3.23 and 2.70). However, the associations of the specific domains of teaching with faculty''s role modelling varied widely across specialties.

Conclusion

This study suggests that faculty can substantially enhance their role modelling by improving their teaching performance. The amount of influence that the specific domains of teaching have on role modelling differs across specialties.     

Effect of the Learning Climate of Residency Programs on Faculty’s Teaching Performance as Evaluated by Residents

Background

To understand teaching performance of individual faculty, the climate in which residents’ learning takes place, the learning climate, may be important. There is emerging evidence that specific climates do predict specific outcomes. Until now, the effect of learning climate on the performance of the individual faculty who actually do the teaching was unknown.

Objectives

This study: (i) tested the hypothesis that a positive learning climate was associated with better teaching performance of individual faculty as evaluated by residents, and (ii) explored which dimensions of learning climate were associated with faculty’s teaching performance.

Methods and Materials

We conducted two cross-sectional questionnaire surveys amongst residents from 45 residency training programs and multiple specialties in 17 hospitals in the Netherlands. Residents evaluated the teaching performance of individual faculty using the robust System for Evaluating Teaching Qualities (SETQ) and evaluated the learning climate of residency programs using the Dutch Residency Educational Climate Test (D-RECT). The validated D-RECT questionnaire consisted of 11 subscales of learning climate. Main outcome measure was faculty’s overall teaching (SETQ) score. We used multivariable adjusted linear mixed models to estimate the separate associations of overall learning climate and each of its subscales with faculty’s teaching performance.

Results

In total 451 residents completed 3569 SETQ evaluations of 502 faculty. Residents also evaluated the learning climate of 45 residency programs in 17 hospitals in the Netherlands. Overall learning climate was positively associated with faculty’s teaching performance (regression coefficient 0.54, 95% confidence interval: 0.37 to 0.71; P<0.001). Three out of 11 learning climate subscales were substantially associated with better teaching performance: ‘coaching and assessment’, ‘work is adapted to residents’ competence’, and ‘formal education’.

Conclusions

Individual faculty’s teaching performance evaluations are positively affected by better learning climate of residency programs.     

Site fidelity,residency, and abundance of bottlenose dolphins (Tursiops sp.) in Adelaide's coastal waters,South Australia

Little is known about the ecology and behavior of southern Australian bottlenose dolphins (Tursiops sp.). This hinders assessment of their conservation status and informed decision‐making concerning their management. We used boat‐based surveys and photo‐identification data to investigate site fidelity, residency patterns, and the abundance of southern Australian bottlenose dolphins in Adelaide's coastal waters. Sighting rates and site fidelity varied amongst individuals, and agglomerative hierarchical cluster analysis led to the categorization of individuals into one of three groups: occasional visitors, seasonal residents, or year‐round residents. Lagged identification rates indicated that these dolphins used the study area regularly from year to year following a model of emigration and reimmigration. Abundance estimates obtained from multisample closed capture‐recapture models ranged from 95 individuals (SE ± 45.20) in winter 2013 to 239 (SE ± 54.91) in summer 2014. The varying levels of site fidelity and residency, and the relatively high number of dolphins found throughout the study area highlights the Adelaide metropolitan coast as an important habitat for bottlenose dolphins. As these dolphins also appear to spend considerable time outside the study area, future research, conservation, and management efforts on this population must take into account anthropogenic activities within Adelaide's coastal waters and their adjacencies.     

Financial incentives of subregional RAWP.

Accounting for the cross boundary flows of residents from one health authority treated by another has been considered by the review of the Resource Allocation Working Party (RAWP) formula by the National Health Service Management Board. A common concern is that the approximate costs used are unfair to those authorities (typically those with teaching hospitals) that are likely to treat more complex cases. This paper argues that when spending exceeds the target allowance for acute services this is more likely to be due to district residents using services at a high rate than to inadequate compensation for inflows. Districts where residents make a high use of services are often those where there are large flows across district boundaries. Since authorities cannot control outflows there is little they can do to reduce their residents'' high use of services. Furthermore, curious financial incentives can be inferred for clinicians in these districts if they were to take effective action to bring their district''s spending to target levels. These problems are discussed to illuminate problems of accounting for cross boundary flows that alternatives to current practice must resolve.     

Resident and transient coyotes exhibit differential patterns of movement behavior across heterogeneous landscapes in the southeastern United States

Coyotes (Canis latrans) are a highly adaptable canid species whose behavioral plasticity has allowed them to persist in a wide array of habitats throughout North America. As generalists, coyotes can alter movement patterns and change territorial strategies between residency (high site fidelity) and transiency (low site fidelity) to maximize fitness. Uncertainty remains about resident and transient coyote movement patterns and habitat use because research has reached conflicting conclusions regarding patterns of habitat use by both groups. We quantified effects of habitat on resident and transient coyote movement behavior using first passage time (FPT) analysis, which assesses recursive movement along an individual''s movement path to delineate where they exhibit area‐restricted search (ARS) behaviors relative to habitat attributes. We quantified monthly movement rates for 171 coyotes (76 residents and 53 transients) and then used estimated FPT values in generalized linear mixed models to quantify monthly habitat use for resident and transient coyotes. Transients had greater movement rates than residents across all months except January. Resident FPT values were positively correlated with agricultural land cover during fall and winter, but negatively correlated with agriculture during spring. Resident FPT values were also negatively correlated with developed habitats during May–August, deciduous land cover during June–August, and wetlands during September–January except November. FPT values of transient coyotes were positively correlated with developed areas throughout much of the year and near wetlands during July–September. Transient FPT values were negatively correlated with agriculture during all months except June and July. High FPT values (ARS behavior) of residents and transients were generally correlated with greater densities of edge habitat. Although we observed high individual variation in space use, our study found substantive differences in habitat use between residents and transients, providing further evidence that complexity and plasticity of coyote habitat use is influenced by territorial strategy.     

Residency Duration and Shelter Quality Influence Vibratory Signalling Displays in A Territorial Caterpillar

Residents are more likely to win territorial disputes than intruders. One explanation for this prior resident advantage is that residents place a higher value on the resource and are therefore more motivated to win. Although value asymmetry models of animal contests often assume that contestants use information about resource value, information on the proximate cues affecting territorial behaviour is often lacking. We use a simple model system – territorial behaviour in the masked birch caterpillar (Drepana arcuata) to identify factors that affect territorial behaviour. Late instar caterpillars occupy solitary silken leaf shelters, which they defend against wandering conspecifics with a vibratory display. We evaluated how a caterpillar identifies itself as the owner and the factors that influence a resident's motivation to signal. To do so, we conducted three experiments between size‐matched residents and intruders to assess how residency duration and shelter quality independently affected territorial displays during the early stages of a contest. Experiment 1 (Time Exp.) demonstrated that resident signalling rates increase with increased duration on the leaf prior to introducing the intruder. Residents also signal more than intruders after residency periods of 1–3 min and longer, demonstrating that residents gather information about resource value shortly after occupying a leaf. Experiment 2 (Squatter Exp.) aimed to disentangle the effects of time on the leaf and silk accumulation. Squatters (individuals in a shelter made by another) placed for 1–3 min on a leaf containing a full silk shelter signalled more to intruders than did caterpillars placed on a fresh leaf for 1–3 min. Experiment 3 (Shelter Removal Exp.) showed that residents whose shelters had been removed signal less than those occupying an intact shelter, despite an equal length of time investing in them. Our experiment is the first to covary both prior residency duration and territory quality, and we find that the motivation of caterpillars to signal is a function of both of these attributes.     

Teacher training for medical faculty and residents.

Since 1984 the University of British Columbia''s School of Medicine has offered teaching improvement project systems (TIPS) workshops on effective teaching techniques; two workshops a year are given for medical faculty members and two a year for residents. The faculty members who conduct the workshops have received training on how to present them. The most powerful learning experience offered by TIPS is the opportunity for participants to present 10-minute teaching segments that are videotaped and later viewed privately by the participants. Eight workshops have been attended by 166 faculty members, and two others have been attended by 42 residents. This project demonstrates faculty development for both the participants and the people who teach the workshops.     

University acknowledges special risks,introduces drug program for anesthetists.

Anesthetists are at special risk for becoming addicted to some of the drugs they work with. The problem was highlighted by the recent death of a physician in Comox, BC, and a doctor''s brush with death at a hospital near Ottawa. The University of Ottawa has responded with a program, the professional Assistance Program for the Impaired or Disabled Physician, that is in the final stages of approval. Although it will be aimed initially at anesthetists and anesthesia residents in the university''s teaching hospitals, there are hopes the program will spread eventually to other medical departments.     

Residents' experiences of abuse,discrimination and sexual harassment during residency training. McMaster University Residency Training Programs.

OBJECTIVE: To assess the prevalence of psychological abuse, physical assault, and discrimination on the basis of gender and sexual orientation, and to examine the prevalence and impact of sexual harassment in residency training programs. DESIGN: Self-administered questionnaire. SETTING: McMaster University, Hamilton, Ont. PARTICIPANTS: Residents in seven residency training programs during the academic year from July 1993 to June 1994. Of 225 residents 186 (82.7%) returned a completed questionnaire, and 50% of the respondents were women. OUTCOME MEASURES: Prevalence of psychological abuse, physical assault and discrimination on the basis of gender and sexual orientation experienced by residents during medical training, prevalence and residents'' perceived frequency of sexual harassment. RESULTS: Psychological abuse was reported by 50% of the residents. Some of the respondents reported physical assault, mostly by patients and their family members (14.7% reported assaults by male patients and family members, 9.8% reported assaults by female patients and family members), 5.4% of the female respondents reported assault by male supervising physicians. Discrimination on the basis of gender was reported to be common and was experienced significantly more often by female residents than by male residents (p < 0.01). Ten respondents, all female, reported having experienced discrimination on the basis of their sexual orientation. Most of the respondents experienced sexual harassment, especially in the form of sexist jokes, flirtation and unwanted compliments on their dress or figure. On average, 40% of the respondents, especially women (p < 0.01), reported experiencing offensive body language and receiving sexist teaching material and unwanted compliments on their dress. Significantly more female respondents than male respondents stated that they had reported events of sexual harassment to someone (p < 0.001). The most frequent emotional reactions to sexual harassment were embarassment (reported by 24.0%), anger (by 23.4%) and frustration (20.8%). CONCLUSION: Psychological abuse, discrimination on the basis of gender and sexual harassment are commonly experienced by residents in training programs. A direct, progressive, multidisciplinary approach is needed to label and address these problems.     

Recent initiatives for radiation oncology resident education in radiation and cancer biology

Nearly all residents from accredited radiation oncology residency programs in the United States are required to take the American College of Radiology (ACR) In-Training examination each year. The test is comprised of three sections: Clinical Radiation Oncology, Radiological Physics, and Radiation (and Cancer) Biology. Here we provide an update on changes to the biology portion of the ACR exam. We also discuss the availability and use of the ACR and biology practice exams as assessment and teaching tools for both the instructors of radiation and cancer biology and the residents they teach.     

Little room for error in Canada's postgraduate training system

Although the majority of physicians entering residency training in Canada will enjoy fulfilling careers in their chosen specialty, today''s postgraduate training system has its critics. Among them are the new graduates who are not satisfied with the residency positions offered to them and practising physicians who would like to re-enter the system to train in a new specialty but find themselves locked out.     

New York hospitals paid to teach fewer physicians

In order to reduce the number of physicians being trained in the US, teaching hospitals in New York are going to be paid not to train residents. Participating hospitals will cut the number of residents they train by up to 25%, but for a time will be paid as if they are still teaching a full complement of trainees. Up to 400 residency positions will be cut annually under the plan.     

Training and Experience of Physicians in the Management of Adequate Anticoagulant Therapy

In a group of seven physicians engaged on a full-time basis in the anticoagulant unit of a teaching hospital, it was demonstrated that errors occur frequently in the management of anticoagulant therapy and that the incidence of such errors decreases during the course of the physician''s training period. Expert supervision and proper organization of the unit permit detection and correction of a high percentage of these errors.     

对编写上海市医学影像科(放射科)住院医师规范化培训教材的思考与探讨

始于2010年的上海市住院医师规范化培训模式是我国新的医师培养模式,经历3年的实践及完善,从制度及培训基地建设与管理、培训模式及教学经验等方面均取得了可喜的成绩。鉴于放射科住院医师在一定程度上的特殊性,本文以我院放射科基地为例,从基地医师的教育及知识背景、后续的继续教育及职业规划等方面对编写上海市医学影像科(放射科)住院医师规范化培训教材的必要性及可行性进行了分析与探讨,然后对编写此教材的定位、知识体系、参编人员构成、教材编写中的注意事项等提出了具体的看法并进行了初步探讨,供各基地教学实施过程中借鉴,共同推动上海市医学影像科(放射科)住院医师规范化工作,提高医学影像学(放射科)规范化培训医师的整体水平。     

Dying in hospital: the residents' viewpoint

A survey of residents'' (junior house officers'') experiences and attitudes to the terminal care part of their work in four Glasgow teaching hospitals showed that even a month after starting work one-fifth of the respondents had not actively managed a dying patient. Sixty-four per cent thought that they had received inadequate teaching in terminal care. Depression and anxiety had been the most difficult symptoms encountered. The residents thought that the ward nursing staff contributed much more than their senior medical colleagues to both the medical and psychological aspects of terminal care. The results indicate a need for more undergraduate education in the most relevant areas, such as coping with the psychological problems of dying patients and their relatives. Newly qualified residents require more support from senior medical staff in looking after the terminally ill.