2026-03-30
Tejas Jade, Alex Yartsev
Introduction: Standardised tests using short-answer questions (SAQs) are common in postgraduate education. Publicly available large language models (LLMs) are designed to simulate conversational language and interpret free-text responses and could potentially be used to grade SAQs. We evaluated the use of an LLM (ChatGPT 4o) to grade SAQs in a postgraduate medical setting. Methods: An observational retrospective study was performed using data from 215 students (557 short-answer responses) enrolled in an online unit of study on mechanical ventilation (2020–2024). Deidentified SAQ responses to three case-based scenarios on mechanical ventilation were presented to ChatGPT with a standardised grading prompt (see Appendix) and rubric. Analysis used mixed-effects modelling, variance component analysis, intraclass correlation coefficients (ICCs), Cohen's kappa, Kendall's W and Bland-Altman statistics. Results: ChatGPT consistently awarded lower marks than human graders, with a mean difference of -1.34 on a 10-point scale. Agreement was poor at the individual level (ICC1 = 0.086), and Cohen's kappa (-0.0786) indicated no meaningful agreement. While ChatGPT showed internal consistency across five sessions (G-value = 0.87), it diverged significantly from human graders. Agreement was poorest for evaluative and analytic items, while checklist and prescriptive rubric items showed less disagreement. Discussion: Though LLMs remain an attractive option for repetitive tasks involving reading and interpretation of language, we caution against the use of LLMs in grading postgraduate coursework. Over 60% of ChatGPT-assigned grades differed from human grades by more than acceptable boundaries for high-stakes assessments. ChatGPT performed best with rubric items that had clearly defined right and wrong answers, those that allocated marks according to the presence of specific terms or concepts in the answer or those that tested recall (e.g., asked for lists of items). Higher-weighted questions that test more complex learning outcomes are likely to be the most affected. Conclusion: Use of consumer-level LLMs with unmodified rubrics disagreed beyond acceptable standards (10% on 10-mark items) for high-stakes assessments. Promising results for certain rubric types (e.g., checklists) suggest prompt engineering and rubric refinement may play roles in improving future performance in automated grading.
2026-03-30
Tim Fawns
In health professions education, many doctoral students and clinicians begin research projects with a fixed destination that inhibits exploration of alternative ways of understanding a problem. Agendas of supervisors, institutions or funders often push the inquiry towards the development or testing of a predefined intervention, which while appealing, bypasses a critical phase of learning about the problem itself, its context and the assumptions shaping the problem. This paper examines the value of holding research problems open, resisting premature closure in defining a research focus or outcomes. I argue that this can enable richer, more nuanced inquiry that addresses complexity, uncertainty and ambiguity. I discuss what it means to keep the territory of inquiry broad to allow important questions to emerge through sustained and disciplined engagement with the field. I then consider methodological and practical implications of holding the research problem open, including how to respond to pressure for clarity and efficiency. I offer strategies and language for legitimising and defending exploration as part of a rigorous and generative mode of research in health professional education.
2026-03-30
Linda Ferrington, Gordana Popovic, Lillian Smyth, Nicola Kerr, Jennifer Lindley, Conor Gilligan, Michelle Gooey, Hannah Garven, Terri Pikora, Emma Schwartzkoff, Gina Arena
Introduction: Patient involvement in healthcare education improves students’ communication skills, compassion and patient-centred care. These skills may be particularly important for conditions like overweight and obesity in children, where stigma and bias remain pervasive. Through consultation with experts working in the field of obesity in children and people with lived experience of obesity, this study aimed to develop learning outcomes (LOs) about overweight and obesity in children and adolescents for medical curricula. Methods: A multi-phase research methodology was employed, utilising a modified Delphi technique with a panel of experts in overweight and obesity in children and a survey of Australian people with lived experience of obesity to identify learning priorities. The final LOs were refined through collaborative constructive iteration, including virtual bulletin board reviews and modified nominal group technique sessions. Results: The study identified key LOs that the research team recommend all medical students should achieve by graduation. While both experts and people with lived experience felt it was important to recognise overweight and obesity as a critical child health issue in medical curricula, there were notable differences in the LOs prioritised by the two groups. In general, experts prioritised evidence-based knowledge and technical skills, such as the use of body mass index (BMI) and growth charts, while individuals with lived experience emphasised communication skills and addressing weight stigma. Conclusion: The findings highlight the importance of integrating diverse perspectives in curriculum design to better prepare medical students to work with children and adolescents with obesity and their families. By incorporating both expert knowledge and lived experiences, the proposed learning outcomes aim to equip future doctors with the skills needed to provide effective and empathetic care. Implementing these LOs in medical education curricula has the potential to improve the quality of care for children and adolescents with obesity.
2025-12-11
Laura Gray, Bryony McNeill, Nicole Mercer, Tim Walker
Introduction: Medical students face complex challenges that impact their learning and wellbeing, particularly those from marginalised backgrounds. Understanding the student experience is at the cornerstone of providing educational environments in which students can weather the challenges of tertiary education and thrive as learners. Methods: We piloted a survey exploring perceptions of personal and social factors affecting study success with over 200 students in an Australian graduate-entry MD program. Results: Financial precarity, employment, relocation and caring responsibilities were commonly reported as negatively impacting wellbeing. Students also described barriers to disclosure and connection, with many reporting mental health concerns and neurodivergence. Despite challenges, students demonstrated resilience and offered practical suggestions for institutional change. Conclusions: This pilot highlights the value of student-centred data in understanding the complex interplay between identity, circumstance and academic engagement. Findings support calls for inclusive, flexible and compassionate educational practices to better support the diversity of learners and their complex circumstances.
2025-09-29
Louise Allen, Damian Castanelli, Kristie Matthews, Robyn Woodward-Kron, Debra Nestel
Introduction: Health professional education (HPE) PhDs have the potential to shape PhD graduates and the HPE workforce. Yet, little is known about who completes them and the characteristics of their dissertations. This research explored the characteristics of HPE PhD graduates (who) and the contribution to knowledge (what) of HPE PhDs. Methods: This study utilised document analysis of PhD dissertations and website data. We searched the dissertation repositories of two Australian institutions for HPE PhD dissertations. Dissertations available in digital format were included. To explore post-candidature aspects, we searched LinkedIn, institutional websites and Google Scholar. We combined elements of qualitative and quantitative content analysis to analyse the data. Results: We identified 100 HPE PhD dissertations and website data for 92 graduates. We found considerable diversity in those completing PhDs and their dissertation topics, methods and formats. Graduates were most commonly nurses, doctors or physiotherapists and came from 20 different health professions. Sixty-six dissertations included publications, with a median of three (range 1–9). Only 30% were fully funded, and more than half of the graduates (65%) remain working in HPE. Conclusion: Our findings from two Australian institutions provide a foundation for a broader discussion of the “who” and “what” of HPE PhD studies in Australia and Aotearoa New Zealand. The dearth of funding for HPE PhDs, and the relatively small number of HPE PhD graduates, may limit their potential impact. Future research could examine the rigour of PhD dissertations or provide more detail on the impact of PhDs on graduates and their future careers.
2025-06-26
Catherine Ronayne, Hanna Olson, Harsha De Silva, Guangzhao Guan, Kathryn Newsham-West, Megan Anakin
Introduction: Interprofessional competencies have been studied using quantitative methods, predominately involving self-reported measures of knowledge and attitudes of students from a limited number of professions. There is scope to investigate interprofessional competencies using qualitative approaches within underrepresented health professions. We aimed to investigate interprofessional competency development of oral health, medical laboratory sciences and dental surgery students during an interprofessional education initiative. Methods: We used an interpretive perspective to frame this qualitative study involving 15 students and 3 facilitators from the three professions during an oral pathology module composed of three 90-minute sessions. Reflective prompts about role understanding and interprofessional communication were used to generate written responses analysed with a reflexive thematic approach and interpreted in relation to interprofessional competencies. Results: Not only were students learning about role understanding and interprofessional communication, but they were also learning about interprofessional values, coordination and collaborative decision making, reflexivity and teamwork. Students appear to experience these competencies as intertwined, multidirectional and interactive. Conclusions: The experience of interconnectedness among interprofessional competencies should be further investigated with students from health professions beyond medicine. This interconnectedness should be considered when designing, teaching and assessing interprofessional competencies. Additionally, the combination of students should reflect professional work relationships and include currently underrepresented professions.
2025-06-26
Grace Greenham, Dannielle Post, Hannah Cockram, Gaynor Parfitt, Max Nelson
Introduction: Military personnel and first responders experience high rates of physical and psychological injury, reporting numerous barriers to accessing healthcare. These barriers are often related to the lack of population-specific knowledge and skills of health professionals, which may be alleviated by improved education and training. This study investigated education and training outcomes of students completing placement within a program—Invictus Pathways Program (IPP)—designed to promote the wellbeing of these populations. Methods: Survey data were collected pre and post placement using the Health Professionals Attitudes Towards Veterans (HPATV) and Mental Illness Clinicians’ Attitudes (MICA-4) questionnaires for both IPP placement students (n = 41) and a control group (n = 29). Data were analysed using a mixed model analysis of variance. Focus groups were conducted following placement and data analysed using thematic analysis. Results: Significant group by time interactions ( p < 0.05) were identified for all HPATV subscales and the MICA-4 subscales of knowledge of mental illness and distinguishing physical and mental health. Three themes were identified, namely “value of the placement as a professional and personal experience”, “knowledge gaps and misconceptions” and “student support” from the focus group data. Discussion: Results suggest students had an increased understanding of occupational culture, population specific concerns and mental ill health following completion of the IPP placement. Improved knowledge in these areas may address healthcare barriers experienced by veterans and first responders, which may subsequently facilitate greater access to services and improve clinical outcomes. Conclusion: Findings demonstrate students have enhanced knowledge of occupational culture, attitudes towards working with veteran and first responder populations and capacity to work with individuals with mental ill health following completion of the IPP placement.