Electronic Letters to:

General Orthopaedics: N. R. Howells, H. S. Gill, A. J. Carr, A. J. Price, and J. L. Rees Transferring simulated arthroscopic skills to the operating theatre: A RANDOMISED BLINDED STUDY J Bone Joint Surg Br 2008; 90-B: 494-499 [Abstract] [Full text] [PDF]

eLetters: Submit a response to this article

Electronic letters published:

Authors' reply:

Jonathan L Rees   (8 May 2008)

Psychometric properties of the assessment are as important as those of the simulator

Anish P Sanghrajka, Amit Amin   (16 April 2008)

Authors' reply: 8 May 2008
Jonathan L Rees, University Lecturer in Orthopaedic Surgery and Consultant Orthopaedic Surgeon Nuffield Department of Orthopaedic Surgey Send letter to journal: Re: Authors' reply: jonathan.rees{at}ndos.ox.ac.uk Jonathan L Rees	Sir, We thank Mr Sanghrajka and Mr Amin for their comments and interest in our paper. Essentially, the aim of our study was to assess the usefulness of orthopaedic surgical training on an appropriate simulator prior to training in the operating theatre. We would disagree that the magnitudes of the global rating scores (GRS) are ‘surprisingly high’ after training on a low-fidelity model. Diagnostic arthroscopy is a simple technical task and the sawbones simulator provides good face validity for this procedure. The sawbones simulator, although low-fidelity, provides practice in exactly what was asked of the trainees in-vivo. The scores were higher for the more basic yes/no Orthopaedic Competency Assessment Project (OCAP) checklist compared with the GRS used. The GRS scores themselves are not that high, with median scores being 24 out of 45 for the simulator-trained group. This score as a percentage compares with the findings of other surgical procedures assessed using seven and eight GRS domains. The OCAP checklist has been developed and validated by others. We did modify the GRS domains used by Reznick as others have done. While a weakness of our study is that we have not yet fully validated this GRS we feel it is appropriate for a number of reasons. The domains are usually selected by a collective expert opinion and we used the nine assessment domains already chosen by the OCAP. These domains match at least four of the original seven GRS domains used by Reznick, and included all the technical domains. We did pilot the GRS on more senior surgeons who outscored the junior trainees, indicating construct validity. Finally, we are not proposing it as a valid and reliable score for use in orthopaedic surgical skills in general, but for the simple technical task in question, scored by the same independent observer it has provided a useful objective measure for the purposes of this study. J.L. Rees MD, FRCS(Tr&Orth) N.R. Howells BSc, MRCS, H.S. Gill DPhil, A.J. Carr ChM FRCS, A. Price MA, DPhil, FRCS(Tr&Orth), Nuffield Department of Orthopaedic Surgery, University of Oxford, Nuffield Orthopaedic Centre, Oxford, UK.
Psychometric properties of the assessment are as important as those of the simulator 16 April 2008
Anish P Sanghrajka, Specialist Registrar The Royal London Hospital, UK, Amit Amin Send letter to journal: Re: Psychometric properties of the assessment are as important as those of the simulator anish{at}dr-anish.com Anish P Sanghrajka, et al.	Sir, We read this paper with interest and we would like to congratulate the authors for conducting such a worthwhile study. Intuitively, one would expect prior practice on a simulator to improve performance in the operating theatre. However, we were surprised at the magnitude of the global rating scores attained by the simulator-group trainees on their first attempt at a real arthroscopy. The Sawbones simulator is of relatively low fidelity, and it is difficult to see how this simulator would improve performance in some of the domains assessed by the global rating scale (GRS). This leads us to question the psychometric properties of the GRS used in this study. Are we correct in understanding that the authors have created this rating system de novo, based on the principles of the GRS of the Surgical Education Research Center in Toronto? The authors refer to the work of Reznick et al1 to support the validity of their GRS; whilst that group effectively demonstrated the psychometric properties of their own structured assessments, their work cannot be used to automatically validate a completely new GRS. Can the authors provide information about the internal consistency, reliability and validity of their own global rating scoring system? Weaknesses in these characteristics could explain the unexpectedly high scores, and would affect the significance of the results of this study. A.P. Sanghrajka, Specialist Registrar, A. Amin, The Royal London Hospital, London, UK. 1. Reznick R, Regehr G, MacRae H, Martin J, McCulloch W. Testing technical skill via an innovative “bench station” examination. Am J Surg 1997;173:226-30.