|Year : 2022 | Volume
| Issue : 2 | Page : 189-193
Neurology exit examination system in India: A survey of examiners' perceptions and recommendations
GR K Sarma1, Saji K John1, Thomas Mathew1, Venkatesh Aiyagari2, Sivaraman Nair3, Gareth J Parry1, Satish Khadilkar4, Parthasarathy Satishchandra5
1 Department of Neurology, St. John's Medical College Hospital, Sarjapura Road, Bengaluru, Karnataka, India
2 Department of Neurological Surgery and Neurology, UT Southwestern Medical Center, Texas, USA
3 Department of Neurology, Sheffield Teaching Hospital NHS Foundation Trust Royal Hallamshire Hospital, Glossop Road, Sheffield S10 2JF, United Kingdom
4 Department of Neurology, Bombay Hospital Institute of Medical Sciences, Mumbai, Maharashtra, India
5 Department of Neurology, National Institute of Mental Health and Neurosciences, (NIMHANS), Bengaluru, Karnataka, India
|Date of Submission||22-Jul-2021|
|Date of Acceptance||28-Oct-2021|
|Date of Web Publication||06-Apr-2022|
G R K Sarma
Professor, Department of Neurology, St. John's Medical College Hospital, Sarjapura Road, Bengaluru - 560 034, Karnataka
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: The traditional Neurology exit examination in India has remained unchanged over the last few decades. In developed countries, objective evaluation methods have replaced the traditional ones. A need for such methods has not been explored in India. Objective: We aimed to study the perceptions and key recommendations of Neurology examiners on the existing examination pattern. Material and Methods: We conducted an online survey of examiners perceptions and recommendations using a set of 10 multiple-choice questions and an open-ended question. Results: 46 examiners provided completed responses suitable for analysis. Nearly equal proportions (30%) of the examiners had 10 years, 10–25 years and >25 years' experience. 92% were not satisfied with current system, 95% did not find adequate time for correction of theory scripts, 90% felt that theory questions were random, and 95% had legibility issues. 84% felt that the practical exams do not test true learning, 98% felt the examination stress impairs the performance and 85% felt that there are no objective criteria to pass the candidate. 83% felt the current system-needed changes. The key suggestions provided by the examiners to improve the system included objective assessments like MCQ, OSCE, OSLER and DOPS, inclusion of larger number of short answer type questions and periodic internal assessments of the candidates. Conclusions: A vast majority of examiners favoured changes to the current examination system and provided key recommendations. A larger study is needed to extrapolate these findings to the rest of India.
Keywords: DM, neurology examination, OSCE, OSLER
|How to cite this article:|
K Sarma G R, John SK, Mathew T, Aiyagari V, Nair S, Parry GJ, Khadilkar S, Satishchandra P. Neurology exit examination system in India: A survey of examiners' perceptions and recommendations. Ann Indian Acad Neurol 2022;25:189-93
|How to cite this URL:|
K Sarma G R, John SK, Mathew T, Aiyagari V, Nair S, Parry GJ, Khadilkar S, Satishchandra P. Neurology exit examination system in India: A survey of examiners' perceptions and recommendations. Ann Indian Acad Neurol [serial online] 2022 [cited 2022 Jul 3];25:189-93. Available from: https://www.annalsofian.org/text.asp?2022/25/2/189/342615
Neurology Exit Exam Study Group Investigators as part of the Neurology Exit Exam (NEE) Group are as Follows:
Raghunandan Nadig, Amar BR1 , Amit Kulkarni2 , Aravind N. Prabhu3, Ashalatha Radhakrishnan4 , A. V Srinivasan5 , B. Vengamma6, Garuda Butchi Raju7 , Debashish Chaudhary8 , Hariram Acharya9, Jacob George10, Jagadish B. Agadi11 , Jeyaraj Pandian12, Jagarlapudi Murali Krishna Murthy13, Kurupath Radhakrishnan14 , Lakshminarasimhaiah Krishnamurthy15 , Mohan Madhusudanan16 , Mahendra Javali17 , Manjunath Mahadevappa18 , Manjunath Desai19 , Nalini Atchayaram20 , Neeraj Bhutani21 , Netravathi Manjunath20 , PT Acharya17 , Rajesh Karlumangala Nagarajaiah22 , Ravi Yadav20 , Sailesh Modi23 , Santhosh Nandanavana Subbareddy24 , Saravanan S25, Shivanand Pai3, Sita Jayalakshmi26 , Salvadeeswaran Meenakshi-Sundaram27 , Srikanth Reddy28 , Srinivas Raju29 , Subhash Kaul30, Sunil Gajanan Gajre31, Suryanarayana Sharma32 , Meenakshisundaram Umaiorubahan33 , UK Mishra34, Veerendra Kumar Mustare20, Vikram Huded9 , Vivek Lal35
Department of Neurology, St. Johnís Medical College Hospital, Sarjapura Road, Bengaluru, Karnataka, India, 1 Department of Neurology, Apollo Institute of Neurosciences, Jayanagar, Bengaluru, Karnataka, India, 2 Department of Neurology, Sparsh Hospitals, Bengaluru, Karnataka, India, 3 Department of Neurology, Kasturba Medical College, MAHE, Manipal, Karnataka, India, 4Department of Neurology, Sree Chitra Thirunal Institute of Medical Sciences and Technology, Trivandrum, Kerala, India, 5Department of Neurology, MGR Medical University, Chennai, Tamil Nadu, India, 6 Department of Neurology, Sri Venkateswara Institute of Medical Sciences, Thirupathi, Andhra Pradesh, India, 7Department of Neurology, Andhra Medical College, Visakhapatnam, Andhra Pradesh, India, 8Department of Neurology, GB Pant Hospital, New Delhi, India, 9 Department of Neurology, Narayana Institute of Health Sciences, Bengaluru, Karnataka, India, 10 Department of Neurology, Government Medical College, Kottayam, Kerala, India, 11 Department of Neurology, Apollo Hospital Sheshadripuram, Bengaluru, Karnataka, India, 12 Department of Neurology, Christian Medical College, Ludhiana, Punjab, India, 13Department of Neurology, Care Hospital, Hyderabad, Telangana, India, 14 Department of Neurology, Avitis Institute of Medical Sciences, Nemmara, Palakkad, Kerala, India, 15 Department of Neurology, JJM Medical College and Hospital, Davangere, Karnataka, India, 16 Department of Neurology, Travancore Medicity Medical College and Hospital, Kollam, Kerala, 17 Department of Neurology, Ramaiah Medical College, MSRIT Post, Bengaluru, Karnataka, India, 18Department of Neurology, Manipal Hospital, Bengaluru, Karnataka, India, 19 Department of Neurology, SDM College of Medical Sciences, Dharwad, Karnataka, India, 20 Department of Neurology, National Institute of Mental Health and Neurosciences, (NIMHANS), Bengaluru, Karnataka, India, 21 Department of Neurology, Santokba Durlabhji Memorial Hospital, Jaipur, Rajasthan, India, 22 Department of Neurology, Trustwell Hospitals and People Tree Hospitals, Bengaluru, Karnataka, India, 23 Department of Neurology, Queens NRI Hospital, Visakhapatnam, Andhra Pradesh, India, 24 Department of Neurology, Manipal, Hospital Whitefield, Bengaluru, Karnataka, India, 25 Department of Neurology, Sagar Hospital, Bengaluru, Karnataka, India, 26Department of Neurology, Krishna Institute of Medical Sciences, Minister Road, 1-8-31/1 Secunderabad, Telangana, India, 27 Department of Neurology, Apollo Speciality Hospitals, Madurai, Tamil Nadu, India, 28 Department of Neurology, Yashoda Hospital, Hyderabad, Telangana, India, 29 Department of Neurology, Bangalore Baptist Hospital and Columbia Asia Hebbal, Bengaluru, Karnataka, India, 30 Department of Neurology, Nizamís Institute of Medical Sciences, Hyderabad, Telangana, India, 31 Department of Neurology, Gajre Hospital, Jalgaon, Maharashtra, India, 32Department of Neurology, Apollo Hospitals Bannerghatta Road, Bengaluru, India, 33Department of Neurology, SRM Institutes for Medical Science, Chennai, Tamilnadu, India, 34 Department of Neurology, SGPGI, Lucknow, Uttar Pradesh, India, 35 Department of Neurology, Post Graduate Institute of Medical Education and Research, Chandigarh, India
| Introduction|| |
The purpose of an exit examination at the completion of training is to ensure a minimum level of competency to grant a license to practice Neurology. Until a few decades ago, medical examinations in the developed countries were in the form of essays and oral assessments. Evaluations derived from such assessments were subjective, arbitrary and not reproducible. Subsequently, standardized tests such as multiple-choice questions (MCQs) or Objective Structured Clinical Examinations (OSCE) were developed., Worldwide, the Neurology residency exit examination system is highly varied,,, and in some countries, there is no formal exit examination.
In India, the neurology exit examinations for granting DM (Doctor of Medicine) degree generally consist of written tests (usually 10 questions each carrying 10 marks) covering basic sciences, clinical neurology, investigations, therapeutics and recent advances, and practical clinical examinations consisting of a “long case” and 2-3 “short cases”. This is followed by neuroradiology spotters, electrophysiology spotters, neuropathology slides and viva voce. There are often variations in this scheme from one university to another. Though this examination pattern appears elaborate and comprehensive, the theory examination and evaluation, viva voce and case presentations are largely unstructured and without a clear blueprint for objective evaluation. During the conduct of the DM Neurology examinations, several opinions and suggestions have been put forward and discussed informally by the examiners regarding the need to revise the current system. In a recent editorial (2020), Pauranik noted the wide variability of quality in DM training and assessment in India, emphasized the need for systematic research in this area and urged the Indian Academy of Neurology to take the lead in this direction. As the number of DM Neurology seats in the country continues to increase over time, it is important to have a system of exit examination that ensures uniform minimum national standards of the trainees. The objectives of our study are to systematically study the perception of examiners from various parts of India on the existing examination pattern for DM Neurology and to know their key recommendations.
| Methods|| |
This study is an online survey of DM Neurology examiners in India, conducted by the senior neurology faculty of a tertiary care medical college hospital in Bengaluru, Karnataka state in South India. This study did not probe the DNB (Diplomate of National Board) Neurology examination system, which is also a Neurology certification exam conducted by the National Board of Examinations. A Google survey form was created with a set of 10 single-response-multiple-choice questions [Table 1] covering theory and practical examination pattern (five questions each). The questionnaire was designed according to the guidelines of the Association for Medical Education in Europe and were internally validated. For each question, there were 4 Likert-type response options, and the responses were set in such a way that the score value was proportionate to the degree of satisfaction with the existing system. In addition to these 10 questions, an open-ended question was posed seeking any suggestions and recommendations from the examiners to improve the examination system. The questionnaire was electronically mailed to 60 examiners who have been in the examiner data base of the college. The university to which the college is affiliated, requires the examiners to be preferably chosen from geographically proximate locations, and consequently, the data base is heavily weighted towards Karnataka and other southern states of the country. Incomplete responses and those without the respondent details were excluded from analysis. In addition to the answers to the questions, respondent data was collected including years of experience in DM examinations, and city/town of practice. The data were subjected to descriptive analysis. The final manuscript was approved by all the respondents with relevant inputs. Descriptive statistics including median, inter quartile and percentages were used to summarize the data. Fisher's exact test was used to find associations. Software version 4.0.2 was used for data analyses (R Core Team, 2020, Vienna, Austria; www.R-project.org/) A value of P ≤ 0.05 was considered statistically significant.
|Table 1: Online survey questionnaire on DM neurology examination pattern: Responses|
Click here to view
| Results|| |
Of 60 respondents, 46 (77%) provided complete data suitable for final analysis. The years of experience as an examiner ranged from 3 to 40 years with a median of 20 years and interquartile range of 12-34 years. Approximately equal proportion had experience of less than 10 years (32.5%), 10–25 years (32.5%) and more than 25 years (35%) as examiners [Figure 1]. There were 40 men and 6 women among the respondents.
|Figure 1: Distribution of the neurology examiners by the number of years of experience|
Click here to view
Nearly half the respondents (24/47) were from the state of Karnataka. Of these, 20 were from the city of Bengaluru. 17 were from various other states of south India and 5 were from the northern states [Figure 2]. This distribution reflects the database of examiners of the college, which is dominated by examiners from proximate geographical locations. In all, 11 states were represented in this survey. The responses to each of the 10 questions are discussed below.
Only 8% of the respondents were extremely satisfied with the current examination system. Nearly half (46.9%) of the respondents are unsatisfied or extremely unsatisfied. Just 5% felt they get adequate time to read the theory answer papers thoroughly and allot the marks appropriately. 55% strongly felt that they do not get adequate time to perform this role. Only 10% felt that theory questions are set thoughtfully to assess the students' learning. The remaining majority felt that the questions are set in a random fashion. 95% had some difficulty to extreme difficulty with regards to the legibility of the answer scripts. Only a minority (16.3%) felt that the practical exams reflect the true learning and competence of the students. About 40% felt that they do not reflect true learning and competence to a large or extremely large extent.
98% felt that the stress and anxiety of practical examination ambience adversely affected the students' performance. Only about 14% felt that the examiners set clearly defined criteria to pass or fail a candidate in the practical examination. Nearly 25% felt there are absolutely no well-defined criteria to reach this decision. Nearly 88% felt that often to very often, there are chances of competent students failing or non-competent students passing the exams. About 49% felt that revamping the current system of theory and practical exams is very much needed. 34% felt that it is needed to some extent to modify the existing system. Only 16% felt that the current system should continue unchanged. 60% strongly felt that an objective system of examination is needed. 20% felt that objective system of examination is not an appropriate method. There was no statistically significant association between years of experience and the responses to any of the questions. Key recommendations provided by the examiners to improve the examination system are shown in [Table 2].
| Discussion|| |
For an assessment of a medical student, no single method is ideal and a mixture of methods is available based on the context and purpose of assessment. The consideration of utility criteria of the assessment tool like objectivity, validity, reliability, educational impact, acceptability, and feasibility can be helpful in making an appropriate choice.” In real life, there is always some swapping between validity and reliability. For high stakes, competitive examinations, more reliable instruments are used, whereas, for formative assessments, tools with higher educational impact should be chosen. There is a need to design tests that ensure that a license to practice is given only to competent and qualified physicians who have achieved a minimum standard. An apt statement by Gleeson needs our attention: “the examiner has a duty not to release a candidate, who is not properly prepared, on an unsuspecting public.”
From this survey, it is apparent that a majority of the examiners are not satisfied to some or a large extent with the current pattern of theory and practical examinations and are in favour of a more objective, valid and reliable system of assessment. A number of suggestions have been put forward for consideration, though no system is perfect and each one has its merits and demerits, For theory examinations, multiple-choice questions provide a more objective assessment than long and short answer type questions and they also eliminate the answer script legibility issues. Including several short answer descriptive questions, instead of few long answer questions, provides an opportunity to cover wider areas of the field of neurology. However, a long essay type of question assesses the ability of a trainee to construct a coherent account of a given topic in a limited time and this cannot be assessed by short answer or multiple-choice questions. Such writing skills are important for academicians and researchers in the field of Neurology. Thus a combination of long answers, short answers and MCQs could strike a balance between the breadth and depth of knowledge of the examinees. Careful structuring and a pre-set marking scheme will be required to avoid the inter evaluator variability in the marking and evaluation of short and long answer questions.
For practical examinations, one of the many suggestions put forward is implementation of OSCE stations. It is noteworthy that some institutes in India (e.g., the All India Institute of Medical Sciences-AIIMS, New Delhi) already have this system in place. and it would be important for other centres and universities to assess the feasibility of replicating this system.
Each of the OSCE stations will have one patient or a scan or EEG result or set of blood results with a pre-set questions. The student will read the question and perform the task in a fixed time. The performance could be recorded electronically. The student will be assessed by at least one examiner. Hence, the results will be the sum of multiple performances assessed by multiple examiners. This method will make it more objective and reduce the chance of individual bias from affecting the results of assessment. The system also provides legal protection for the examiners, in case the trainee legally challenges the results.
The National Board of Examinations has been conducting case scenario-based, objective questions to assess the clinical skills. Some suggestions given by the survey participants are concurrent with this model. It will be interesting to see if this model can be incorporated in the DM examination system without diluting the existing system of bedside clinical assessment. Some colleges have been using OSLER (objective structured long examination record) for assessing the long case. This could be adopted by all universities to bring in some uniformity and objectivity in the assessment of bedside cases. A major lacuna in the current system is that a trainee's skills in procedures like nerve conduction studies, electromyography and electroencephalography are not assessed and only the interpretation of the recorded graphs is assessed. This could be overcome by using DOPS (directly observed procedural skills) method Here the examiner observes the candidate performing a task like EMG. The task is divided into different components like consent, explanation of the procedure to the patient, actual conduct of the test, interpretation of the results, and explanation of the results to the patient. Examiner will score each of these components. A minimum of DOPS done during training and signed by a qualified faculty member in a logbook or electronic portfolio could also be used. This type of work-based DOPS provides an opportunity for the trainee to get feedback and develop the skills further. In addition, a comprehensive review and recommendation by Ranjan et al. on National Exit exam (NEXT) discusses these issues in detail and is applicable to Neurology DM examination as well.,
During an oral examination numerous sources of error occur, to which examiners are subject in the framework., For example, with the primacy effect, first impressions dominate over later impressions and, with the recency effect, later impressions are more lasting. In the halo effect, the perception and evaluation of one property outshines the perception and evaluation of other properties. Antipathy, sympathy and the composition of the examiners also have an influence on the evaluation of the test performance. Jayawickramarajah et al. demonstrated that two-thirds of the questions in an unstructured oral examination exclusively examined factual knowledge. An additional problem of an unstructured oral examination is the high probability of an occurrence of Construct Irrelevant Variance (CIV) due to the fact that too few examiners are used. CIV could occur, for example, when the testing of the competence in “clinical decision making” is influenced by the appearance, fear, language skills or clothing of the examinee. Construct Underrepresentation (CU) is a further hurdle that must be considered in the context of an unstructured oral examination, since, for example, two to three clinical scenarios that are tested cannot cover the entire range of the subject area to be tested. Concerns about the validity of this traditional form of examination have led to it being replaced by written examinations or structured oral examinations in some countries.
There is a need to explore these options to bring in more objectivity, validity, and reliability to oral examinations.
The strength of this study is that about 65% of the respondents had experience of more than 10 years and 35% had greater than 25 years of experience. Thus, their perceptions and suggestions would be of immense value in guiding policy makers in their efforts to bring about a change in the examination system. The respondents are also from different states of the country, giving a reasonably broad coverage on the examination pattern in these regions. The main limitation of this survey is the restricted geographical representation of the respondents in a vast country like India. Thus, these results cannot be extrapolated to the country without conducting larger surveys of examiners from other parts of the country. Another limitation is the small number of questions provided. We have also not included questions on other important domains like communication skills, humanistic qualities and counseling skills. However, this study is still relevant and valuable as a starting point for the universities and examiners to review the current system of examinations and as a stimulus for a gradual shift to a more objective, valid and reliable system.
To conclude, this online survey of DM Neurology examiners showed that a vast majority of the examiners are not satisfied with the current system of assessment. Their key suggestions to improve the system include objective assessments like MCQ, OSCE, OSLER and DOPS, inclusion of larger number of short answer type questions in addition to the long answer type questions, periodic internal assessments and training or orientation of examiners. A larger detailed survey from other parts of the country followed by consensus guidelines developed by a working group of the Indian Academy of Neurology are essential to bring about the much-needed improvements. Implementation of such guidelines hopefully ensures uniform minimum national standards for the neurology exit examination in India.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Norcini J, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, et al
. Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011;33:206-14.
Harden MR, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. Br Med J 1975;1:447-51.
Newble D. Techniques for measuring clinical competence: Objective structured clinical examinations. Med Educ 2004;38:199-203.
Bergen DC, Good D. Neurology training programs worldwide: A world federation of neurology survey. J Neurol Sci 2006;246:59-64.
Wills AJ. Neurology postgraduate training: What is to be done? J Neurol Neurosurg Psychiatry 2004;75:1513-5.
Thiessen N, Fischer MR, Huwendiek S. Assessment methods in medical specialist assessments in the DACH region-Overview, critical examination and recommendations for further development. GMS J Med Educ 2019;36:78.
Pauranik A. What NEXT for neurology education and assessment? Ann Indian Acad Neurol 2020;23:151-2. [Full text]
Artino AR Jr, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational research: AMEE Guide No. 87. Med Teach 2014;36:463-74.
van der Vleuten CP, Schuwirth LW. Assessing professional competence: From methods to programmes. Med Educ 2005;39:309-17.
Gleeson F. AMEE Medical Education Guide No. 9. Assessment of clinical competence using the Objective Structured Long Examination Record (OSLER). Med Teacher 1997;1:7-14.
Ranjan P, Ranjan R, Kumar M. National Exit Test: How will one size fit all? Ann Indian Acad Neurol 2020;23:145-9. [Full text]
Rajan R, Radhakrishnan DM, Srivastava AK, Vishnu VY, Gupta A, Shariff A, et al
. Conduct of virtual neurology DM final examination during COVID-19 pandemic. Ann Indian Acad Neurol 2020;23:429-32. [Full text]
Wiles CM, Dawson K, Hughes TA, Llewelyn JG, Morris HR, Pickersgill TP, et al
. Clinical skills evaluation of trainees in a neurology department. Clin Med (Lond) 2007;7:365-9.
Davis MH, Karunathilake I. The place of the oral examination in today's assessment systems. Med Teach 2005;27:294-7.
Memon MA, Joughin GR, Memon B. Oral assessment and postgraduate medical examinations: Establishing conditions for validity, reliability and fairness. Adv Health Sci Educ 2010;15:277-89.
Jayawickramarajah PT. Oral examinations in medical education. Med Educ 1985;19:290-3.
Lamping DL. Assessment in health psychology. Can Psychol 2007;26:121-39.
[Figure 1], [Figure 2]
[Table 1], [Table 2]