Medical Assessment: Knowledge Components, Validity, and ENT Standard

This assignment solution provides a comprehensive overview of medical assessment, focusing on knowledge components, different types of validity, and standard-setting methods, particularly in the context of an Ear, Nose, and Throat (ENT) surgery clerkship. The solution begins by dissecting knowledge-based questions, illustrating how they assess recall, procedural understanding, conceptual knowledge, and the application of principles. It then delves into the evaluation of an MCQ question, examining construct, face, content, concurrent, and predictive validity, with detailed explanations and examples. Furthermore, the assignment explores standard-setting methods like the modified Angoff technique, emphasizing its application in ENT surgery course assessments to ensure that passing students demonstrate mastery of core knowledge and competencies. The document highlights the importance of aligning assessment with learning objectives and using evidence-based approaches to determine fair and reliable pass marks.

MEDICAL ASSESSMENT
Student name
Professor’s name
Affiliation
Date

PART A:
SECTION 1: THE KNOWLEDGE COMPONENTS QUESTION
Testing Question example Ideal answer
Knowledge What is the normal range of blood
pressure in an adult?
Systolic blood pressure of between
90 to 120 mmHg and diastolic blood
pressure of between 60 to 90 mmHg
Procedure What is the procedure of measuring blood
pressure in an adult? Get the equipment ready and have
the patient seated in a relaxed
manner in a quiet room.
Introduce yourself to the patient and
gain consent to take his blood
pressure.
Wrap a properly sized cuff of the
sphygmomanometer around the
upper arm with the lower edge of the
cuff 5 cm from the cubital fossa.
Palpate for the brachial artery pulse
then lightly press the bell of the
stethoscope on the brachial artery at
the lower edge of the cuff.
Inflate the cuff rapidly to around 200
mmHg until the brachial artery pulse
is no longer heard using the
stethoscope.
Slowly release air from the cuff at a
rate of around 3 mm per second.
Listen keenly with the stethoscope as
you observe the sphygmomanometer.
The first Korotkoff sound
corresponds to the subject's
systolicblood pressure. The diastolic
pressure corresponds to the
disappearance of the last Korotkoff
sound.
Document the pressure in both arms
and calculate the difference; repeat

the procedure with the patient in
supine position and standing
position.
Measure the blood pressure for two
additional times if the subject’s blood
pressure is elevated, waiting for
around ten minutes between
measurements.
Concept What are the dangers of chronic high
blood pressure?
Dangers of high blood pressure
include cerebrovascular accidents
due to rapture of cerebral
microvasculature, accelerated
artherosclerosis and arteriosclerosis,
renal failure, heart failure,
hypertensive retinopathy
Principle
Explain why blood pressure measurement
in different positions yield different
results.
Blood pressure discrepancies with
varying positions are caused by
inputs from compensatory
mechanisms from the autonomic
nervous system and baroreceptors to
maintain the blood pressure within
the normal range.
The knowledge question simply requires the student to state the normal blood pressure
range, which is a universal fact that can be obtained from published literature. The student is not
required to think but to only remember the information and answer the question.
The second question on the procedure requires the student to outline the steps that are to
be followed in order to take blood pressure of a person. The student in this case is expected to be
systematic and chronological in outlining the steps as well as keep in mind the rationale
informing every step in the procedure.
The third question on concept requires the student to have read and create links in
knowledge on the pathophysiology of complications of uncontrolled hypertension. Hypertension
plays a major role in the pathogenesis of vascular disease such as arteriosclerosis due to
hyalinization of the arterial and capillary walls and acceleration of formation of atheroma in

atherosclerosis due increased endothelial injury that favors the fibrogenic processes. The
complications of hypertension mainly involve end organ damage such as nephropathy,
retinopathy, heart failure and stroke. Hypertensive nephropathy is secondary to hyalinization of
capillaries in the glomerulus, inhibiting the process of ultrafiltration. Hypertensive retinopathy is
secondary to rapture of capillaries in the retina leading to haemorrhage in the retina and eventual
loss of vision. Hypertensive heart disease is caused by elevated systemic blood pressure that
makes the heart to pump harder to counter the systemic pressure. This eventually leads to
increased metabolic demands of the myocardium giving rise to ischemic heart disease and
remodeling of the left ventricle that decreases the stroke volume leading to the signs and
symptoms of heart failure. long standing hypertension can also cause haemorrhagic stroke due to
rapture of vessels in the brain such as the middle cerebral artery. This leads to haemorrhage into
the brain and eventual infarction of the areas of the brain that are supplied by the affected artery
and pressure effect caused by the haematoma. This will manifest clinically with abnormal
findings on neurological examination.
The fourth question on principle requires the student to understand the physiologic basis
of variation in blood pressure with position and the mechanisms of short term regulation of blood
pressure. In this question, the student has to remember that changing positions varies regional
flow of blood which in turn affect inputs from the autonomic nervous system and the
baroreceptor mechanisms to maintain the blood pressure within the normal ranges.
SECTION 2
Consider the following MCQ:
The following statement is false as regards head injury patients
a. Patients with acute subdural hematoma present with loss of consciousness
b. Nasogastric tube insertion is contraindicated in patients with suspected fracture of base of
the skull
c. Skull fracture is a predisposition for epidural hematoma

d. Radiograph of the skull is the investigation of choice in a patient with reduced level of
consciousness
e. The normal range of intracranial pressure is between 10-30 mm Hg
The correct answer is d
Type Describe each type of validity
Describe how you would determine
each type of validity using your MCQ
as an example
Construct Construct validity refers to the degree
to which an assessment covers a
particular trait by demonstrating the
consistency of test score
interpretations with nomological
network entailing observational and
theoretical terms (Cronbach & Meehl,
1955).
This MCQ overs the basic concepts that a
medical student should understand as
regards the physiology, pathophysiology,
investigations and management of
patients who present with head injury.
Face Face validity answers the question of
whether a test appear to assess what it
claims to. The purpose of the test
should be clear even to naïve
responders.
This MCQ has a high face validity
because it aims to test for the student
understanding of principles of
management of head injury patients. For
instance, the patient should know that
insertion of a nasogastric tube is an
absolute contraindication in patients with
suspected base of skull fracture because
the tube can divert to the brain instead of
the stomach and lead to very serious
complications when it is used to
administer fluids and for aspiration.
Furthermore, the investigation of choice
for a patient with a lucid interval is a CT
scan or MRI and not a plain radiograph of
the skull. The MCQ is testing the
student’s understanding of head injury
and not any other topic.
Content This refers to the degree to which an
assessment measures the scope of the
subject. The questions asked should be
properly chosen for the level of
students as well as the content under
examination.
This MCQ has content validity because it
tests the minimum basic knowledge that
an undergraduate student is expected to
have regarding head injury. The options
are within the confines of the medical
student knowledge and not that of a
neurosurgeon.
Concurrent This refers to the extent of This MCQ has concurrent validity

relationship between a test and an
established standard that is existing
concurrently.
because it tests on the standard published
principles of management of patients with
head injury. The correct options are in
agreement with existing protocols that
have been published to guide
management of head injury patients and
prevent unnecessary costs and life
threatening complications.
Predictive This refers to the level to which an
assessment accurately presages future
occurrence of a criterion.
Students who score this question
correctly are likely to perform better in
Emergency department and intensive care
because they understand the principles of
recognition and management of head
injury patients.
Construct validity posits a test to cover as much of the essentials of the subject matter
extensively in order to give a clear picture of the students understanding of the content of the
subject under examination. From the example of the above MCQ, the question seeks to test
different aspects of diagnosis and management of head injury patients.
Regarding face validity, the question tests basic principles of head injury management
that are contained in the undergraduate curriculum for medical students hence it will be fair for
their level of knowledge.
The content of the subject being covered in the question is within the scope of what the
medical student is taught and is required to demonstrate mastery hence the question has content
validity.
As regards concurrent validity, the correct options in the MCQ are in congruence with the
latest publications on diagnosis and management of head injury patients.
Students who are able to score this MCQ correctly are more likely to do well in
traumatology, emergency medicine and intensive care courses as well as professional practice

because they demonstrate mastery of principles of evaluation and management of head injury
patients.
PART B: OPTION 2
Ear, nose and throat (ENT) surgery clerkship in the bachelor of medicine and bachelor of
surgery (MBChB) course is aimed to impart to the medical students the requisite knowledge and
competence in skills that are essential in diagnosing, managing and preventing
otorhinolaryngological diseases. The medical student undertaking this course is expected to
demonstrate knowledge as well as competence through making definitive diagnosis, analysis of
clinical data and develop an elaborate management plan entailing either surgical or non-surgical
options, or both. Assessment for this course involves both written MCQs and OSCE examination
at the end of the clinical rotation. The scope of the examination cuts across otology, rhinology,
laryngology to upper aero digestive tract topics as well as different equipment used in the ENT
specialty.
Standard setting in an examination context refers to the process of deciding the pass mark
or cutoff points that determines people who pass and those who fail the test. For a test to be
standard, it should not be so hard that students who could have passed end up failing or so easy
that those who were to fail end up passing. The aim of standard setting process is to ensure that
students who pass the test have demonstrated mastery of the core knowledge of the course as
well as the competencies that are being assessed in the examination. Standard setting methods
that can be employed in assessment of ENT surgery course include modified Angoff, Borderline,
Hofstee and Ebel.
The modified Angoff technique of standard setting was developed by Agoff in 1971 to
facilitate decision making in multi-component tests such as MCQ examinations that form an

essential component of assessment of medical students. In this technique, the examiners are
additionally served with the exact test scores of previous tests to enable them to determine the
chances of a borderline students answering a particular question accurately. This comparison
with previous results of the same test enables the examiners to become more realistic and
reconsider the pass mark for the test making it evidence based. This also makes the test more
trainee-centered than examiner-centered one hence increasing its face validity.
For the MCQ examination of ENT surgery course, the examiners should imagine and
discuss the abilities of a borderline candidate and reach a consensus. Each examiner should then
consider each MCQ item, one after the other then score the borderline candidate between 0 to 1
based on the probability of borderline candidates getting the right answer. The probabilities of
each examiner are totaled then divided by the number of MCQs to determine the mean
probability per examiner. The means of all examiners are summed up then divided by the total
number of examiners to give the provisional mean pass mark for the whole test. Examiners are
then provided with the actual results from previous test to reconsider, where necessary, their
original probability estimates in the light of previous results provided and recalculate the pass
mark for the entire MCQ examination.
According to Mogapi (2012), the Angoff technique has three primary elements:
formulation of the borderline candidates of a test, identification of specific assessment questions,
and the employment of experts to determine whether a borderline candidate will appropriately
perform each of the assessment components. The merits of modified Angoff technique are many.
First and foremost, modified Angoff technique has a high face validity since it focuses on the
difficulty of the content of the examination and the level of competence that is required of the
examinee to pass. Secondly, the technique is recyclable in that if a question is repeated in a

different examination with the same context, the previously predicted difficulty can be re-used
saving on time required to make fresh judgements. Thirdly, the modified Angoff technique has
been highly studied and there are many publications on the same making it more justifiable for
employment in high stakes examinations, like ENT surgery that involves the lives of patients,
hence holds up when questioned in court. The disadvantages of this technique include
inconsistency in defining the borderline candidate by the examiners. This requires the examiners
to be experts in their specialty in order to distinguish the borderline from the average candidate.
Secondly, the technique is labour intensive and time consuming since the judges have to look at
every assessment item. This can encourage the examiners to rush through the process when they
become fatigued and impatient. Thirdly, sample bias can negatively impact the accuracy and
reliability of the assessment set using this modified Angoff technique. A large sample of
examiners of different ages, levels of seniority, genders and ethnicities are required for the
success of this technique making it hard to get examiners who meet these specifications. Finally,
this technique relies on back-up from a criterion-referenced technique such as borderline
regression since it does not use real examination data to estimate the pass mark.
The borderline regression technique is an examinee-centred method of standard setting
that is applied mainly in OSCE. In this technique, the candidate is awarded a “global score” for
each station in a circuit by the examiner at the end of the assessment after actual marks have
been awarded based on the assessor’s judgment on the overall performance and ability of the
student in that particular station. The scale for the global score ranges from three to five options,
where the examiner choses the most appropriate score for the candidate for example, pass,
borderline pass, borderline fail, and fail. Advantages of using borderline regression technique
include reliability, as the technique uses real test data to calculate the pass mark for a station

using a single test, or many tests if available. The technique can also be assessing the
effectiveness of an OSCE station by identifying problems encountered by the examinees in that
particular station such as unbalanced marking scheme that can be pointed out by poor
discrimination in the borderline regression graph. Borderline regression technique is easy to use
by the examiners because it is not time consuming to award a global score for each candidate
when proper training has been conducted. Finally, the technique is fundamental in backing up
other standard setting techniques such as Angoff method. Disadvantages of using borderline
regression technique include difficulty in understanding the definition of borderline candidates
hence necessitating the need to conduct intensive examiner training that is costly and time
consuming. Secondly, the technique encourages box ticking of the overall candidate’s
competence in a station at the expense of counting marks scored. Finally, the technique requires
large number of candidates and examiners for reliability making it unsuitable for examinations
with low enrollment.
Hofstee method is a test-centred method of standard setting whereby the examiners
estimate the lowest acceptable failure rate, the highest acceptable failure rate, the minimum pass
mark, even if all candidates fail, and the maximum pass mark in case all examinees passed. The
average of the four estimates is calculated by the examiner then superimposed on a graph of
actual test scores. A rectangle is created by the examiner estimates, which the actual examinee
test score curve bisects. In the event the rectangle falls outside the curve, another standard-setting
method has to be resorted.
In Ebel (1972), examiners categorise examination items into three main categories:
“essential”, “important” and “indicated” (Case & Swanson, 1998). The proportion of borderline
examinees who are able to correctly answer question in each category is estimated by the

examiners. The average of estimated proportion in each category is multiplied by the
corresponding number of questions to arrive at the cutoff for the specific category. The pass
mark for the whole test is obtained by summing the cutoffs for each category. The disadvantage
of this technique is that the examinees can pass highly in one category then fail massively in a
category with essential questions implying that the student will pass the test despite having no
grasp of the basics of the course.
I believe the modified-Angoff technique is the best method to standard the assessment for
ENT surgery course because it gives input from both the tutor and backs up the estimates of the
borderline candidates with evidence from previous results tests. This improves the validity of the
test because each question is analysed thoroughly for suitability for the level of education of the
students. Again, it ensures that students who are competent pass the exam whereas those who are
incompetent fail the exam. The technique also boasts of reduced level of bias amongst examiners
which assists in determining a clear fail.
The major barrier to implementing modified-Angoff technique is that it is based on the
examiner who is an expert in the field being tested in comparison to the student who might be
having limited knowledge gained from the few weeks of rotation in the ENT surgery field.
Furthermore, the previous results used in moderating the pass mark are from a previous cohort
that might have been taught well than the current candidates being examined. To mitigate these
hitches in implementing the modified-Angoff technique, the examiners should also factor in the
scope of knowledge that has been covered by the current cohort of candidates to standard the
examination. To solve the problem of knowledge gap between the expert and the students, the
curriculum of the ENT surgery course should be used in selecting questions for the examination.

PART C: SELF ASSESSMENT
Self-assessment refers to the process of evaluating your own progress and comparing the
results obtained with what is expected of you. According to Adams et al (2016), self-assessment
first entails self-grading then followed by understanding what is considered as excellent work in
a learning exercise.
A self-assessment tool is composed of items that seek to analyse of one’s strengths and
weaknesses, personality, preferences as well as biases that enables the person to become aware
of their innermost desires, volition and belief systems.
To put this process of self-assessment into perspective, appendix 1.0 shows a sample of a
checklist that is meant to be used to enable fourth year medical students to do a self-assessment
of how they conduct themselves during their clerkship in the medical rotation. The merits of this
questionnaire include: the students are able to identify and investigate their needs, priorities,
perspectives, priorities and preferences which are key to their learning in the clinical area; the
students are able to easily analyse how they spend time on different activities during their
rotation; it can be used to address sensitive topics that the student might be afraid to express to
the lecturers; the students have adequate time to think and respond to the items in the checklist at
their own desired pace and time and the students are also able to identify their strengths,
opportunities, weaknesses and threats to their learning in the clinical areas. The student will
assess their performance in different aspects influencing their learning and award a single mark
for every option. The scores will be summed up at the end and calculated as a percentage to give
the overall self-assessment score for the candidate. A score below 50% will imply that the
student has to work on their strengths and weaknesses immediately to salvage their performance
in the clinical areas. The problems associated with use of the checklist include inability to rate