Comprehensive Exam Report: Student Ratings of Instructor Performance

This comprehensive exam report investigates the influence of instructor personality traits on student ratings of instructor performance. The study employs an online survey research design, specifically a correlational approach, to gather data from university students. The independent variables include the Big Five personality traits (agreeableness, conscientiousness, extraversion, neuroticism, and openness to experience), while the dependent variable is the student's evaluation of the instructor's performance. A stratified and simple random sampling method will be used to select 200 instructors from four universities. The report reviews existing literature highlighting the use of student evaluations in teaching, the potential biases involved, and the research gap concerning the effects of instructor personality. The report also discusses the advantages of online surveys, the sampling procedure, and the variables involved. The goal is to explore the connection between teacher personality and student perceptions of their teaching effectiveness, contributing to a deeper understanding of factors influencing student evaluations. The report aims to address the limitations of current student evaluation systems and provide insights for improving instructor performance appraisal processes. The research design includes online surveys, and the sampling procedure involves a combination of stratified and simple random sampling to ensure a representative sample of instructors.

Running Head: COMPREHENSIVE EXAM 1
Comprehensive Exam
Name:
Institutional Affiliation:

COMPREHENSIVE EXAM 2
Research Methodology
Background literature
Instructors play a critical role in the teaching and learning process, particularly in the
modern education systems. They have an essential influence in the short and long-term
outcomes of their students in a variety of aspects such as grades, health, assessment scores,
extracurricular activities and personal behavior (Chetty, Friedman, & Rockoff, 2014; Kell,
2019). Various researchers have explored the impact of instructor personalities on the rating
of teachers. For instance, Clayson (2013) indicated that the first impression that the students
have of their teachers determines the final evaluations of their teachers. Similarly, Mori and
Tanabe (2012) investigated the association between the students’ perceptions of the
personality of their teachers and the evaluation of teachers’ performance among Japanese
students learning foreign languages. Just like the previous studies, the researchers found that
the teachers’ personalities significantly influence the student evaluation of their teacher in
language classes. An extension of the study (Mori & Tanabe, 2013) indicated that teacher
personality has the highest role in determining teacher evaluations on the basis of teachers’
nationality. Therefore, there is an important need to ensure that teachers work effectively to
support students in achieving their study outcomes.
Teachers differ in their performance with regard to impacting students’ outcomes. In
most of the universities across the world, student evaluation of teaching is widely used as a
strategy for improving the quality of teaching and learning in the institutions. The system of
using students’ feedback plays a critical role in providing essential information that is used
for appraisal of instructors and providing evidence that for assessing the accountability of the
institutions (Mori & Tanabe, 2015; Spoorenm, Brockx, & Mortelmans, 2013).The
disadvantages of using student evaluation in teaching is the incidence of biasness. While the
method has gained popularity across leaning institutions, there exist various possibilities of

COMPREHENSIVE EXAM 3
biases in the process. Instructors are likely to get a higher score in the student evaluations
based on their lenience with the students, and the performance of the students. For instance,
students who perform well in their studies are likely to rate their teachers highly. Teacher
personality in the use of student evaluation is another major source of bias that influences the
validity of the student evaluations. The teacher personality plays a critical role in the decision
of students regarding their teachers’ performance (Mori & Tanabe, 2015). Additionally,
instructors who the students perceive to be more accommodative and tolerant may he rated
highly (Mori & Tanabe, 2015). The types of factors that students use for rating their teachers
may be informal, and biasness may affect the outcome of the evaluation.
Research gap
The aim of the study is to explore the possible associations between instructor
personalities and student ratings of instructor performance. For many decades, the use of
student evaluation testing has raised numerous concerns on the validity of students’
evaluations of teaching for measuring the effectiveness and capabilities of their instructors.
Some of the challenges associated with the use of student evaluations for teaching include
biasness on the basis of gender and origin, validity and reliability (Boring, Ottoboni, &
Stark, 2016; Braga, Paccagnella, & Pellizzari, 2014; Hornstein, 2017; Spooren, Brockx, &
Mortelmans, 2013). However, despite the increased criticism of the assessment strategy, it
continues to gain popularity in many countries in the world (Boring et al., 2016; Hornstein,
2017; Spooren et al., 2013).
The validity of student assessment is based on the argument that as students’ attends
their classes; they are able to interact with their instructors and can easily identify their
teaching abilities and overall performance. However, the effectiveness of the students’
assessment may be questionable due to a potential disparity in the the objectives of the
administration and the intentions of the students in rating their instructors. The students may

COMPREHENSIVE EXAM 4
focus on their grades while the administration focuses on the quality of students’ learning
(Hornstein, 2017).
To ascertain the biasness in the student rating of teaching, Boring et al. (2016)
conducted nonparametric statistical tests of two datasets comprising of students and teachers
in six compulsory first-year courses at a French university and a US university with a broad
range of courses. Braga et al. (2014) explored the students’ rating of their teachers in
different courses using data from Bocconi University archives. The findings indicated that
student allocate a higher rating to classes with higher grades. The results of the research
demonstrated inefficiencies in the student rating method in terms of validity to measure
teaching quality and efforts.
Researchers have investigated the validity of student rating of their teachers indicating
the evidence of biasness, based on gender and students’ grades. Despite the bias evidences in
student evaluation of teaching, many institutions continue to use it as an important strategy
for performance appraisal, assessing the effectiveness of teaching and promotion. The use of
student evaluations to measure instructors’ performance needs to be explored further to
inform policy change and promote fairness in the instructors’ performance appraisal
processes. The aim of the study is to explore the effects of personality traits of teachers in
influencing students’ ratings of their performance.
Research Design
In this study on-line survey research design will be used. The survey research design
refers to the process of gathering information from participants based on their responses to
preset questions. Phillips (2017) explained that the survey methodology is applied to
investigate human-related phenomena such as opinions and emotions. Human phenomena
often entail data that cannot be observed or obtained from archives and secondary sources.
Paradis, O'Brien, Nimmon, Bandiera, and Martimianakis (2016) mentioned that surveys are

COMPREHENSIVE EXAM 5
used in the documentation of attitudes, perceptions, beliefs and knowledge of a population
using a clear sample of individuals. The survey approach is appropriate for this study as it
entails collection of the data from university students to explore how they rate their
instructors; performance based on their personalities.
The correlational survey research design is suitable in this study. Correlational
research design is used to explore the relationship between one or more variables within the
same population or between different variables in two populations. Correlational research
design is critical in informing casual references and establishing a basis for evidence-based
practice (Curtis, Comiskey, & Dempsey, 2016). The correlational survey design is
appropriate for exploring the relationship between personality traits of teachers and students’
ratings of their performance.
Online interviews eliminate the biases arising from interviewer effects in the research
process. The interviewer effects refer to the presence of the interviewer while the respondents
are filling the questionnaires. The presence of interviewer may cause the respondents to
deliver the socially desired answers; hence threatening the effectiveness of the in-person
surveys. Additionally, the interviewer may unintentionally give cues regarding his or her
expectations of the responses through pronunciation or time allocation for different questions
(Wiersma, 2013). Other advantages of using internet-based survey research approach include
easier access to populations, allowing generalization of the research findings, access to
diverse participants in terms of their gender and age, and accessibility of larger sample sizes.
Online surveys also save time that could have otherwise been used for travelling and
physically administering the questionnaires. Online approaches are also cost effective as only
a few resources are used to reach hundreds of participants (Nayak & Narayan, 2019; Rice,
Winter, Doherty, & Milner, 2017).
Sampling Procedure

COMPREHENSIVE EXAM 6
A sample refers to a finite subset containing participants drawn from the given set of
individuals with the characteristics of interest to the researcher. The entire subset is called the
target population. The sample frame refers to group of individuals drawn from the target
population after the application of a sampling process. Sampling refers to the process of
selecting individuals from the sampling frame. There are various methods that can be applied
in the process of sampling based on the needs of the researcher (Martínez-Mesa et al., 2016).
Generally, the researchers use probability and non-probability sampling approach in
both quantitative and qualitative research method. In non-probability sampling, the chance of
selecting some individuals to participate in the study is null. In probability sampling, all
individuals within the target population have an equal probability of selection to participate in
the study (Martínez-Mesa et al., 2016). Probability sampling refers to a method where the
likelihood of selecting each unit is the same (Sharma, 2017). The probability sampling is
suitable for quantitative research studies to eliminate bias, promote generalization, and
enhance replicability of the study.
The probability sampling methods entails simple random sampling, multistage
sampling systematic random sampling, stratifies sampling and cluster sampling (Martínez-
Mesa et al., 2016). In this study, a combination of stratified sampling and simple random
sampling are suitable for ensuring the selection of 200 instructors from a population of 1000
instructors in 4 universities. The stratified random sampling is applicable in the first phase of
the sampling process which entails grouping of the participants. Martínez-Mesa et al. (2016)
referred the groups to which the target population is first assigned as strata. Sharma (2017)
explained that the each stratum should contain members with similar characteristics or
attributes. In the first stage, all instructors can be grouped based on the university where they
teach, to produce four groups or strata. The simple random sampling method is then used to
select participants from the strata. The second stage entails selection of participants from

COMPREHENSIVE EXAM 7
each strata using simple random sampling. Martínez-Mesa et al. (2016) defined simple
random sampling as the process of randomly selecting participants from a full list of sample
units using a table of random numbers or computerized selection. Sharma (2017) explained
that the selection of a sample from each stratum should be done in a way that ensures
proportionality of the size of the stratum in relation to the the population. The method is
appropriate for obtaining a random sample from the strata.
There are various advantages of using stratified sampling. For instance, given the
large population of 1000 instructors, the sampling method helps to eliminate human bias in
the selection of the sample. The stratified random sample consists of participants that are a
representative of the study population. The stratified random sampling uses random selection
of participants, meaning that the study outcomes can be generalized; hence ensuring external
validity (Sharma, 2017) the survey research will include 200 instructors from four
universities.
Independent Variables and Dependent Variables
Independent variables
The independent variables in the study will include the big five personality traits,
which will be used for students’ evaluations of their instructors. The model was developed
between the 1980s and 1990s providing a solution to a long struggle, where researchers
lacked a personality testing framework. Five prominent adjectives were identifies as recurring
in most of the personality literature, and were regarded as the five major dimensions of
personality. The traits include agreeableness, conscientiousness, extraversion, neuroticism
and openness to experience (Kell, 2019).
Agreeableness an agreeable person demonstrates attributes such as kindness, warmth
and cooperation. Agreeable persons have behaviors tending towards use of negotiation in
conflict resolution, and helping others. They are able to deal with frustrations that may occur

COMPREHENSIVE EXAM 8
in their interactions with others, show empathy to distressed persons, and strive to maintain
harmonious relationships with others.
Conscientiousness refers to the attributes of being responsible, hardworking and
thorough in accomplishing various tasks. Conscientious people demonstrate outcomes such
as improved performance in their work, longevity, and marital stability, accomplishment of
expectations and delay of gratification with the aim of achieving a long-term goal (Kell,
2019).
Extraversion is an adjective used to refer to talkative, energetic and bold people,
Extraverts are often emotionally positive, comfortable with intimacy and interdependence,
able to create social environment that are positive and friendly, and secure and focus on
developing positive stimuli (Kell, 2019).
Neuroticism refers to individuals with attributes of tension, nervousness, and
discontentment. Such people often tend towards depressive behaviors, alcohol abuse and
heart diseases. They do not demonstrate self-efficacy and satisfaction with relation. They act
impulsively when angry, feel insecure and self-conscious and are prone to emotional
frustrations (Kell, 2019).
Openness to experience is a phrase that can be used to describe people with
attributes of creativity, curiosity and imagination. They often portray behavior such ad
divergent thinking, appreciation of art, and liberal political views. People who have a high
openness to experience tend to appreciate originality and novelty, easily identify the emotions
of others, and attract people with similar attributes as theirs (Kell, 2019).
The big five personality traits have been used widely in the field of education; hence
it a suitable framework for guiding this research. Kırkağaç, and Öz (2017) cited that the Big
Five Personality Traits Model stands out in educational research since personality is a major
factor in educational performance of both the students and stakeholders. Further, the five

COMPREHENSIVE EXAM 9
traits contained in the model are dynamic and stable at the same time and clearly indicate
personal needs. The terms used in the model are also universal and can be used in a variety of
contexts. Researchers have specifically used the the big five personality traits model to
explore the influence instructor’s personality traits on student evaluation for teachers. For
instance, Patrick (2011) explored the role of personality trait based on the students’
perceptions in predicting the students evaluations of their teachers, the personalities were
based on the Big five inventory, which entails five personality dimensions including
conscientiousness, agreeableness, extroversion, stability and creativity or openness. The
findings of the study by Patrick (2011) indicated that that openness, extroversion,
agreeableness and conscientiousness are positively correlated with student evaluation of the
courses and teachers while stability or neuroticism had a negative correlation. Therefore, the
Big Five Personality Traits Model is suitable for exploring the association between
personality traits of the instructors and students’ ratings of their instructors’ performance.
Research Instrument
The Ten-Item Personality Inventory (TIPI) developed by Gosling, Rentfrow, and
Swann (2013 is suitable for this study. Hanif (2018) confirmed that since its development, the
instrument has significantly gained popularity among researchers in various fields globally.
The TIPI is a short instrument that is used in measuring the Big-Five dimensions, which
include Extraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness.
The instrument comprises of ten items, which are pairs of adjectives describing various traits.
The respondents of the ten items questionnaire rant the items in a scale comprising of 7
points, with 1 representing ‘strongly disagree’ and 7 indicating ‘strongly agree’. The scale is
preferable for students as it is rather short and completing the questionnaire does not take
more than five minutes (Bąk et al., 2014; Hanif, 2018). The TIPI questionnaire is compatible
with the five factor model as Gosling et al. (2003) developed the instrument with the model

COMPREHENSIVE EXAM 10
in mind. The aim of Gosling et al. (2003) was to develop a simple research instrument and
ensure optimization of content validity. Therefore, the three aspects that the TIPI successfully
achieved include reaching a detailed coverage of the Big Five dimensions using only a few
items, identifying items that could be used to represent both roles of each dimension,
whenever possible, the used items were not evaluative extreme, and avoided the use of
negations in the items. Also, the TIPI reduces incidences of redundancy in its descriptors
(Hanif, 2018). The characteristics of TIPI make it easy for students to use in assessing the
personality traits teachers and instructors. The TIPI will be extended to accommodate
assessment the performance of teachers based on the reports of the students. The structured
questionnaires will help in assessing the association between the personality traits the the
perceived performance of the instructor. Figure 1 demonstrates the TIPI instrument. The
respondent is expected to write a number alongside the traits depending on the extent to
which he or she agrees to the statement.

COMPREHENSIVE EXAM 11
Figure 1: TIPI instrument.
According to Brito-Costa, Moisão, De Almeida, and Castro (2015), the TIPI is a valid
tool for assessing personality traits in small samples with limited time, due to its low internal
consistency. Gosling, Rentfrow, and Swann (2003) also tested the validity of TIPI and found
that the instrument has adequate levels of convergence with the big five measures in the
observer, self and big-five measures; a high test-retest reliability, and prominent patterns of
external correlates. It also has a significant convergence with the ratings of self and the
observer. Storme, Tavani, and Myszkowski (2016) noted that the TIPI is sometimes regarded
as a challenge to psychometrics as it is sometimes regarded as having low reliability and
validity due to poor Cronbach’s α and confirmatory factor analysis (CFA) indices. However,
TIPI is viewed as a reliable and valid tool for short measures such as those involving the Big-

COMPREHENSIVE EXAM 12
five personality traits due to its stability over time and satisfactory temporal stability and
meaningful divergent or convergent validity.
Dependent Variable
The dependent variable in the study is student ratings of instructor performance. The
student rating for teachers has for long been used to assess the performance of teachers. The
student evaluation for teachers is an appropriate measurement for the dependent variable.
Research instrument
The 7 point Likert scale is appropriate for measuring the ratings that students assign to
their instructor’s performances. The Likert scale is an important tool for use in statistical
research. Joshi, Kale, Chandel, and Pal (2015) cited that a Likert scale refers to a set of items
or statements that are used to assess a hypothetical phenomenon under investigation. The
statements trigger their level of agreement with the items using a metric scale with the aim of
understanding the attitudes and opinions of the participants towards the issue (Joshi et al.,
2015).
In the assessment, students are expected to rate their instructors on a Likert scale to
provide answers to predetermined statements that are mean to trigger the assessment of
behavior and effectiveness of the instructors. The categories of choices that the students may
choose from include “Extremely poor,” “very poor,” “poor,” “satisfactory,” “good,” “very
good,” and “excellent”. Numbers (1 to 7) are then attached to each category to ensure easier
statistical analysis pf the outcomes.
Joshi et al. (2015) confirmed that the 7 point Likert scale is more effective compared
to the 5 point scale, as it ensures reliability of the participants’ responses. The 7 point scale
comprises of more varieties of views indicating that the participants have a better chance of
describing exactly what they feel and think about the issue. The 7 point scale is more exact
and increased the likelihood of understanding the reality about the person or thing being