Learning and Development Evaluation: Definition, Types, and Importance

This presentation provides an introduction to evaluation in a Learning and Development Environment. It covers the definition, types, and purpose of evaluation, evaluation in the L&D environment, the importance of knowledge transfer, evaluation theories, methods of analysis of data, challenges to L&D evaluation, and the role of L&D evaluation in the knowledge economy.

Learning and Development
Evaluation

Introduction
Learning and development is apart of every organization. Organizations are required to train their employees using a myriad
of techniques and training programmes. However, a crucial part of learning and development is evaluation, an aspect that is
often forgotten. This presentation in an introduction to evaluation in a Learning and development Environment. (CIPD, 2015)
“‘...half of learning and development content is developed from scratch by internal/external L&D practitioners.’” (CIPD, 2015)

1.1Evaluation: Definition, Types and Purpose
This section will acquaint user with the definition of evaluation, some basic types and the
purpose of conducting evaluation

Definition
"Evaluation is a process that critically examines a program. It involves collecting and analyzing information about a program's
activities, characteristics, and outcomes. Its purpose is to make judgments about a program, to improve its effectiveness, and/or to
inform programming decisions development of an internal structure or heirachial ranking of jobs". (Patton, 1987).

Types of Evaluation
There are various types of evaluation, Most Notably: (The Regenets of the University of Minnesota, 2018)
Formative Evaluation
Summative Evaluation

Types of Evaluation (Contd..)
Some other types of evaluation are:
Process Evaluation

Types of Evaluation (Contd..)
Outcome Evaluation

Types of Evaluation (contd.)
Impact Evaluation

Purposes of Evaluation
a) Diagnostics
b) Measurement
c) Experimentation/ Exploration

1.2 Evaluation in the L & D Environment
This section is a very brief introduction to some theories that postulate some ideas
regarding what consists of a good evaluation and how evaluation in a learning and
development environment must be approached.

Learning and Development Evaluation in a Modern
Firm
The Case Study of Zocdoc:
Zocdoc is a digital healthcare Company that allows patients to book doctor appointments online. In this process, the Company is required to
store large amounts of data regarding doctors, patients, insurance networks etc. The company reached a critical mass of patient and doctor
operations where it's data storage architecture capacity began to fall short and the Company had to transfer their business data from local
server architecture to cloud using a third party cloud management system. The firm had to teach all of it's development employees new
software development techniques to migrate to the Cloud. The firm also decided to train its non-programmer employees in SQL. In 12
months, the Company taught various teams to use newer software architecture techniques as well as SQL. Using SQL allowed Zocdoc to save
costs of server, improve the efficiency of database and also allowed better "deep learning" which would allow the Company to match patients
with doctors more efficiently by creating more efficient algorithms and data models.(Amazon Web Services, 2017), (Smith, 2017)

Evaluation in L&D Environment
● Learning and Development process can be extremely personal. Hence, no
learning and development programme can be one size fits all. Evaluation
helps make the process fit better and contextual to the organization
● It is likely that employee welfare would be disregarded in the process of
conducting an L&D programme, Learning and Development Evaluation may
help highlight employee needs and help solve the problems of employees.
(Allamano, 2018)

1. 3 The Importance of Knowledge Transfer and the
Role of L&D Evaluation
This section is an introduction to the role of L&D evaluation in the context of
transfer of knowledge and the importance of transfer of knowledge in India.

Transfer of Knowledge
As pointed out in the case study of Zocdoc, the firm had to build new tools, in
order to cope up with the changing environment. As the firm grows, all new
employees shall have to engage in behavioural changes and learn the procedures
and processes of work at the firm. For example, while a programmer may already
have knowledge of SQL, the usage of SQL in context of Zocdoc would be unique.
Thus, almost every new employee must be trained. Hence, a certain amount of
knowledge and skill transfer would be required for every employee.
Additionally, the process also allows Zocdoc to build a knowledge base or it’s own
intellectual capital.

Evaluation and Return on Investment (ROI) in
Learning Programmes
● Learning and Development programmes must be tailored to maximise the
retention of the knowledge and skills transferred through the programme. This
will help maximise the Returns on Investment made in learning and
development.
● Learning and Development evaluation can be used to tailor the programmes
in such as way that retention in maximised.
● Additionally, L&D evaluation may help point out the errors within the L&D
programmes and reduce these errors in the future programmes.

2.1 L&D Evaluation Theories
This section provides a brief introduction to the various Learning Evaluation and
Development theories. There is a brief explanation regarding the aspect of
Learning and Development Evaluation that is believed to be important by each
theory.
This section also provides a brief comparison of these theories.

Theory Description Advantages Disadvantages
Utilization
Focussed
The focus is to identify the primary
purpose of evaluation and the primary
intended users. All steps in evaluation
must follow from there serving to fulfil
the primary intention or “utility” of the
evaluation.
It does not offer any specific content or
method
The end users and end
goal are pre-defined
instead of working with
abstract goal and
undefined users. Hence,
the odds of missing the
context are very low.
It may be too focussed
and the resultant
evaluation may not be as
broad based as intended.
Values
engaged
The values are pre-defined and
evaluation is conducted designed
around these values. The values
prescribed must include stakeholder
values.
This is a very responsive
and democratic approach
to evaluation since user’s
values are considered in
the design of the
evaluation .
It may not be possible t
gain quantitative and
tangible metric based
evaluations. The answers
may be more descriptive
in nature.
Empowerm
ent
evaluation
Participants are provided with tools for
self assedsment and evaluation by a
trained guide.
It is expected that the
evaluation will more
credible and will have a
transgormative effect
Participant may not
necessarily have the
inclination for the
evaluation

2.2 Evaluation Process: Sources of Data
In order to conduct an evaluation, data can be sourced from various documents and notes that were maintained during the
course of the programme or all round the year, in various other evaluations. This section introduces some of such sources
of data, among many and provides a very brief overview of how such sources could be utilized. In addition, this section
also introduces the KirkPatrick model.

Basic Steps of L&D Evaluation
L&D evaluation follows all the basic steps of any other programme evaluation
since L&D evaluations are similar to all other programmes. However, L&D
evaluation, could also, be specific to a context (as given in the KirkPatrick Model).
The basic steps of evaluation are as described below while the Kirk Patrick model
has been described ahead.
a) Definition of stakeholders
b) Description of the program
c) Focus on the design of the evaluation
d) Evidence gathering
e) Extracting conclusions
f) Present findings for further use (Northwest Center for Public Health
Practice, 2018)

KirkPatrick Model: Key Steps within the Evaluation
Process
Transfer
Results

KirkPatrick Model: Key Steps within the Evaluation
Process
The Kirk Patrick Model is the best known model or theory for Learning and development evaluation.
According to KirkPatrick Model, there are four levels to Learning evaluation:
Reaction
Learning

2.3 Methods of Analysis of Data
Evaluation data can be analysed in a number of ways. This section is an
introduction to some methods in which analysis of evaluation data can be
conducted to disseminate the key findings. While a number of approaches to
analysis can be used, only the broad approached of quantitative and qualitative
analysis are introduced here.

Qualitative Methods
Qualitative Methods of evaluation , generally, tend to utilize data to form non-
parametric, directional conclusions. (Patton, 2013)
For example, a case study may be used to understand whether the overall
process is working or not . In such a case, the result of a qualitative analysis will
yield suggestion such as
“The trainer must be better prepared”
“The trainee must be given advance notice of the programme”

Quantitative Methods
Quantitative Methods help understand things from the perspective of benchmarks
and usually used to set standards and benchmarks. (PAtton, 2013)
For example, some outcomes of a qualitative analysis
“ The training programme must have at least 3 tasks”
“New trainees should be given 30 minutes to execute one SQL query”
“ Trainee 8 ranked 3rd in the process”

3.1 Challenges to Learning and Development
Evaluation
Learning and development Evaluation is a not a key priority in the modern
workplace currently, as organizations are more concerned with directly
productive tasks such a sales, operations etc. However, this may change
as constant innovation becomes more important.
This sections introduces us to some key challenges to L&D evaluation.

Challenges
Costs of Evaluation
Diversion of capacity to directly productive work tasks
The difficulty in evaluation
(CIPD, 2015)

3.2 The Knowledge Economy
This section is an introduction to how l&D evaluation fits in a knowledge economy .

Characteristics of a Knowledge Economy:
Importance of Learning and Development
Mechanical and Unskilled Jobs are becoming Sparse
Rise of Creative jobs and the need for innovation
The rising importance of innovation and intellectual capital
(OECD, 2015)

3.3 L & D Evaluation and Intellectual Capital
Intellectual Capital refers to the knowledge base acquired by an organization that
allows the organization to indirectly allows the organization to create more
revenue.
Intellectual Capital may refer to a broad array of items such as Intellectual
Property, Creative Assets (logos, brand design) etc. (OECD, 1996) Intellectual
capital is built upon the specialist knowledge developed within the organization

L &D Evaluation and Accounting for Intellectual
Capital
In the case study mentioned in 2,1, Zocdoc could have chosen to hire experts in the skills that it desired. However, learning and
development allowed the Company to train it's current employees, who were already familiar with the system, to grow further with the
Company. This created Intellectual Capital for Zocdoc. In addition, the employees gained from this by learning new skills which would
help them develop transferable skills.
- The growth of Intellectual Capital must be measured quantitatively as well as qualitatively.
- As Intellectual Capital becomes important, evaluation of learning and development gains a greater role
- Challenges include difficulty in measurement of creativity, designing evaluation in a complex environment.
- In a fast paced environment such as the one mentioned in the case study, L &D evaluation may not be a priority

3.4 Conclusion
Evaluation of Learning and development is an important aspect of training programmes
Learning and Development Evaluation helps foster an environment of innovation and creativity
In the context of a knowledge economy, more and more organizations will need to adopt learning and development as a
regular practice.

References
Amazon Web Services (2017, December 22). Zocdoc: From Monolith, to Less Than 100 ms with Amazon ECS and Kinesis. Retrieved from
://www.youtube.com/watch?v=nHU3AU3Ez1Y&t=190s
Allamano, Cara Brennan. “New Data: How Learning Impacts Engagement and Productivity.” HR Dive, 6 Nov. 2018,
www.hrdive.com/news/new-data-how-learning-impacts-engagement-and-productivity/541080/
CIPD. (2015, May). Annual Survey Report: Learning and Development 2015. Retrieved October, 10, from cipd.co.uk/learninganddevelopmentsurvey
Patton, Michael Quinn. How to Use Qualitative Methods in Evaluation. Sage, 1987.Patton, Michael Quinn. How to Use Qualitative Methods in
Evaluation. Sage, 1987.
Qualitative research and evaluation methods Michael Patton - SAGE - 2013
Empowerment Evaluation - David Fetterman, Abraham Wandersman. American Journal of Evaluation - 06 / 2007
Northwest Center for Public Health Practice,, University of Washington. “Six Steps of Program Evaluation.” Northwest Center for Public Health
Practice, Northwest Center for Public Health Practicewww.nwcphp.org/evaluation/tools-resources/program-evaluation-tips.
OECD. (1996). Science and technology policy. Retrieved October 12, 2016, from http://www.oecd.org/sti/sci-tech/1913021.pdf
Regenes of University of Minnesota, 2018. “Different Types of Evaluation.” Different Types of Evaluation | CYFAR, cyfar.org/different-types-
evaluation.“Different Types of Evaluation.” Different Types of Evaluation | CYFAR, cyfar.org/different-types-evaluation.
Kirkpatrick Partners, 2018 “Our Philosophy.” Kirkpatrick Partners, The One and Only Kirkpatrick Company®,
www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model.

References (contd.)
Hall, Jori N., et al. “Values Engagement in Evaluation.” American Journal of Evaluation, vol. 33, no. 2, 2011, pp. 195–207.,
doi:10.1177/1098214011422592.
Patton, Michael Quinn. Utilization-Focused Evaluation. Sage Publications, 2008.
SAGE. “The Principles of Empowerment Evaluation with David Fetterman.” YouTube, YouTube, 21 Nov. 2014, www.youtube.com/watch?
v=B8pZ3_ntX6E. SAGE. “The Principles of Empowerment Evaluation with David Fetterman.” YouTube, YouTube, 21 Nov. 2014,
www.youtube.com/watch?v=B8pZ3_ntX6E. SAGE. “The Principles of Empowerment Evaluation with David Fetterman.” YouTube, YouTube, 21 Nov.
2014, www.youtube.com/watch?v=B8pZ3_ntX6E.
Smith, C. “Outgrowing Our Monolith.” Is It Better to Bandage a Wound or Leave It Uncovered When It Is Healing? | Zocdoc Answers, 30 Oct. 2017,
www.zocdoc.com/about/blog/tech/outgrowing-our-monolith/.“Outgrowing Our Monolith.” Is It Better to Bandage a Wound or Leave It Uncovered
When It Is Healing? | Zocdoc Answers, 30 Oct. 2017, www.zocdoc.com/about/blog/tech/outgrowing-our-monolith/.“Outgrowing Our Monolith.” Is It
Better to Bandage a Wound or Leave It Uncovered When It Is Healing? | Zocdoc Answers, 30 Oct. 2017,