Evaluating L&D in a Knowledge Economy

This presentation explores the importance of evaluating learning and development (L&D) in a knowledge economy. It discusses the different evaluation theories and models used in L&D, as well as how evaluation fits within L&D processes and systems. The presentation also highlights the challenges and implications of evaluating L&D activities and explores the concept of intellectual capital and its accounting in a knowledge economy.

Evaluating L&D in a
Knowledge Economy
Name of the student:
Name of the university:

Introduction
 Knowledge economy is based on intellectual capital
 The international network of the brainpower is called
knowledge economy (Bathelt, Golfetto and Rinallo 2014)
 Evaluation and growth of knowledge, skills, capability,
experience and intellectual growth are required in this
economy.
 Learning and development (L&D) as well as evaluation are
important for the growth of knowledge economy.
 Through L&D and evaluation, the purpose of independent
learning is achieved.
 Gathering and developing knowledge on business and
evolving technologies is a prerequisite of success in the
modern business world and it creates the need for evaluation
of the L&D processes (North and Kumta 2018).

Evaluation: Definition
 Definition of Evaluation:
 Evaluation: A systematic process of critical examination
and determination of the merit, significance and worth
of a subject against some criteria governed by a set of
standards (Rossi, Lipsey and Henry 2018).
 In the context of L&D, evaluation refers to purposeful
gathering, analysis as well as discussion of the
evidences from different relevant sources regarding the
worth, quality and impact of learning (Nielsen and
Kreiner 2017)
 Evaluation process is dependent on judgment of the

Types of evaluation
1)Formative
2)Summative
3)Process
4)Outcomes
5)Impact (Venable, Pries-Heje and
Baskerville 2017)

Purposes of evaluation
1)Adequacy assessment
2)Plausibility assessment
3)Probability evaluation
(Stufflebeam and Coryn 2014)

How evaluation fits within L&D
processes and systems?
 Evaluation of L&D processes and systems is
necessary to understand the effectiveness of the
programs and knowledge obtained from those
(Kearns 2014)
 Ensures the efficiency of an organization’s learning
and development initiatives and programs for the
employees
 Learner’s satisfaction level is assessed and validated
for assessing the effectiveness of L&D programs
 Evaluation methodologies or parameters are set
before the learning process is implemented

How evaluation fits within L&D
processes and systems? (Contd.)
 Evaluation process starts after the L&D programs are completed.
Based on feedback and learners’ performances, modifications and
improvements in the programs are done.
 Without evaluation, further improvement in the L&D programs is
not possible
 Models like ADDIE can be used to evaluate the L&D programs
 Iteration process under evaluation is used to eliminate unnecessary
and ineffective steps or contents from L&D programs
 Aligned with organizational strategy and is useful for predicting
future strategies (Beevers and Rea 2016)
 Essential for designing improved teaching/learning strategy in
future based on the findings/outcome of the current L&D programs

ADDIE model of evaluation
 ADDIE stands for:
 Analysis
 Design
 Development
 Implementation
 Evaluation (Almomen et al. 2016)
 It is a flexible and dynamic guideline for creating
effective training programs and L&D contents
 Evaluation is done to see if the organizational goals
are being met after training is imparted to the learners

Learning transfer
 Learning transfer refers to the efficient application
of the knowledge attained from the L&D programs
into real business problems (Weiss, Khoshgoftaar
and Wang 2016)
 It is the bridge between the learning from training
programs and L&D activities and real business
results (Cormier and Hagman 2014).
 Effective learning transfer maximizes the L&D
outcome.

Importance of learning transfer
into the workplace
 Importance lies in ability of the employees to apply
learning to practice and meet the employers’
expectations.
 Successful transfer of learning results in higher returns
on investment and higher profit for a business (Belenky
and Schalk 2014)
 Learning is aimed at enhancing knowledge and
potentiality of employees in a business and transfer
enables them to execute the knowledge acquired in
solving business problems (Renta-Davids et al. 2014)

Three evaluation theories
 Objectives-based approach vs
naturalistic approach
 Educational connoisseurship
model
 Metrics-based model
(Kirkpatrick's Four Levels)

Objectives-based approach vs
Naturalistic approach
 Objective based approach determines the degree of achievement
of educational objectives
 In naturalistic approach, key concerns and issues of participants
are addressed (Stavropoulou and Stroubouki 2014)
 Data type: Qualitative
 Advantages: Objective based approach is straight-forward and
follows scientific tradition, while in naturalistic approach, contexts
are acknowledged and used for benefitting those being studied
 Disadvantages: In objective based approach, unintended
outcomes are ignored and takes no account of students as
individuals with all their differences (Xie 2016), while in
naturalistic approach, participants may identify criteria with little
educational worth.

Educational connoisseurship model
 The model is developed by American educationist Elliot W. Eisner
 The model focuses on the qualitative assessment of the social
development of students.
 Three main dimensions reflecting three qualitative actions:
descriptive, interpretative and evaluative or appraisal
dimension. These stages are identified as Eisner’s art of
disclosure, where criticism is also important (Khanipoor, Amini
and Bazrafcan 2017)
 Data type: Qualitative
 Advantages: Individualistic approach of learning is emphasized
in this model, and evaluation of curriculum should be based on
appreciation and criticism
 Disadvantages: The model is cumbersome and strenuous for
the evaluators and good evaluators are required who have better
knowledge and judgments (Xie 2016).

Metrics-based model (Kirkpatrick's Four
Levels) Kirkpatrick's Four Level Training evaluation model is one mostly used
metrics based model for analyzing learning effectiveness (Kirkpatrick and
Kirkpatrick 2016).
 It is useful for assessing if the learners are able to put the knowledge into practice
 Four levels: Reaction, Learning, Behavior and Results
 Reaction: to identify the feedback of the learners to measure how well they
received it
 Learning: specific learning objectives are set and measured before and after
training
 Behavior: This demonstrates how well people have received training
 Results: This represents the outcomes of training, that is, application of
knowledge and skills acquired through training
 Data type: both qualitative and quantitative
 Advantages: it provides logical structure and process for measure learning and
feedback collection is faster. It also helps in aligning business objectives and L&D
programs (Stufflebeam and Coryn 2016)

Compare and contrast three different
evaluation theories
Objectives-based
approach vs
naturalistic approach
Educational
connoisseurship
model
Metrics-based model
(Kirkpatrick's Four
Levels)
More individualistic
approach, on the basis
of educational
objectives
Focuses more on the
creativity and based on
appreciation and
criticisms
Detailed systematic
process consisting of 4
levels of evaluation
Positive outcomes are
only considered, while
unintended outcomes
are ignored
Depends more on
interpretations and
judgments
More logical and
scientific
Deals with qualitative
data
Deals with qualitative
data
Deals with both
qualitative and
quantitative data

Key steps in the evaluation process
 Identifying purpose
 Key stakeholders
 Timing
 Methodology
 Reporting

Sources and methods of analysis
 Evaluation process involves data collection
regarding the L&D programs and its analysis.
 Sources of data collection are (Rossi, Lipsey and
Henry 2018):
 Leaners satisfaction ratings
 Learner results
 Teacher evaluations
 Customer evaluations
 Analysis process: Quantitative and qualitative

Three challenges to evaluate
L&D
 Lack of funding
 Lack of evaluation expertise
 Limited capacity

Characteristics of a knowledge economy
 The economy is solely based on the knowledge-based
industries.
 Knowledge is the primary factor of production and also
considered as wealth
 High level of investment is done in open-innovation, creativity
and knowledge production, such as, education
 Technology-intensive activities, such as, high-tech
manufacturing takes place, such as, electronics, computer,
aerospace etc. (Hadad 2017)
 Service sector industries mainly comprise of education,
software design, healthcare etc.
 It is mainly digital economy and virtualization is highly
important

Characteristics of a knowledge
economy (Contd.)
 The explicit knowledge includes facts, figures, and
quantitative data
 The tacit knowledge includes work mechanisms,
experience, intuition, judgment and handling highly skilled
human resources
 Knowledge is used to generate tangible and intangible
values and expertise is considered as a critical factor along
with other economic resources, where skilled human capital
is a key component (Bathelt, Golfetto and Rinallo 2014)
 Knowledge gained through experience is also as significant
as formal training and education and hence, L&D activities
are highly important

Implications for evaluating L&D
activity Assessing whether the intended L&D objectives have
been met
 Ensuring continuous improvement in the L&D process
 Examining whether the resources are used efficiently or
wisely and justifying investment in knowledge generation
 Assessing the value for money of the L&D activities and
programs as well as the risks
 Examining the availability of the expertise for generating
evaluation frameworks and analyzing the information
provided by the learners (North and Kumta 2018).
 Identifying the strengths and weaknesses of the learners
and take improvement measures accordingly

Intellectual capital
 Intellectual capital is the set of knowledge in an
organization, which contributes towards improving the
competitive position of the organization through value
additions to key stakeholders (Berzkalne and Zelgalve
2014).
 It is the difference between the value of the tangible
net assets and its market capitalization.
 Three key components of intellectual capital is human
capital, external capital and internal capital (Wang,
Wang and Liang 2014).

Intellectual Capital Accounting
 It is the process of measuring the value of
intellectual capital of an organization.
 Intellectual capital accounting is significant in a
knowledge economy.
 It is considered as a challenging job as the
intellectual capital in intangible and hard to measure
(Zambon and Dumay 2016)
 It is done to maximize the value of a firm by
combining intellectual capital with other type of
capitals.

References
 Almomen, R.K., Kaufman, D., Alotaibi, H., Al-Rowais, N.A., Albeik, M. and Albattal, S.M., 2016. Applying the
ADDIE—analysis, design, development, implementation and evaluation—instructional design model to
continuing professional development for primary care physicians in Saudi Arabia. Int J Clin Med, 7(08), pp.538-
546.
 Bathelt, H., Golfetto, F. and Rinallo, D., 2014. Trade shows in the globalizing knowledge economy. OUP Oxford.
 Beevers, K. and Rea, A., 2016. Learning and Development Practice in the Workplace. Kogan Page Publishers.
 Belenky, D.M. and Schalk, L., 2014. The effects of idealized and grounded materials on learning, transfer, and
interest: An organizing framework for categorizing external knowledge representations. Educational
Psychology Review, 26(1), pp.27-50.
 Berzkalne, I. and Zelgalve, E., 2014. Intellectual capital and company value. Procedia-Social and Behavioral
Sciences, 110, pp.887-896.
 Braeckman, L., Venema, A., Hermans, L., Van Zoelen, S.A., De Ridder, M., Van der Laan, G. and Van Dijk, F.,
2018. 1147 Development and evaluation of a training programme on scientific research and occupational
health surveillance in turkey.
 cipd.co.uk, 2019. Evaluating Learning & Development | Factsheets | CIPD. [online] CIPD. Available at:
https://www.cipd.co.uk/knowledge/fundamentals/people/development/evaluating-learning-factsheet [Accessed
9 Sep. 2019].
 Cormier, S.M. and Hagman, J.D. eds., 2014. Transfer of learning: Contemporary research and applications.
Academic Press.
 Hadad, S., 2017. Knowledge economy: Characteristics and dimensions. Management dynamics in the
Knowledge economy, 5(2), pp.203-225.

References (Contd.)
 Kearns, P., 2014. Organizational Learning and Development: From an Evidence Base.
Routledge.
 Khanipoor, F., Amini, M. and Bazrafcan, L., 2017. Evaluation of educational program in the
Master of Medical Education by Eisner's educational connoisseurship and criticism
model. Journal of education and health promotion, 6.
 Kirkpatrick, J.D. and Kirkpatrick, W.K., 2016. Kirkpatrick's four levels of training evaluation.
Association for Talent Development.
 Nielsen, T. and Kreiner, S., 2017. Course evaluation for the purpose of development: what can
learning styles contribute?. Studies in Educational Evaluation, 54, pp.58-70.
 North, K. and Kumta, G., 2018. Knowledge management: Value creation through organizational
learning. Springer.
 Renta-Davids, A.I., Jiménez-González, J.M., Fandos-Garrido, M. and González-Soto, Á.P., 2014.
Transfer of learning: Motivation, training design and learning-conducive work effects. European
Journal of Training and Development, 38(8), pp.728-744.
 Rossi, P.H., Lipsey, M.W. and Henry, G.T., 2018. Evaluation: A systematic approach. Sage
publications.
 Stavropoulou, A. and Stroubouki, T., 2014. Evaluation of educational programmes-the
contribution of history to modern evaluation thinking. Health Science Journal, 8(2), pp.193-
204.

References (Contd.)
 Stufflebeam, D.L. and Coryn, C.L., 2014. Evaluation theory, models, and
applications (Vol. 50). John Wiley & Sons.
 Venable, J., Pries-Heje, J. and Baskerville, R., 2016. FEDS: a framework
for evaluation in design science research. European journal of
information systems, 25(1), pp.77-89.
 Wang, Z., Wang, N. and Liang, H., 2014. Knowledge sharing, intellectual
capital and firm performance. Management decision, 52(2), pp.230-
258.
 Weiss, K., Khoshgoftaar, T.M. and Wang, D., 2016. A survey of transfer
learning. Journal of Big data, 3(1), p.9.
 Xie, Q., 2016. Theories of Training Program Evaluation Frameworks and
Methodologies. In English Language Training in the Workplace (pp. 63-
77). Springer, Cham.
 Zambon, S. and Dumay, J., 2016. A critical reflection on the future of
intellectual capital: from reporting to disclosure. Journal of Intellectual
capital.

Thank You!