Importance of Objective Assessment in Image Processing - Lab Report

This lab report details an image quality assessment (IQA) experiment conducted as part of the Multimedia Communications course (ELEC 30004). The assignment focuses on objective IQA methods, utilizing MATLAB to analyze the quality of images. The report includes an introduction to objective...Read More

Lab Assignment Report 1
Fall 2017
Multimedia Communications-
Lab Assignment
(ELEC 30004)
(Title)
DONE BY
 Student name and Student ID
 Student name and Student ID
Session:
Instructor:
DECLARATION

Lab Assignment Report 2
I hereby solemnly and sincerely declare, to the best of my knowledge and belief that the
report here is the true reflection of the Multimedia lab assignment,
Student’s Name: ………………………………………………………………………………
Registration No.: ………………………………………………………………………………
Signature: ………………………………………… Date: ………………………………….
INSTRUCTOR’S DETAILS
Supervisor’s Name: ………………………………………………………………….
Phone: ………………………………….. E-mail: …………………………………..
Signature: ……………………………… Date: ……………………………………..

Lab Assignment Report 3
ACKNOWLEDGEMENT
I would like to acknowledge my instructor…………. for the support in the Multimedia
Communications course that helped me do this lab assignment.

Lab Assignment Report 4
ABSTRACT
Currently, the demand for digital image-based applications has grown considerably. There have
been widespread image processing applications that require high-quality image processing
techniques. As a result, several Image Quality Assessment (IQA) has been proposed before.
However, this lab assignment focuses on various objective IQA methods that were studied and
their importance discussed. The project was carried on two images of similar dimensions
whereby one was an original image and the other one a degraded image. Particularly, the project
aimed at calculating “Mean Square Error (MSE), Peak Signal to Noise Ratio (PSNR),
Normalized Absolute Error (NAE), Maximum Difference(MD), Structural Content (SC),
Average Difference (AD), and the Normalized Cross Correlation for the images” (Mohammadi,
Moghadam and Shiran, 2014).

Lab Assignment Report 5
TABLE OF CONTENTS
Contents
DECLARATION..............................................................................................................................................2
ACKNOWLEDGEMENT.................................................................................................................................3
ABSTRACT....................................................................................................................................................4
TABLE OF CONTENTS...................................................................................................................................5
LIST OF FIGURES..........................................................................................................................................6
LIST OF TABLES............................................................................................................................................7
LIST OF ABBREVIATION................................................................................................................................8
LIST OF SYMBOLS.........................................................................................................................................9
INTRODUCTION.........................................................................................................................................10
Objective Assessment in Image Processing...........................................................................................10
Problem Statement...............................................................................................................................11
Objective...............................................................................................................................................12
Limitations.............................................................................................................................................12
Requirements........................................................................................................................................12
METHODOLOGY.........................................................................................................................................12
Design Procedure..................................................................................................................................12
MATLAB DESIGN, RESULTS AND ANALYSIS................................................................................................13
CONCLUSION.............................................................................................................................................19
Bibliography...............................................................................................................................................20

Lab Assignment Report 6
LIST OF FIGURES
Figure 1……………………………………………Original image
Figure 2…. ………………………………………. Degraded image
Figure 3……………………………………………. MSE output
Figure 4…. …………………………………………PSNR output
Figure 5……………………………………………. NAE output
Figure 6…………………………………………….MD output
Figure 7…………………………………………….SC output
Figure 8………………………………………………AD output
Figure 9………………………………………………NCC output

Lab Assignment Report 7
LIST OF TABLES
Table 1…………………………………………………MATLAB Results

Lab Assignment Report 8
LIST OF ABBREVIATION
1. IQA… Image Quality Assessment
2. MSE… Mean Square Error
3. PSNR… Peak Signal to Noise Ratio
4. NAE… Normalized Absolute Error
5. MD… Maximum Difference
6. SC… Structural Content
7. AD… Average Difference
8. NCC… Normalized Cross Correlation

Lab Assignment Report 9
LIST OF SYMBOLS
∑
x
y
f ( x , y)… … Summation of the function f (x , y ) from x to y

Lab Assignment Report 10
INTRODUCTION
Objective Assessment in Image Processing
Objective IQA is a technique that uses mathematical models to predict the image quality.
Contrary to Subjective IQA, Objective IQA is automatic and accurate hence giving better results.
Consequently, it is widely applied in various fields. For instance, in control systems to monitor
image quality. Such systems adjust themselves automatically resulting to best quality image data.
Additionally, objective IQA can be used to select among alternatives the best algorithm that
gives higher quality images (Mohammadi, Moghadam and Shiran, 2014).
Mean Square Error (MSE)
MSE is a measure of the signal fidelity by comparing two signals. Fidelity measure provides the
quantitative score which measures the level of similarity or the degree of distortion between the
two images. If x and y represent the original and degraded signals respectively then, the measure
of signal quality, MSE can be given as . Often, the MSE can be
converted to PSNR by the relation, . Where, L=2n
-1 refers to the range of
the acceptable pixel intensities whose value is 255. PSNR is measured in decibels and from the
relation, we can deduce that it is inversely proportional to MSE.
Average Difference (AD)
“Average difference refers to the average of the difference between the original image and the
degraded image. It is given by the equation” (WEI et al., 2013).:

Lab Assignment Report 11
Maximum Difference (MD)
“Refers to the maximum difference between the reference image and the distorted image” (WEI
et al., 2013).
Normalized Cross-Correlation (NK)
NK refers to the closeness to which the reference image and the degraded images are quantified.
Structural content (SC)
“It is a measure of the similarity between a reference image and the distorted image” (WEI et al.,
2013). Where, x(i,j and y (i , j) represents the original image and the
distorted image respectively”.
Problem Statement
Image processing is applied in various fields. Therefore, there is the need to measure the quality
of the images used. The photos could be degraded due to physical limitations from the time they
were captured to the time they are humans view them. Therefore, knowing such distortions could
help designers to code and develop systems that have the highest sensitivity to these distortions.
Notably, subjective evaluation is used to quantify visual image quality. However, such methods

Lab Assignment Report 12
are expensive, time-consuming and lack automation. On the contrary, objective computational
metrics are automatic and can measure image quality and record the results without human
intervention. Objective IQA could eliminate the need for inconvenient, expensive, and time
consuming subjective image quality assessment means.
Objective
The main aim of the lab assignment is to design and simulate “objective Image Quality
Assessment” methods using MATLAB-based algorithms. The lab also focuses on using two test
images to carry out the lab and note the results to be obtained.
Limitations
Despite the fact that MSE and PSNR are used to measure the image quality, the methods are still
susceptible to energy of errors.
Requirements
MATLAB 2016 and two images (the original image and the degraded image) will be used in the lab
assignment.
METHODOLOGY
MATLAB 2016 software has designed the various image quality assessment metrics. MATLAB
software possesses “matrix handling capabilities” and excellent graphics. Additionally,
MATLAB provides a powerful inbuilt toolbox thus offering a conducive environment for
technical computing. Most importantly, it has a “separate toolbox for image processing
applications” (Mohammadi, Moghadam and Shiran, 2014).
Design Procedure
Step 1: Study the metrics already developed for measuring image quality.

Lab Assignment Report 13
Step 2: Select the original image and corrupt it with some noise to obtain the distorted form of
the image.
Step 3: Develop the algorithms and simulate the methods
Step 4: Analyze the results obtained and deduce conclusions based on the analysis.
MATLAB DESIGN, RESULTS AND ANALYSIS
Question 7 (a)
The images considered were image1_Original.jpg and image2_Distorted.jpg where the latter
represents the degraded image. Both images are of the size 512 by 512 pixels.
Figure 1: Original Image (Hooker, 1973) Figure 2: Degraded Image (Hooker, 1973)

Start
Load Original and
Distorted Images
Analyze the
Results Obtained
Evaluate Image Quality
Parameters and Display
Results
End
Lab Assignment Report 14
System Flow Chart
For the quality assessment of the two images, the steps of evaluation are summarized as shown
in the flow chart.
The results obtained are shown in table 1
Table 1: MATLAB results
Assessment Method Value Obtained
“Mean Square Error (MSE)” 7.17
“Peak Signal to Noise Ratio (PSNR)” 39.6098815
“Normalized Absolute Error (NAE)” 0.01
“Maximum Difference (MD)” 220
Structural Content (SC) 1.01

Lab Assignment Report 15
Average Difference (AD) 1.7
Normalized Cross Correlation (NCC) 1.00
Figure 3: MSE Output
Figure 4: PSNR Output

Lab Assignment Report 16
Figure 5: NAE Output
Figure 6: MD Output

Lab Assignment Report 17
Figure 7: SC Output
Figure 8: AD Output

Lab Assignment Report 18
Figure 9: NCC Output
Question 7 part (c)
“Importance of objective assessment in image processing”
Over the recent past, the demand for digital image-based applications has seen considerable
growth in all sectors of the economy. For instance, there have been widespread image processing
applications ranging from the medical research to industrial applications. Often, these
applications require high-quality image processing techniques as required by human quality
judgments. As a result, the efficiency and reliability of image quality evaluation mechanisms
have become fundamental (WEI et al., 2013). Therefore, Image Quality Assessment (IQA) can
be done by either subjective quality assessment or through objective quality assessment.
Objective Image Quality Assessment methods use mathematical models to predict as well as
measure the image quality accurately and automatically. As such, an ideal model mimics the
expected quality levels of an average human. The conventional objective method employed is
full reference IQA where the original, perfect image is used as a reference. Additionally, in
reduced reference IQA the undistorted original image is partially available. Also, objective IQA

Lab Assignment Report 19
employs no-reference IQA when the reference image is unavailable. Objective quality
assessment methods are widely preferred than subjective processes due to many strengths.
First, objective IQA, as opposed to subjective IQA methods, are less expensive and simple to
calculate with less computational complexities. Since they are software-based, a given algorithm
can be implemented to simplify all the computational works. Hence it becomes smooth and
faster to carry out evaluations to ascertain the image quality. Mainly, MSE and PSNR are
sensitive to varied types of distortions.
Secondly, these methods can offer the “repeatability and reliability” missing in subjective image
evaluations. “A machine-vision-based system can provide detailed information about individual
attributes that count to the overall perception of image quality” (WEI et al., 2013). Such a system
not only characterizes line-quality and dot quality, but it also describes color reproduction and
other details. It is imperative to maintain a close correlation between the machine attributes and
observer-based response. However, an objective assessment system provides additional
information that can be used to evaluate various causes of a given defect (Mohammadi,
Moghadam and Shiran, 2014).
Additionally, the methods provide for automation thereby boosting the capabilities and benefits
significantly. An automated system that is designed well can handle assessment of a large
volume of images as well as many image attributes. Therefore, such a system is critical in failure
analysis and control of statistical processes.
CONCLUSION
Image quality measurement is paramount in various applications. "In the recent past, efforts have
been made to develop objective image quality evaluation techniques and algorithms"(Shanableh,

Lab Assignment Report 20
2015). As a result, the lab instilled an understanding of the various objective methods of Image
Quality Assessment and how they can be modeled in MATLAB.

Lab Assignment Report 21
Bibliography
WEI, J., LI, S., LIU, W. and ZANG, Y. (2013). Objective quality evaluation method of stereo
image based on steerable pyramid. Journal of Computer Applications, 32(3), pp.710-714.
Subjective and Objective Quality Assessment of Image: A Survey
Mohammadi, P., Moghadam, A. and Shiran, S. (2014). Subjective and Objective Quality
Assessment of Image: A Survey. Subjective and Objective Quality Assessment of Image, 57(3),
pp.1-51.
Shanableh, T. (2015). A regression-based framework for estimating the objective quality of
HEVC coding units and video frames. Signal Processing: Image Communication, 34, pp.22-31.
Hooker, D. (1973). Retrieved from https://en.wikipedia.org/wiki/Lenna#/media/File:Lenna.png