Analysis of Virtualization and Cloud Computing with IaaS and MapReduce

This comprehensive report delves into the realms of cloud computing and virtualization, examining key concepts such as Infrastructure as a Service (IaaS) and the MapReduce programming model. It provides an overview of cloud computing's background, motivation, and research objectives, including the exploration of IaaS software, particularly Amazon Web Services (AWS), and its various services. The report also covers the functionalities of MapReduce and Apache Hadoop, illustrating their application through examples. Furthermore, it presents a literature review, design parameters, and analysis of autoscaling AMIs and Hadoop load monitoring. The evaluation section discusses generating test loads and analyzing autoscaling frameworks. The report concludes with recommendations for further research and potential future scopes in the field of cloud computing and virtualization, highlighting the significance of these technologies in modern data management and processing.

Running Head: VIRTUALIZATION AND CLOUD COMPUTING
VIRTUALIZATION AND CLOUD COMPUTING
Name of the Student
Name of the University

1VIRTUALIZATION AND CLOUD COMPUTING
Table of Contents
Chapter 1: Introduction..............................................................................................................4
1.1. Background.....................................................................................................................4
1.2.Motivation........................................................................................................................4
1.3. Research Aim and Objectives.........................................................................................5
1.4. Research Questions.........................................................................................................6
1.5. Research Methodology....................................................................................................6
1.6. Research Outlines............................................................................................................7
1.7. Conclusion.......................................................................................................................7
Chapter 2: Description...............................................................................................................9
2.1. IaaS Software..................................................................................................................9
2.1.1. Amazon Services....................................................................................................11
2.1.2. Access Credentials.................................................................................................12
2.2. MapReduce and Apache Hadoop..................................................................................13
2.2.1. MapReduce............................................................................................................13
2.2.2. Apache Hadoop......................................................................................................13
2.2.3. Apache Hadoop by an Example.............................................................................13
Chapter 3: Literature Review...................................................................................................15
3.1. Introduction...................................................................................................................15
3.2. Background data on Cloud Computing.........................................................................15

2VIRTUALIZATION AND CLOUD COMPUTING
3.3. Amazon Simple Storage Service...................................................................................17
3.4. MapReduce Functions effort-based..............................................................................20
3.5. Motivation of the Project..............................................................................................21
3.6. Conclusion.....................................................................................................................21
Chapter 4: Design Parameters & Analysis...............................................................................23
4.1. Autoscalable AMIs........................................................................................................23
4.1.1. Requirements..........................................................................................................24
4.1.2. Meeting the Requirements.....................................................................................26
4.2. Hadoop Load Monitoring..............................................................................................28
4.2.1. Heartbeat................................................................................................................29
4.2.2. System Metrics.......................................................................................................29
4.2.3. Ganglia...................................................................................................................29
4.2.4. Amazon CloudWatch.............................................................................................30
4.2.5. Conclusion..............................................................................................................30
Chapter 5: Evaluation...............................................................................................................32
5.1. Generating Test Load....................................................................................................32
5.1.1. HDFS Synthetic Load Generator...........................................................................32
5.1.2. Apache Hadoop Sort Benchmark...........................................................................32
5.1.3. Identifying Relevant Metrics..................................................................................33
5.2. Autoscaling Framework................................................................................................33
5.2.1. Load Average Autoscaling.....................................................................................33

3VIRTUALIZATION AND CLOUD COMPUTING
5.2.2 Amazon AutoScaling..............................................................................................33
5.2.3. Custom Scaling Framework...................................................................................34
5.2.4. Scaling the Cluster Down.......................................................................................34
5.2.5. Apache Hadoop and Cluster Changes....................................................................34
Chapter 6: Conclusion, Recommendation and Future Works..................................................36
6.1. Conclusion.....................................................................................................................36
6.2. Recommendation for Further Research........................................................................37
6.3. Future Scope..................................................................................................................39
References................................................................................................................................40

4VIRTUALIZATION AND CLOUD COMPUTING
Chapter 1: Introduction
1.1. Background
Cloud computing has been used all over the world because of the versatility and
flexibleness for Data storing and transferring the data keeping required secrecy. This
technology provides demanded IT products and a service, where virtualization has a major
role to play, that is distributes computing on internet and delivery of computer service
provided over internet [94 p.164]. Virtualization in Cloud Computing provides services to the
cloud clients. This is a brand new concept of business paradigm that has its basic concept in
multi-tenancy, virtualization and shared infrastructure. The concept of IT directs to the
configuring, manipulating and accessing of online applications, offering online storage,
infrastructure and application. It is based of computing resources, combination of software
and hardware which are delivered as network service [81]. Deployment Models and Service
Models are two working model for cloud computing. Service model is also called the
reference model. Service model is divided in to IaaS, PaaS and Saas service cloud.
1.2. Motivation
Cloud computing and virtualization startups have several reasons to move from an
intranet based, models dependent of capital-purchase of infrastructure in IT to some cloud-
based service and utility style demand [10]. This manages the shifting computing
requirements by giving better flexibility in the service provided to the organization. The
cloud based infrastructure is much more versatile from both service and security perspective.
Scalability in cloud computing provides greater flexibility by managing the shifting
computer in requirements in the services they have purchased. This is a more versatile service
in terms of scalability than in local, intranet-based infrastructure [14]. Reliability in the terms

5VIRTUALIZATION AND CLOUD COMPUTING
where the cloud vendors creates greater redundancy in the system than the organization can
help in building onto the intranet, the cloud vendor hence spreads the infrastructure
investment price across entire base of the customers allocating necessary resources [59].
Virtualization since the cloud-based IT infrastructure can be geographically dislocated and
virtualized. It infrastructure startups have been freed from being considered as the physical
location of its data centers and business operations decisions. Affordability within the
traditional infrastructure startups is generally not received. Then marginal cost provided to
the cloud computing features like enhancement of security or many other unaffordable
services are offered for free to the startups using the cloud computing options [95 p. 169].
There are even security options are provided and somewhat are still required in the
cloud services: User data hiding from other users (individual or organization) of the same
cloud service, securing the data from the cloud provider itself, securing and hiding
computation within several servers and securing the computation between unauthorized and
unreliable parties [3].
1.3. Research Aim and Objectives
Here, in this report we are going to discuss the IaaS approaches to migrating in cloud
based organization database to the cloud, an approach to migrating database to the cloud
service. Discussing the advantages and disadvantages of each approach and recommend with
reason why this approach has been selected. Discussing further optimizing the resources of
the compute by allocating it and considering the dynamic processing workload deserves
investigation and finally the MapReduce programming model supporting that distributes
computing large data sets on clusters of computing resources.

6VIRTUALIZATION AND CLOUD COMPUTING
1.4. Research Questions
1) Optimizing resources that have been allocated and then considering dynamic
processing workload deserves investigation.
2) An Infrastructure used for the scalability of computing resources.
3) Effective sentiment analysis of twitter data by using map reduce programming
4) Does the MapReduce program model supports the distribution of large dataset
computing on the computer resources of cluster level?
1.5. Research Methodology
Cloud computing has been spoken as a vital topic because of the innovative
implementation and approach. The most vital point about cloud computing was that, there
was a presence of redefined computing that could be included in anything that people do or
want to do. There is a consistent discussion on cloud computing such that to represent a shift
in paradigm. In this chaptermain focus point is the implementation of specific procedure and
framework which would be used in presence of the research work [46].Ones the security
issue relate aspect would be resolved any other aspects are resolved, the cloud computing
could enable the enterprise forexpandsion. Expansion can be in various fields such as its
infrastructure, outsource the whole infrastructure, resulting in a better flexibility, a wider
choice of resources related to computing and significant cost saving.What appears as a
emerging is the taxonomy of cloud computing includes: infrastructure as a service
(INFRASTRUCTURE AS A SERVICE); platform as a service (PLATFORM AS A
SERVICE) and software as a service (SOFTWARE AS A SERVICE) [9].Consideration such
as data privacy, network performance, security and economics are likely to be a donating
sector in both within the organization and outside the organization. Cloud computing has
gained acceptance and popularity. There are many concepts through which cloud computing
can be related.

7VIRTUALIZATION AND CLOUD COMPUTING
1.6. Research Outlines
The two types of research approaches are Inductive and deductive. The planning and
framing out the appropriate theories is the main function of inductive approach, this helps in
the fulfillment of the main objectives of the approach. Whereas, in the deductive approach the
research support are generally conducted. Hence, the research theories are based on the
previously studied theories and models [6].
At first the research subject has been covered by the researcher and then the
researcher aims at planning and framing the exact model and theory related to previous study.
The main aim of the exploratory design of research is to evaluate different context of the area
in which research is to be done. The researcher adopts the inductive approach and then
divides the research work into various parts. According to the data that could be defined in
the terms of information sets, can be considered to be useful and relevant to the research.
According to inductive approach of research, it has an effect over a longer time period [28].
Therefore, these collection of information can be utilized in the particular study is one of the
most significant aspect conduct any particular work of research.
In the case of deductive approach the procedure of comparison are detected and are
compared over the researches theory and hence frames new models or theories accordingly
[16].This had stated that exploratory design usage does not ensure assurance may or may not
be addressed of the conclusive solutions at the end of the research in particular.
1.7. Conclusion
In this chapter, we have discussed the basic concept of cloud computing and
virtualization along with the need and requirement of this cloud services. Virtualization in
Cloud Computing provides services to the cloud clients. This is a brand new concept of
business paradigm which is based on the concept of multi-tenancy, virtualization and shared

8VIRTUALIZATION AND CLOUD COMPUTING
infrastructure. This chapter includes the motivation of this project analysis, along with
research aim and objectives, the required questions and methodology for the research to carry
onward.

9VIRTUALIZATION AND CLOUD COMPUTING
Chapter 2: Description
2.1. IaaS Software
Definition of IaaS approach: Infrastructure as a Service (IaaS) refers to the type of
cloud computing that can provide virtualization via computer resources over the internet. It
resembles a self-service model that can access, monitor and manage any remote datacenter
infrastructure. Like the computers that can stores, virtualizes network or networking services
like the firewalls [68]. There is a chance of the customer buying IaaS platforms instead of
using hardware on the basis of the outright consumption.
IaaS users as comparing to other approaches are responsible to manage the data,
applications, middleware, runtime and OS system approaches. This platform provides offers
to the database system, manages queries, servers, hard drives and network over the layer of
virtualization. Here, user can install the required platform with IaaS infrastructure over the
top [8]. On availability of new versions these users get responsible to update the system.
Software of IaaS provider combines or works solo for providing service to the
organization or individual worldwide. Some of the examples of IaaS software:
1) Microsoft Azure: This is a public cloud service providing platform and
infrastructure. This offers wider array of integrated web service which can
help in maintaining and building solution for any type and size of business [63
p-141]. Microsoft Azure is a combination of both IaaS and PaaS service.
2) Google Cloud Platform: It refers to cloud-host platform that enables the users
to use pre-defined web services and tools for creating web based solution for
the business need. This platform helps to provide flexibility and scalability of

10VIRTUALIZATION AND CLOUD COMPUTING
IaaS, supporting both Google-offered web-service building blocksand third
party software service and allocation.
3) IBM SoftLayer: This cloud infrastructure have hardware-based servers, virtual
servers and storage options are stored and delivered over the public, private
and management network systems. This kind of infrastructure offers solution
to the e-commerce, big data, government, digital marketing, gaming, private
clouds and reseller hosting.
4) CloudSigma: This resembles a cloud hosting service which hosts servers in the
cloud, helps in manufacturing or construct a hybrid or cloud-based
infrastructure. This helps in increasing the availability and performance
without sacrificing visibility. This service transfers and migrates the existing
system effortlessly to the clouds ensuring the visibility of the system and the
data privacy and security with proper data protection standards.
5) Oracle Cloud Infrastructure: The business uses this cloud service to run the
workloads, replicate networks, VPNs, back up data offsite and much more.
6) Amazon Web Services: AWS or the Amazon Web Services is a IaaS based
platform that the business of the organization can use to create web-based
solution. It provides wider variety of individual services as well and combines
together to cover the business need like content delivery, database
management, storage, networking, mobile services, analytics and more. AWS
helps in constructing the solution they need for the business in aspect of IaaS,
leaving the physical management of infrastructure to the Amazon. AWS
provides a marketplace where third-party providers offer AWS-integrated
solutions [47].

11VIRTUALIZATION AND CLOUD COMPUTING
There are several other software providing IaaS service like Rackspace, Rackspace,
VmwarevCloud Air, Verizon Enterprise’s information technology (IT), SingleHop,
ServerCentral, Citrix Workspace Cloud, IronOrbit, CDI (Computer Design and Integration),
Virtustream, Interoute’s Infrastructure as a Service (IaaS) and Aviatrix [96 p. 64].
2.1.1. Amazon Services
As discussed in the previous section, Amazon Web Services (AWS) is a cloud based
platform for manufacturing and constructing solution for business using the integrated web
services [5]. This provides hardware infrastructure that may enable the business to spread
wider. AWS other than web services also provides platform services like email, mobile
development, analytics, calendaring and application testing. Increases the productivity and
efficiency of the business development like the management tools, developer tool, security
services, identity protection services and other application services. But there is a dedicated
network connection between its services and your business [77]. Through Internet of Things
(LoT) web services AWS provides web services by using chip-embedded objects to integrate
the business. This is a global service provider located in data centers, compliances and
security programs for ensuring the development in the business in industry such that the
service can meet the regional and global compliance standards of government and industries.
AWS can provide a Virtual Private Cloud (VPC), which logically isolates AWS space, so one
can have complete control over the environment. This software also supports hybrid
environment maintaining the system resources on the premises.
There are several benefits of AWS as this platform does not only provides IaaS
offerings.