Big Data ICT707: Comprehensive Analysis of Regression Models

This assignment solution provides an overview of regression models in the context of big data, focusing on their application in predicting continuous target variables. It begins by introducing the importance of data integration in large organizations and then delves into various regression techniques and their uses, such as predicting stock returns, loan default amounts, customer lifetime value, and recommendations. The assignment covers feature extraction, target variable transformation, and the training of regression models using machine learning libraries like Spark. It also references techniques like Gradient Boosted Trees and Linear Regression, highlighting their practical applications and the statistical methods used to find the best regression model. The solution concludes with a discussion on coding practices and provides a list of references to support the concepts discussed. Desklib offers this and many other solved assignments for students.

Introduction:
For large organizations preparing huge amounts of information for instance client
information, money related information, fabricating information. It is difficult to
do the capacity to get to every one of the information is an urgent one. So
information mix assumes a huge part in each association. Information mix is
happening by joining a few sources and give bound together form of those assets.
For instance a few organizations may need to work together each other. At that
point they join their databases. This includes information combination that implies
a few information incorporated to give single adaptation of those information. In
the Information joining has a few classes that incorporates center information
reconciliation.
Data science pratice
Decision tree building
Post Building Decision Trees in Python is the topic of when look at the tree with
decision trees contains numerical continuous variable is depending on this. This
kind of choice trees can be called likewise regression tree. Characterization of
trees, as the name suggests are utilized to isolate the dataset into classes having a
place with the response variable. Cleveland, W., & Hafen, R. (2014).
Classification is the problem where happening in typical related that can be caught
in the fields such as machine learning, data mining and predictive analysis. By then
they join their databases. This incorporates data blend that suggests a couple of
data consolidated to give single adjustment of those data. In the Information
joining has a couple of classes that consolidates focus data compromise. Chang
mang yen. (2011).

Gradient Boosted Tree:
This technique is used to build the predicative models. This technique is most
powerful and machine learning process. And also used to classification problem
and regression problems. Iterative functional gradient descent algorithm is present
in this gradient tree technique. Nascimento, M., & Amorim, G. (2017)
This is also similar like decision tree building by the same way it also do the tasks
like of trees, as the name proposes are used to segregate the dataset into classes
having a place with the reaction variable. Grouping is where occurring in regular
related that can be gotten in the fields, for example, machine learning, information
mining and prescient investigation. Piccolo, G. (2016).
This algorithm is used to reduce the cost of the function. We have to use decision
tress for implementing this algorithm. And this algorithm is used to improve the
quality of each learner. This algorithm is used for web search engines. There are
three type of elements. They are:
(i) Additive model
(ii) Loss function
(iii) Weak learner

Bike sharing data set:
There are many ride share companies are there like lyft and uber they provide
business models with good and convenient and also in the form of efficient and
affordable, convenient price for customers who owns or operates vehicle. Re, A.
(2012)
This advanced system is used to easily get the bikes for rent from one particular
place. Then we have to return back at another place. Now a days this method is
used to avoid the traffic in the environmental. And then reduce the pollution. And
500 bike sharing function are there in world wide. This system is used to generate
to the data sets in real world. This data sets are used to research purpose. And also
this system is used to virtual sensors. Thelwall, M. (2016)
This sensors are used to sense the mobility in the particular place. Next this
system is used to monitoring the main events from the city and get the data. This
process is related to the seasonal process such as precipitation, seasons, weather
condition, hour if the day and day of weak.
Linear regression:
Linear regression is technique which follows statistical data analysis and it is
mostly used for determining relationships between variables where one is
dependent variable and another one is independent variable and there are different
types of variables are available and they are linear, simple linear and multiple
linear regressions. Wang yon. (2018).
Formula for linear regression is
Y = a_1*X_1 + a_2*X_2 + a_3*X_3 ……. a_n*X_n + b
There are some differences are there between the simple and multiple regressions
and the differences are:

1. Simple linear regression: it uses independent variable and used for predicting the
values from a dependent variable.
2. Multiple linear regression: similar like linear regression but which uses two or
more variables and those variables are independent and those are used for
predicting value to a dependent variable.
Finding best regression model:
For a decent regression model, we need to incorporate the factors that you are
particularly trying alongside different factors that influence the reaction keeping in
mind of the end goal to maintaining the strategic distance from one-sided comes
about. Minitab statistical software or programming offers some measures and
techniques that assistance you indicate your show in the best way. And it will
survey the normal strategies and also to do take after the connections to perusing
my more point by point posts about each. Weihs, C., & Ickstadt, K. (2018).

Coding:

References

Cleveland, W., & Hafen, R. (2014). Divide and recombine (D&R): Data science for
large complex data. Statistical Analysis and Data Mining: The ASA Data Science
Journal, 7(6), 425-433. doi: 10.1002/sam.11242
Nascimento, M., & Amorim, G. (2017). The Enviroment Education in Pratice:
Challenges ahead Legislation. Amadeus International Multidisciplinary
Journal, 1(2), 31. doi: 10.14295/aimj.v1i2.11
Piccolo, G. (2016). Ethics consultation in clinical pratice of transplants. Medicina E
Morale, 64(6). doi: 10.4081/mem.2015.6
Re, A. (2012). Interdisciplinary Research and pratice in Ergonomics. Journal of
Biological Research - Bollettino Della Società Italiana Di Biologia
Sperimentale, 85(1). doi: 10.4081/4064
Thelwall, M., & Thelwall, M. (2016). Data Science Altmetrics. Journal of Data And
Information Science, 1(2), 7-12. doi: 10.20309/jdis.201610
Weihs, C., & Ickstadt, K. (2018). Data Science: the impact of statistics. International
Journal Of Data Science And Analytics. doi: 10.1007/s41060-018-0102-5
Wang yon. (2018). Song Jun-pil’s Neo-Confucian Theory and Social Pratice
Movement - Focus on the relevance of Hanju School -. 한한한한한, null(70), 183-
205. doi: 10.18399/actako.2018..70.007
Chang mang yen. (2011). Pratice Of Interpretation-Study for Wen, Tian Xiang's
Jidushi. CHINESE LITERATURE, 66(null), 27-44. doi:
10.21192/scll.66..201102.002