CPS 5951-02: Advanced Software Engineering Paper Analysis - Fall 2019

This assignment analyzes a paper titled "Software Engineering for Machine Learning: A Case Study," focusing on AI software development at Microsoft. The analysis addresses the authors' objectives, the presence and relevance of a literature review (or lack thereof), and any proposed new ideas, such as a customized machine-learning process model. It evaluates the use of theories, proofs, and inferences within the paper, highlighting strengths like the mixed-method approach (combining surveys and interviews) and weaknesses, such as the absence of a literature review and theoretical underpinnings. The assignment also discusses the applicability of the mixed-method approach, identifies defects in the paper, and proposes resolutions. It concludes by summarizing the authors' conclusions and comparing the paper to another reference on the same topic, offering a comparative analysis to determine which paper is better and why.

Running Head: CPS 5951-02 Advanced Software Engineering Fall 2019 1
CPS 5951-02 Advanced Software Engineering Fall 2019
Student Name
Institution
Course
Date

CPS 5951-02 Advanced Software Engineering Fall 2019 2
What do the authors want to achieve in the paper?
The authors wanted to analyze how the growing challenges of Artificial Intelligence
software development approach can be managed. To achieve this objective, the authors chose to
interview Microsoft employees. The choice of the study considered the fact that Microsoft
software development has more customer-focused AI features than any other company.
According to the authors, the company had integrated agile software methodology which has AI-
specific workflow informed by the past AI and data science applications (Amershi et al., 2019).
In order to get the answer to this question, Microsoft employees were interviewed on how they
have managed to overcome the challenges of AI software development as well as the integral
issues in the development of AI applications.
From their analysis, they discovered three essential differences exhibited in the
application building process and other platforms for training machine learning models they had
not witnessed before. First, they discovered that machine learning was all about data. Secondly,
they discovered that building softwares for extensibility and customizability required the
development teams to not only have software engineering skills but also a deep knowledge on
machine learning (Amershi et al., 2019).
About literature review- are they related to the objective presented in this paper? Please explain it.
A literature review is the evaluation of the literature sources in a chosen or given topic
area. The literature review documents the arguments of other sources in respect to the chosen
topic or subject matter. Basically it has four objectives which include surveying the literature on
the chosen topic, synthesizing the information contained in those sources, critically analyzing the
information gathered while identifying the knowledge gap and then presenting the facts in an

CPS 5951-02 Advanced Software Engineering Fall 2019 3
organized manner (Hart, 2018). A literature review shows the learners that the author has in-
depth knowledge on the subject matter and that they understand where the research fits. Based on
this description of a literature review, it is clear that the study chosen did not have a literature.
This is among the weaknesses of this study because it shows that the authors did not survey prior
sources about their topic.
Any new idea proposed in this paper? If yes, what is it?
The paper has proposed some set of best practices which can be used to build
applications and other platforms which rely on machine learning. Also, it has proposed a
customized machine-learning process model which can be used to assess the progress of the
software development teams to ensure that they come up with excellent AI applications.
According to the authors, the process maturity metric helps the development team to identify
how far they have gone in the journey of building an AI application.
Is there any theory, proof, inference, or deduction made in this paper?
Theories, proofs and inferences are very important in research studies because the
assumptions made in them enable readers to evaluate their discussions explicitly. Also, they
connect researchers to the existing knowledge and give them the basis for their hypothesis and
the choice of research methods (Siebenkäs & Stelzer, 2018). Consequently, articulating theory
assumptions in a research study enables the researchers to address the questions “why” and
“how” as well as identifying the limits to different generalizations. The study did not have any
theories, inferences or proofs. This is another weakness of this study because it could not connect
researchers to the existing knowledge.
What is good and what is bad for this paper?

CPS 5951-02 Advanced Software Engineering Fall 2019 4
It is good that this paper has used a mixed-method approach of data collection which is
among the best approaches in data collection. Mixed methods approach is a data collection
approach that involves the collection, analysis and integration of both quantitative and qualitative
data collection methods. As observed in the paper, surveys and interviews were used in the
collection of data (Morse, 2016). Surveys are grouped under the quantitative data collection
methods whereas interviews are grouped under the qualitative approach to data collection.
Mixing both qualitative and quantitative approaches in this study enabled the researchers to gain
depth and breadth of understanding and support while equipoising the weaknesses which are
inherent when each approach is used independently ( Palinkas et al., 2015). The main advantage
of using a mixed method approach in research is the possibility of triangulation to examine a
single phenomenon. Triangulation enables researchers to identify a single aspect of a
phenomenon more accurately because it is approached from different points of view.
It is bad that this paper could neglect a very important chapter of a research paper like
literature review. As a result, the paper has ended up lacking support documents and arguments
which could have supported its topic and subject matter. Also, it did not consider any theories,
proofs and inferences which could have enabled the authors to evaluate their arguments
explicitly (Siebenkäs & Stelzer, 2018).
If you find something good, how can you apply it?
I would apply mixed-method approach of data collection in my future research studies.
Mixed methods approach is a data collection approach that involves the collection, analysis and
integration of both quantitative and qualitative data collection methods. This is because mixing
both quantitative and qualitative approaches will enable me to gain depth and breadth of

CPS 5951-02 Advanced Software Engineering Fall 2019 5
understanding and support while equipoising the weaknesses which are inherent when each
approach is used independently ( Palinkas et al., 2015).
Can you find any defect in this paper? If you find any defects, do you have any resolution?
Neglecting some of the crucial research paper chapters is the main defect of this paper
which has seen the paper lack support documents and arguments which could have supported the
topic and subject matter. To avoid this defect, I would suggest the future authors to list down the
key sub-sections of a research paper as a guide before embarking on it.
What did the author summarize in this article, do you agree?
In summary, the paper has mentioned that Microsoft has made significant efforts to
develop extensive portfolio of AI platforms through the integration of machine learning into the
existing development processes and cultivating on the growing ML talent. I totally agree with
this summary considering the trends reported in the company as far as machine learning and AI
is concerned.
Have you found any other references to talk about the same topic?
Yes, I have come across another reference talking about machine learning in software
engineering (Di Stefano & Menzies, 2002)
Di Stefano, J. S., & Menzies, T. (2002, November). Machine learning for software engineering:
Case studies in software reuse. In 14th IEEE International Conference on Tools with Artificial
Intelligence, 2002.(ICTAI 2002). Proceedings. (pp. 246-251). IEEE.
Can you compare them and tell me which paper is better in your opinion? Why?

CPS 5951-02 Advanced Software Engineering Fall 2019 6
The second paper is better than the original paper because its analysis of this topic is deep
rooted. This has made it more clear and explorative than the first paper. Also, it has considered
almost all the important subsections of a research paper like the use of theories and literature
review unlike the first paper which was very shallow and neglected the two sub-sections of a
research paper.
References
Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., ... & Zimmermann, T. (2019,
May). Software engineering for machine learning: a case study. In Proceedings of the
41st International Conference on Software Engineering: Software Engineering in
Practice (pp. 291-300). IEEE Press.
Di Stefano, J. S., & Menzies, T. (2002, November). Machine learning for software engineering:
Case studies in software reuse. In 14th IEEE International Conference on Tools with
Artificial Intelligence, 2002.(ICTAI 2002). Proceedings. (pp. 246-251). IEEE.
Hart, C. (2018). Doing a literature review: Releasing the research imagination. Sage.
Morse, J. M. (2016). Mixed method design: Principles and procedures. Routledge.
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K.
(2015). Purposeful sampling for qualitative data collection and analysis in mixed method
implementation research. Administration and policy in mental health and mental health
services research, 42(5), 533-544.

CPS 5951-02 Advanced Software Engineering Fall 2019 7
Siebenkäs, A., & Stelzer, D. (2018, August). Assessing Theories for Research on Personal Data
Transparency. In IFIP International Summer School on Privacy and Identity
Management (pp. 239-254). Springer, Cham.