Ethical Implications of an Untested AI System: A Detailed Report

This report addresses the ethical concerns surrounding the release of untested AI systems, emphasizing the potential for software bugs, data breaches, and reputational damage. The report highlights a scenario where a manager pressures an employee to launch a system without testing to meet a client's deadline, discussing the implications of such actions, including financial losses, data security risks, and potential harm, particularly in sensitive applications like healthcare. The report analyzes the Healthcare.gov system failure as a case study. It proposes solutions such as rigorous testing, implementing ethical guidelines, and fostering open communication within development teams. Furthermore, the report advocates for a teleological approach, prioritizing actions that result in the greatest good and least harm, and emphasizes the importance of ethical imagination and anticipating potential misuse of the software. References to relevant literature support the analysis, providing a comprehensive overview of the ethical challenges and potential solutions in AI system development.

Running head: THE UN-TESTED SYSTEM ETHICAL ISSUE 1
The Un-Tested System Ethical Issue
Name
Institutional Affiliation
(Word Count 1045)

THE UN-TESTED SYSTEM ETHICAL ISSUE 2
The Un-Tested System Ethical Issue
Ethical Issue Identification
In most cases, firms dealing with product design and development should determine a
product’s readiness for release. This step takes place after spending resources such as money,
time, and labor to identify a problem, create a solution, and test the system. In this video, a
manager is pressurizing an employee to launch a system without undertaking tests on it to meet a
client’s deadline (Al-Saggaf, 2016). Sometimes, product launch does not depend on a new
application being ready. However, it involves informing the world that the product has been
released and ready for use. Clients start loading information into the software database since they
believe it is secure and bug-free.
Edwards and Steinke (2008) notes that software errors and bugs are a normal part of an
IT application. As such, many program vendors provide no expectation of correctness or
guarantee of reliability when releasing a software. Additionally, companies find it easier to
launch a product, spend more resources on marketing, and then fix bugs after complaints from
users. Edwards and Steinke (2008) argues that the ICT vendor has the ethical responsibility for
testing and correcting a product.
Implications of the Ethical Problem
The manager believes that the team should release the product without testing to impress
the client and avoid losing business. However, the employee responsible for the launch believes
that the piece of software should be tested first to ascertain that it works properly and meets user
requirements. On the contrary, it can damage the reputation of the product vendor for launching
an untested product. For instance, the software can suddenly crush or spit out the wrong output
that may lead to loss of trust due to the presence of bugs on the code. Moreover, client may lose

THE UN-TESTED SYSTEM ETHICAL ISSUE 3
critical data through a program that has defects. As a result, a bug-plagued software can become
costly for both the client and service provider. Previous studies have shown that this unethical
move in ICT has been prevalent and detrimental and has led to an annual loss of $59.5 billion in
America (Waychal and Capretz, 2016). The costs are borne by the developer and users. In some
cases, if the program is installed in a medical equipment, it can be dangerous or cause death.
A software product is prone to errors due to its overwhelming sophistication. In includes
millions of lines of codes that are prone to defects. As such, developers should be given enough
time to identify and correct bugs to prevent future loss or damage on firm’s reputation.
According to Richardson and Gwaltney (2005), improved testing enables earlier identification
and correction of software bugs and can eliminate a third of defect-related loss. Unfortunately,
most software developers are driven by greed to release products early without proper testing.
Al-Saggaf, Burmeister, and Weckert (2015) reasonably argued that human weakness could push
them to exploit product vulnerabilities to obtain a capital gain. Their study found that greed
factor triggered unethical conduct in the field of ICT. Ineffective corporate and IT policies can
lead to abuse of workers and company assets by those in power for personal gains. As a result,
more than half of software errors are discovered after formal product launch.
An example of the impact of this ethical issue involves the 2013 Healthcare.gov massive
system failure caused by lack of rigorous evaluation before implementation (Roy, 2013). The
healthcare exchange product was rolled out to support Affordable Care Act. However, the
government stifled effective agile development and appropriate testing caused the development
cost to swell three times from the budgeted amount. In contrast, if the defect was identified
earlier, fixing it would have been cheaper.
Solution to the Ethical Problem

THE UN-TESTED SYSTEM ETHICAL ISSUE 4
During the development process, the project team should test a product to identify bugs.
Developers can deploy an online tracking solution to record defects and implement a streamlined
approach to resolving such defects. Identified bugs should be ranked according to their severity
and impact. In this case, the developer in the video can choose to delay the launch date to ensure
that the system is tested for release. The system is only considered safe and reliable after the
defect triage is complete and the developers have completed correcting the defects. This live
software is now guaranteed to be free from defects and can be adopted by a client.
Staying ahead of this ethical issue requires integrity and vigilance. The firm should
implement a measure makes client’s data safety paramount, resisting scenarios of greed and
corruption. Notably, meeting such standards can be challenging for firm managers and owners
who focus on revenue creation and sales improvement. On the other side, the developer should
bring up such ethical issues in early stages of product development for the CEO to communicate
to the client before the deadline. Apparently, ethical imagination is required for products that are
used by different stakeholders. The developer should consciously use some imagination and
avoid assuming that users will think in a similar way. In other words, a programmer should think
of a way that a client will use a properly developed software improperly. Imagining a worst case
scenario in the early stages will help launch a bug-free system on time.
When faced with an ethical dilemma such as the one presented in the video, the
developers primary obligation is to communicate on the manner and share his or her concerns
with management. As such, the firm should ensure that they have implemented internal
processes that encourage sharing of challenges and concerns. Similarly, the management is
required to listen to the employee to create a solution. The concerns and facts should be
documented for future reference.

THE UN-TESTED SYSTEM ETHICAL ISSUE 5
This ethical issue can be solved through a teleological theory that focuses on the actions
and the consequences (Guy, 1990). The manager and the developer should choose an action,
among a set, that will lead to a greatest good and least harm. The set of actions, according to the
YouTube video, include letting the system go live, developer resigning, or fighting back.
From this discussion, it is hard to negotiate ethical issues involving an active decision. It
gets more challenging when the challenge involves a problem that can be postponed and labeled
a task that can be fixed later after a product has been launched. However, a company should
avoid launching un-tested programs since a single bug can cause inconceivable harm and loss.

THE UN-TESTED SYSTEM ETHICAL ISSUE 6
References
Al-Saggaf, Y. (2016, October 20). A dilemma in IT: Select action end of video and see its
consequence [Video file]. Retrieved from https://www.youtube.com/watch?
v=mugeCY3vbxo
Al-Saggaf, Y., Burmeister, O., & Weckert, J. (2015). Reasons behind unethical behavior in the
Australian ICT workplace: An empirical investigation. Journal of Information,
Communication and Ethics in Society, 13(3/4), 235-255.
Edwards, J. N., & Steinke, G. (2008). Ethical issues in the software quality assurance function.
Communication of the IIMA, 8(1), 53-58.
Guy, M. E. (1990). Ethical decision making in everyday work situations. New York: Quorum
Books.
Richardson, J., & Gwaltney, J. W. (2005). Ship it! A practical guide to successful software
projects. Pragmatic Bookshelf.
Roy, A. (2013). Obamacare’s website is crashing because it doesn’t want you to know how
costly its plans are. Forbes. Retrieved from
https://www.forbes.com/sites/theapothecary/2013/10/14/obamacares-website-is-crashing-
because-it-doesnt-want-you-to-know-health-plans-true-costs/#464665f57a5f
Waychal, P. K., & Capretz, L. F. (2016). Why a testing career is not the first choices of
engineers. 123rd Annual Conference of the American Society for Engineering Education,
14994.