SENG6210A: Software Engineering Masters Project

SENG6210A: Software Engineering Masters
Project Part A
Project: Log Analysis
SUBMITTED BY:-DALJEET KAUR
STUDENT ID:-c3298354

What is log
A log is detailed writing of tasks done by a computer program.
In computing, a log file keeps the records of operating system’s actions, messages
between different users of a communication software. Logging is the act of keeping
a log. messages are written in a single log file.
In a clear way, logging is just a fancy word to define a process of writing
down everything you do.
There are various types of logging. We can log every operation of an
application, during the occurrence of errors, or in some critical
operations done by a user, especially if someone wants to get control of
who’s doing what for audit purposes.
Applications can log at a code level for debugging or at a user level for
audits and forbidden access register.
Every operating system has routines or services running on the
background with the task of taking notes of everything that is happening.
This way a person can detect security breaches, malfunctions etc.
In simplest way, logging is keeping a record of all data, processes, and
results in a program. This is part of a much more grand, complex
process, though, so you want to program with a clear goal in mind, and
not try to do several programming disciplines at once.
Event logs
it records events taking place in the execution of a system in order to provide
an audit trail that can be used to understand the activity of the system and to
diagnose problems. They are necessary to understand the complexity of system.
It is beneficial to combine log file entries from multiple sources. This approach along
with statistical analysis, may result in correlations between seemingly unrelated
events on different servers.
Transaction log
These logs record changes in the stored data to allow the database to recover
from crashes or errors and maintain the stored data in a stable state. Thus,
database systems usually have both general event logs and transaction logs.
Message logs
Internet Relay Chat (IRC), instant messaging (IM) programs, peer-to-peer file
sharing clients with chat functions, and multiplayer games (especially MMORPGs)
commonly have the ability to automatically log (i.e. save) textual communication,
both public (IRC channel/IM conference/MMO public/party chat messages) and
private chat messages between users. Message logs are almost universally plain
text files, but IM and VoIP clients (which supports textual chat, e.g. Skype) might
save them in HTML files or in a custom format to ease reading and encryption.

How logs are generated and used
A log is the automatically generated when any task or event occur on computer. It is
the time-stamped documentation of events relevant to a particular system. Every
kind of software application even and systems itself produce log files.
For example: -
On a Web server, an access log lists all the individual files that people have
requested from a website. These files will include the HTML files and their imbedded
graphic images and any other associated files that get transmitted. From the server’s
log files, an administrator can identify numbers of visitors, the domains from which
they’re visiting, the number of requests for each page and usage patterns according
to variables such as times of the day, week, month or year.
Log Analysis Tools
As more and more companies move to the cloud, log analytics, log analysis, and log
management tools and services are becoming more critical. DevOps engineers,
system administrators, site reliability engineers, and web developers can all use logs
to make better data-driven decisions.
There are a lot of different log analysis tools out there, below are 5 of the most
popular ones. Each one of them has its own unique search features, live tail queries,
etc. Some of the tools listed are more log management services, but they also offer
more efficient ways to analyse your logs compared to looking at the raw data.
1. Loggly
Loggly is a cloud-based logging management developed in 2009. Some of its clients
are EA, Autodesk, SendGrid, Atlassian, Sony Pictures, and Citrix.
Loggly features: -
has both free and paid plans starting at $49 per month.
 ability to see a bird’s eye view of your logs.
 remove the noise with a few simply.
2. Logentries

It is similar tool as loggly because it is also a cloud-based logging management developed in
2010. Some of their clients are the Discovery Channel, Macy’s, LogMeIn, Yummly, and
Parallels.
Logentries Features:-
Has both free and paid plans starting at $39 per month.
 can filter logs in real-time, giving you much deeper insights into web
application and system events as the occur.
3. Splunk
Splunk is quite famous in the log and application management space. It is developed
in 2003 They offer great solutions for larger enterprise customers.
Splunk Features:-
has both free and paid plans.
offers a great way to collect, store, index, search, correlate, visualize, analyze
and report on any machine-generated data to identify and resolve operational
and security issues in a faster, repeatable, and more affordable way.
a great way to spot and visualize possible trends.
4.logz.io
it provides real-time, actionable insights into your log analytics data with hosted ELK
as a service. Some of its clients are Netflix, Facebook, LinkedIn, Cisco, and
Microsoft. it is probably one of the biggest competitors to Splunk.
logz.io Features: -
has both free and paid plans .
 can filter results by server, application, or any custom parameter until you
reach the source of the problem
Alternative of splunk
5. GoAccess
GoAccess is a fast, terminal-based log analyser. It quickly analyses and view web
server statistics in real-time without needing to use your browser.
GoAccess Features:-
it is open source,