COIT20249: Data Mining and Machine Learning in Cybersecurity Report

This report provides a comprehensive overview of data mining and machine learning techniques applied in the field of cybersecurity. It delves into various aspects, including misuse/signature detection, anomaly detection, hybrid detection systems, scan detection, and network traffic profiling. The report explores the use of classical machine-learning paradigms like association rules, artificial neural networks, support vector machines, decision trees, and Bayesian networks. It also covers unsupervised learning methods such as K-means clustering and principal component analysis. Furthermore, it addresses challenges like modeling large-scale networks, discovering threats, and preserving privacy. The report also examines emerging challenges in cybersecurity, including threats from malware, botnets, and cyber warfare, and explores privacy preservation techniques. Finally, the report discusses research directions and potential solutions for future advancements in cybersecurity.

Data Mining and
Machine Learning
in Cybersecurity
Sumeet Dua and Xian Du
Lffi) CRC Press
\ V J Taylor & Francis Group
Boca Raton London New York
CRC Press is an imprint of the
Taylor & Francis Croup, an informa business
AN AUERBACH BOOK

Contents
List of Figures xi
List of Tables xv
Preface xvii
Authors xxi
1 Introduction 1
1.1 Cybersecurity 2
1.2 Data Mining 5
1.3 Machine Learning 7
1.4 Review of Cybersecurity Solutions 8
1.4.1 Proactive Security Solutions 8
1.4.2 Reactive Security Solutions 9
1.4.2.1 Misuse/Signature Detection 10
1.4.2.2 Anomaly Detection 10
1.4.2.3 Hybrid Detection 13
1.4.2.4 Scan Detection 13
1.4.2.5 Profiling Modules 13
1.5 Summary 14
1.6 Further Reading 15
References 16
2 Classical Machine-Learning Paradigms for Data Mining 23
2.1 Machine Learning 24
2.1.1 Fundamentals of Supervised Machine-Learning
Methods 24
2.1.1.1 Association Rule Classification 24
2.1.1.2 Artificial Neural Network 25

vi • Contents
2.1.1.3 Support Vector Machines 27
2.1.1.4 Decision Trees 29
2.1.1.5 Bayesian Network 30
2.1.1.6 Hidden Markov Model 31
2.1.1.7 Kaiman Filter 34
2.1.1.8 Bootstrap, Bagging, and AdaBoost 34
2.1.1.9 Random Forest 37
2.1.2 Popular Unsupervised Machine-Learning Methods 38
2.1.2.1 £-Means Clustering 38
2.1.2.2 Expectation Maximum 38
2.1.2.3 ^-Nearest Neighbor 40
2.1.2.4 SOM A N N 41
2.1.2.5 Principal Components Analysis 41
2.1.2.6 Subspace Clustering 43
2.2 Improvements on Machine-Learning Methods 44
2.2.1 New Machine-Learning Algorithms 44
2.2.2 Resampling 46
2.2.3 Feature Selection Methods 46
2.2.4 Evaluation Methods 47
2.2.5 Cross Validation 49
2.3 Challenges 50
2.3.1 Challenges in Data Mining 50
2.3.1.1 Modeling Large-Scale Networks 50
2.3.1.2 Discovery of Threats 50
2.3.1.3 Network Dynamics and Cyber Attacks 51
2.3.1.4 Privacy Preservation in Data Mining 51
2.3.2 Challenges in Machine Learning (Supervised
Learning and Unsupervised Learning) 51
2.3.2.1 Online Learning Methods for Dynamic
Modeling of Network Data 52
2.3.2.2 Modeling Data with Skewed Class
Distributions to Handle Rare Event Detection 52
2.3.2.3 Feature Extraction for Data with Evolving
Characteristics 53
2.4 Research Directions 53
2.4.1 Understanding the Fundamental Problems
of Machine-Learning Methods in Cybersecurity 54
2.4.2 Incremental Learning in Cyberinfrastructures 54
2.4.3 Feature Selection/Extraction for Data with Evolving
Characteristics 54
2.4.4 Privacy-Preserving Data Mining 55
2.5 Summary 55
References 55

Contents • vii
Supervised Learning for Misuse/Signature Detection 57
3.1 Misuse/Signature Detection 58
3.2 Machine Learning in Misuse/Signature Detection 60
3.3 Machine-Learning Applications in Misuse Detection 61
3.3.1 Rule-Based Signature Analysis 61
3.3.1.1 Classification Using Association Rules 62
3.3.1.2 Fuzzy-Rule-Based 65
3.3.2 Artificial Neural Network 68
3.3.3 Support Vector Machine 69
3.3.4 Genetic Programming 70
3.3.5 Decision Tree and CART 73
3.3-5.1 Decision-Tree Techniques 74
3.3.5.2 Application of a Decision Tree
in Misuse Detection 75
3.3.5.3 CART 77
3.3.6 Bayesian Network 79
3.3.6.1 Bayesian Network Classifier 79
3.3.6.2 Naive Bayes 82
3.4 Summary 82
References 82
Machine Learning for Anomaly Detection 85
4.1 Introduction 85
4.2 Anomaly Detection 86
4.3 Machine Learning in Anomaly Detection Systems 87
4.4 Machine-Learning Applications in Anomaly Detection 88
4.4.1 Rule-Based Anomaly Detection (Table 1.3, C.6) 89
4.4.1.1 Fuzzy Rule-Based (Table 1.3, C.6) 90
4.4.2 A N N (Table 1.3, C.9) 93
4.4.3 Support Vector Machines (Table 1.3, C.12) 94
4.4.4 Nearest Neighbor-Based Learning (Table 1.3, С П ) 95
4.4.5 Hidden Markov Model 98
4.4.6 Kaiman Filter 99
4.4.7 Unsupervised Anomaly Detection 100
4.4.7.1 Clustering-Based Anomaly Detection 101
4.4.7.2 Random Forests 103
4.4.7.3 Principal Component Analysis/Subspace 104
4.4.7.4 One-Class Supervised Vector Machine 106
4.4.8 Information Theoretic (Table 1.3, C.5) 110
4.4.9 Other Machine-Learning Methods Applied
in Anomaly Detection (Table 1.3, C.2) 110
4.5 Summary I l l
References 112

viii • Contents
Machine Learning for Hybrid Detection 115
5.1 Hybrid Detection 116
5.2 Machine Learning in Hybrid Intrusion Detection Systems 118
5.3 Machine-Learning Applications in Hybrid Intrusion Detection.... 119
5.3.1 Anomaly—Misuse Sequence Detection System 119
5.3.2 Association Rules in Audit Data Analysis
and Mining (Table 1.4, D.4) 120
5.3.3 Misuse—Anomaly Sequence Detection System 122
5-3-4 Parallel Detection System 128
5.3.5 Complex Mixture Detection System 132
5.3.6 Other Hybrid Intrusion Systems 134
5.4 Summary 135
References 136
Machine Learning for Scan Detection 139
6.1 Scan and Scan Detection 140
6.2 Machine Learning in Scan Detection 142
6.3 Machine-Learning Applications in Scan Detection 143
6.4 Other Scan Techniques with Machine-Learning Methods 156
6.5 Summary 156
References 157
Machine Learning for Profiling Network Traffic 159
7.1 Introduction 159
7.2 Network Traffic Profiling and Related Network Traffic
Knowledge 160
7-3 Machine Learning and Network Traffic Profiling 161
7.4 Data-Mining and Machine-Learning Applications
in Network Profiling 162
7.4.1 Other Profiling Methods and Applications 173
7.5 Summary 174
References 175
Privacy-Preserving Data Mining 177
8.1 Privacy Preservation Techniques in PPDM 180
8.1.1 Notations 180
8.1.2 Privacy Preservation in Data Mining 180
8.2 Workflow of PPDM 184
8.2.1 Introduction of the PPDM Workflow 184
8.2.2 PPDM Algorithms 185
8.2.3 Performance Evaluation of PPDM Algorithms 185

Contents • ix
8.3 Data-Mining and Machine-Learning Applications in PPDM 189
8.3.1 Privacy Preservation Association Rules (Table 1.1, A.4).... 189
8.3.2 Privacy Preservation Decision Tree (Table 1.1, A.6) 193
8.3.3 Privacy Preservation Bayesian Network
(Table 1.1, A.2) 194
8.3.4 Privacy Preservation K N N (Table 1.1, A.7) 197
8.3.5 Privacy Preservation £-Means Clustering
(Table 1.1, A.3) 199
8.3.6 Other PPDM Methods 201
8.4 Summary 202
References 204
9 Emerging Challenges in Cybersecurity 207
9.1 Emerging Cyber Threats 208
9.1.1 Threats from Malware 208
9.1.2 Threats from Botnets 209
9.1.3 Threats from Cyber Warfare 211
9.1.4 Threats from Mobile Communication 211
9.1.5 Cyber Crimes 212
9.2 Network Monitoring, Profiling, and Privacy Preservation 213
9.2.1 Privacy Preservation of Original Data 213
9.2.2 Privacy Preservation in the Network Traffic
Monitoring and Profiling Algorithms 214
9.2.3 Privacy Preservation of Monitoring and
Profiling Data 215
9.2.4 Regulation, Laws, and Privacy Preservation 215
9.2.5 Privacy Preservation, Network Monitoring, and
Profiling Example: PRISM 216
9.3 Emerging Challenges in Intrusion Detection 218
9.3.1 Unifying the Current Anomaly Detection Systems 219
9.3.2 Network Traffic Anomaly Detection 219
9.3.3 Imbalanced Learning Problem and Advanced
Evaluation Metrics for IDS 220
9.3.4 Reliable Evaluation Data Sets or Data Generation Tools 221
9.3-5 Privacy Issues in Network Anomaly Detection 222
9.4 Summary 222
References 223
Index 225