Signature intrusion detection systems (SIDS) are based on pattern matching techniques to find a known attack; these are also known as Knowledge-based Detection or Misuse Detection (Khraisat et al., 2018). In SIDS, matching methods are used to find a previous intrusion. In other words, when an intrusion signature matches with the signature of a previous intrusion that already exists in the signature database, an alarm signal is triggered. For SIDS, host's logs are inspected to find sequences of commands or actions which have previously been identified as malware. SIDS have also been labelled in the literature as Knowledge-Based Detection or Misuse Detection (Modi et al., 2013). Figure 1 demonstrates the conceptual working of SIDS approaches. The main idea is to build a database of intrusion signatures and to compare the current set of activities against the existing signatures and raise an alarm if a match is found. For example, a rule in the form of "if: antecedent -then: consequent" may lead to "if (source IP address=destination IP address) then label as an attack ". SIDS usually gives an excellent detection accuracy for previously known intrusions (Kreibich & Crowcroft, 2004). However, SIDS has difficulty in detecting zero-day attacks for the reason that no matching signature exists in the database until the signature of the new attack is extracted and stored. SIDS are employed in numerous common tools, for instance, Snort (Roesch, 1999) and NetSTAT (Vigna & Kemmerer, 1999). Traditional approaches to SIDS examine network packets and try matching against a database of signatures. But these techniques are unable to identify attacks that span several packets. As modern malware is more sophisticated it may be necessary to extract signature information over multiple packets. This requires the IDS to recall the contents of earlier packets. With regards to creating a signature for SIDS, generally, there have been a number of methods where signatures are created as state machines (Meiners et al., 2010), formal language string patterns or semantic conditions (Lin et al., 2011). The increasing rate of zero-day attacks (Symantec, 2017) has rendered SIDS techniques progressively less effective because no prior signature exists for any such attacks. Polymorphic variants of the malware and the rising amount of targeted attacks can further undermine the adequacy of this traditional paradigm. A potential solution to this problem would be to use AIDS techniques, which operate by profiling what is an acceptable behavior rather than what is anomalous, as described in the next section. Page 2
Learn new skills or earn credit towards a degree at your own pace, with no deadlines, using free courses from Saylor Academy. We're committed to removing barriers to education and helping you build essential skills to advance your career goals. Choose a course below, or check out our full course catalog. Log in or Sign up to enroll in courses, track your progress, gain access to final exams, and get a free certificate of completion! Anomaly-based network intrusion detection plays a vital role in protecting networks against malicious activities. In recent years, data mining techniques have gained importance in addressing security issues in network. Intrusion detection systems (IDS) aim to identify intrusions with a low false alarm rate and a high detection rate. Although classification-based data mining techniques are popular, they are not effective to detect unknown attacks. Unsupervised learning methods have been given a closer look for network IDS, which are insignificant to detect dynamic intrusion activities. The recent contributions in literature focus on machine learning techniques to build anomaly-based intrusion detection systems, which extract the knowledge from training phase. Though existing intrusion detection techniques address the latest types of attacks like DoS, Probe, U2R, and R2L, reducing false alarm rate is a challenging issue. Most network IDS depend on the deployed environment. Hence, developing a system which is independent of the deployed environment with fast and appropriate feature selection method is a challenging issue. The exponential growth of zero-day attacks emphasizing the need of security mechanisms which can accurately detect previously unknown attacks is another challenging task. In this work, an attempt is made to develop generic meta-heuristic scale for both known and unknown attacks with a high detection rate and low false alarm rate by adopting efficient feature optimization techniques.
Today, the world has numerous inventions and technological developments with proliferation of the Internet. Advances in business forced the organizations and governments worldwide to invent and use sophisticated and modern networks. These networks mix a variety of security aspects such as encryption, data integrity, authentication, and technologies like distributed storage systems, voice over Internet protocol (VoIP), wireless access, and web services. Enterprises are more available to these systems. For instance, numerous business associations enable access to their administration on the system through intranet and web to their partners; endeavors empower clients to connect with the systems by means of web-based business exchanges that enable representatives to get to data by methods for virtual private systems. This usage makes it more vulnerable to attacks and intrusions. A security threat comes not only from the external intruders but also from internal user in the form of abuse and misuse. A firewall simply blocks the network but cannot protect against intrusion attempts. In contrast, intrusion detection system (IDS) can monitor the abnormal activities on the network. Intrusion detection systems play a vital role in research and development with an increase in attacks on computers and networks [1]. Intrusion detection systems monitor the events occurring in a computer system or networks for analyzing the patterns of intrusions. IDS examine a host or network to spot the potential intrusions. Host-based systems explore the system calls and process identifiers mainly related to the operating system data. On the other hand, network-based systems analyze network-related events like traffic volume, IP address, service ports, and protocol used. Intrusion detection systems will
The intrusion detection systems are broadly classified as
A misuse detection system is also called as signature-based detection that uses recognized patterns [2]. These patterns describe suspect, collection of sequences of activities or operations that can be possibly be harmful and stored in database. It uses well-defined patterns of the attack that exploits the weaknesses in system. The time taken to match with the patterns stored in the database is minimal. A key benefit of these systems is that the patterns or signatures can easily develop and understand the network behavior if familiar. It is more efficient to handle the attacks whose patterns are already maintained in the database. The major restriction of these signature-based approaches is that they can only detect the intrusions whose attack patterns are already stored in the database. For every attack, its signature is to be created. Attacks whose patterns are not present in the database cannot be detected. Such technique can be easily deceived as they are dependent on a specific set of expressions and string matching. In addition, the signature works well only against fixed behavioral patterns; they fail to handle the attacks with human interference or attacks with inherent self-modifying behavioral characteristics. These detection systems are also ineffective in cases where client works on new technology platforms such as no operation (NoP) generators, encoding, and decoding payloads. The efficiency of the signature-based systems decreases due to the need of creating dynamic signatures for different variations. With growing volume of signatures, the performance of the engine also might lose the momentum. Because of this, intrusion detection frameworks are conducted on multiprocessors and Gigabit cards. IDS developers develop new signatures before the attackers develop solutions, in order to prevent any new kind of attacks on the system. Network behavior is the major parameter on which the anomaly detection systems rely upon. If the network behavior is within the predefined behavior, then the network transaction is accepted or else it triggers the alert in the anomaly detection system [3]. Acceptable network performance can be either predetermined or learned through specifications or conditions defined by the network administrator. The crucial stage of behavior determination is regarding the ability of detection system engine toward multiple protocols at each level. The IDS engine must be able to understand the process of protocols and its goal. Despite the fact that the protocol analysis is very expensive in terms of computation, the benefits like increasing rule set assist in lesser levels of false-positive alarms. Defining the rule sets is one of the key drawbacks of anomaly-based detection. The efficiency of the system depends on the effective implementation and testing of rule sets on all the protocols. In addition, a variety of protocols that are used by different vendors impact the rule defining the process. In addition to the aforesaid, custom protocols also add complexity to the process of rule defining. For accurate detection, the administration should clearly understand the acceptable network behavior. However, with strong incorporation of rules and protocol, the anomaly detection procedure would likely to perform more efficiently. However, if the malicious behavior falls under the accepted behavior, in such conditions it might get unnoticed. The major benefit of the anomaly-based detection system is about the scope for detection of novel attacks. This type of intrusion detection approach could also be feasible, even if the lack of signature patterns matches and also works in the condition that is beyond regular patterns of traffic. Advertisement In Figure 1, common intrusion detection framework (CIDF) integrated with Internet Engineering Tasks Force (IETF) and Intrusion Detection Working Group (IDWG) has successfully achieved efficient performance in representing the framework. This group defines a basic IDS structural design based on four functional modules. Common intrusion detection framework architecture. Event modules (E-Modules)are defined as a combination of sensing elements and are engaged in continuous monitoring of the end system. In addition, these modules are also involved in processing the information events to the bottom three modules for further analysis. Analysis modules (A-Modules)analyze the events and detect probable aggressive behavior, in order to ensure that some kind of alarm generated in essential conditions. Data storage modules (D-modules)store the data from the E-Modules for further processing by the other modules. Response modules (R-Modules)are used to provide the response to the transactions based on the information obtained from the analysis module. Figure 2 represent the Common anomaly-based network IDS. The functional stages normally adopted in the anomaly-based network intrusion detection systems (ANIDS) are as follows: Common anomaly-based network IDS. Formation of attributes:In this stage, preprocessing of the attributes is done based on the target system. Observation stage:A model that is built on the basis of behavioral features of the specified system where observations of intrusions can be carried out either through automatically or by manual detection procedure. Functional stage: It is also called as detection stage. If the characterizing system model is available, it will match with the observed traffic. Advertisement Figure 3 represents the taxonomy of anomaly-based intrusion detection techniques. They are statistical based, cognitive based or knowledge based, machine learning or soft computing based, data mining based, user intention identification, and computer immunology. Classification of anomaly-based intrusion detection techniques. Statistical-based techniques use statistical properties such as mean and variance on normal transaction to build the normal profile [4]. The statistical tests are employed to determine whether the observed transaction deviates from the normal profile. The IDS assigns a score to the transactions whose profile deviates from the normal. If the score reaches the threshold, alarm is raised. The threshold value is set based on count of events that occur over a period of time. Statistical-based techniques are further classified into operational model or threshold metric, time series model, Markov process model or Marker model, parametric approaches, statistical moments or mean and standard deviation model, multivariate model, and nonparametric approaches. The main advantages of statistical-based techniques are as follows:
The disadvantages of statistical-based techniques are as follows:
Knowledge-based techniques are used to extract the knowledge from the specific attacks and system vulnerabilities. This knowledge can be further used to identify the intrusions or attacks happening in the network or system. They generate alarm as soon as an attack is detected. They can be used for both misuse and anomaly-based detection [5]. The knowledge-based techniques are broadly classified as state transition analysis, expert systems, and signature analysis. The knowledge-based techniques possess good accuracy and very low false alarm rates. The knowledge gathered makes security analyst easier to take preventive or corrective action. The knowledge-based techniques are maintaining the knowledge of each attack based on the careful and detailed analysis performed; it is a time-consuming task. A prior knowledge to update the each attack is a difficult task. The knowledge-based IDS can detect the attacks whose patterns are known, but it is difficult to detect the inside attacks. One of the solutions is data mining techniques. The core idea is to extract the useful patterns and also the previously ignored patterns from the dataset [6]. The data mining-based techniques are further classified into clustering, association rule discovery, classification, K-nearest neighbor, and decision tree methods. The key advantages of data mining-based techniques are as follows:
The key disadvantages of data mining-based techniques are as follows:
Machine learning can be characterized as the capacity of a program or potentially a framework to learn and improve their performance on a specific task or group of tasks over a time [7]. Machine learning strategies emphasize on building a framework that enhances its execution based on previous results, that is, it can change their execution strategy based on recently acquired data. Machine learning-based techniques are broadly classified as Bayesian approaches, support vector machines, neural networks, fuzzy logic, and genetic algorithms. Their key advantage is flexibility, adaptability, and capture of interdependencies. The disadvantage is high algorithmic complexity and long training times. Intrusion detection system can be built based on the features that categorize the user or the system usage, to distinguish the abnormal activities from normal activities. During the early investigation of anomaly detection, the main emphasis was on profiling system or user behavior from monitored system log or accounting log data. The log data or system log may contain UNIX shell commands, system calls, key strokes, audit events, and network packages used. Computer immunology is a field of science that includes high-throughput genomic and bioinformatics approaches to immunology. The main objective is to convert immunological data into computational problems, solve these problems using statistical and computational approaches, and then convert the results into immunologically meaningful interpretations. Advertisement The NSL-KDD [8] dataset is a refined version of its predecessor KDD99 dataset. NSL-KDD dataset comprises close to 4,900,000 unique connection vectors, where every connection vector consists of 41 features of which 34 are continuous features and 07 are discrete features. Each vector is labeled as either normal or attack. There are four major categories of attacks labeled in NSL-KDD: denial of service attack, probing attack, users-to-root attack, and remote-to-local attack.
Advertisement Although many methods and systems have been developed by the research community, there are still a number of open research issues and challenges. Some of the research issues and challenges of AIDS are as follows:
Advertisement The preprocessed set of network transactions are partitioned based on its labeling (“normal” transactions as one set, “DoS” transactions as the other set and similar other range of sets). Unique values of each feature value set fivNTSin the resultant normal transactions set (NTS) and its percentage of coverage are: fiv=fiv1c1fiv2c2fiv3c3fiv4c4..…………fivjcjE1 The procedure for feature optimization for each attack Akis as follows:
It is imperative from the implementation of the above procedure that optimal features of a specific attack Akcan be identified. Further, the optimal features are ordered using the canonical correlation values. The values with lower than threshold are considered as optional set of features. Reducing the features leads to lesser computational complexities to the minimal level. The optimal features shall be used for further assessing the impact scale intrusion of type Ak. Advertisement The approach for measuring the proposed feature association support fasmetric considers the network transaction of the training dataset. The feature categorical values used in the network transactions are in the form of two independent sets. These values are used to develop a duplex graph between them. Let f1f2f3.……fn∀fi=fiv1fiv2.………fivmbe the set of categorical features values used for forming the set of network transactions T. Here Tis a set of network transaction records of the given training set such as: T=t1t2t3.……tn∀ti=valf1,valf2,..…valfi,valfi+1,……valfnE2 Categorical values of the set of features related to every network transaction shall be considered as transaction value set tvsand all transaction value sets are treated as “STVS.” In the description above in Eq. 2, valfican be expressed as valfi∈fiv1fiv2……fivm. The term “feature” refers to the current categorical value of the feature. The two features “valfi” and “valfj,” “valfi” are connected with “valfj” if and only if valfivalfj∈tvsk.
wvalf1↔valf2=ctvs∣STVS∣E3
E=tvsivalj:valj∈tvsitvsi∈STVSvalj∈vE4
fasfivj=∑k=1∣STVS∣utvsk:fivj→tvsk≠0∑k=1∣STVS∣utvskE5
faistvsi=1−∑j=1mfasvalj∃valj∈V:valj⊂tvsi∣tvsi∣E6
faist=∑i=1∣STVS∣faistvsi∣STVS∣E7
sdvfaist=∑i=1∣STVS∣faistvsi−faist2STVS−1E8
Advertisement The total number of records chosen for the test is 25% of the actual dataset, that is, 34,361. The combination of test records chosen is from various categories such as Probe, DoS, U2R, R2L, and Normal. The difference between CC average and standard deviation of CC is called as lower bound of CC threshold. The sum of CC average and standard deviation of CC is called as upper bound of CC threshold. The records that identified to be normal are 19.8% of the total test data records, with observations of 4.7% of it as “false negatives” and 15.1% of it as “true negatives.” The cumulative number of records that are detected as “intruded transactions” is 80.2%, with 75.3% of them being “truly intruded transactions” of test data records and the “false positive” percentage of 4.9% of test data records.
Comparison of performance metrics of FCAAIS and FAIS. As per the results obtained, the proposed model is found to be accurate up to 90.4%. The experiments are conducted on the same dataset using “anomaly-based network intrusion detection through assessing Feature Association Impact Scale (FAIS)” [14]. The results depict that the proposed model is also scalable and effective for detecting the scope of intrusion from a network transaction. Despite the fact that the FAIS model proposed shows 88% accuracy, the major limitation is process complexity in training the system. Such process complexities of designing the scale using FAIS are due to the number of features selected for assessing the scale. The issue of selecting the optimal features for training the Intrusion Detection System using Association Impact Scale is significantly addressed in the FCAAIS [15] model. Table 1 indicates the comparison of performance metrics such as precision, recall/sensitivity, specificity, accuracy, and F-measure of FCAAIS over FAIS. Figure 4 indicates that the accuracy of FCAAIS with optimal features is 91%, whereas the FAIS accuracy with all features is 88%. The precision of the FCAAIS model with optimal features and FAIS with all features is 92%. The other performance metrics such as sensitivity, specificity, and F-measure is calculated on FCAAIS over FAIS. The sensitivity, specificity, and F-measure are 96, 49, and 95%, respectively, for FCAAIS, whereas sensitivity, specificity, and F-measure are 95, 46, and 91%, respectively, for FAIS. The performance metrics observed for FCAAIS over FAIS. According to the results, the accuracy of FCAAIS (selected feature set using canonical correlation) minimized the process complexity of designing the scale using FAIS (Figure 5 and Table 2). The process computational time observed for FCAAIS over FAIS.
Process computational time of FCAAIS and FAIS. The observed time complexity is adaptable, as the completion time is not directly related to the ratio of features count, which is due to the higher CC threshold as shown in Figure 6. Hence it is obvious to conclude that the applying canonical correlation toward optimized attribute selection is significant improvement to the FAIS model (shown in Figure 6). The FCAAIS consumption of time under divergent canonical correlation thresholds. It is observed that applying canonical correlation toward optimized attribute selection results in 3% improvement in the accuracy of FAIS [14]. Table 3 indicates precision, recall, and F-measure values calculated under divergent canonical correlation threshold values (Figure 7).
Precision, recall, and F-measure values calculated under divergent canonical correlation threshold. Performance analysis of the prediction accuracy of FCAAIS under divergent canonical correlation threshold value. Advertisement It is desirable for anomaly-based network intrusion detection system to achieve high classification accuracy and reduce the process complexity of extracting the rules from training data. In this chapter, a canonical correlation analysis is proposed to optimize the features toward designing the scale to detect the intrusions. The selection of optimal features simplifies the process of FAIS. The experiments were conducted using a benchmark NSL-KDD dataset. The results indicate that the accuracy of FCAAIS with optimal features is 91%, whereas the FAIS accuracy with all features is 88%. The precision of the FCAAIS model with optimal features and FAIS with all features is almost close to 92%. It is observed that applying canonical correlation toward optimized attribute selection has 3% improvement in the accuracy of FAIS. The other performance metrics such as sensitivity, specificity, and F-measure is calculated on FCAAIS over FAIS. The sensitivity, specificity, and F-measure are 96, 49, and 95%, respectively, for FCAAIS, whereas they are 95, 46, and 91%, respectively, for FAIS.
Submitted: July 31st, 2018 Reviewed: October 28th, 2018 Published: June 11th, 2019 © 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |