A hybrid and learning agent architecture for network intrusion detection

A hybrid and learning agent architecture for network intrusion detection

Accepted Manuscript A hybrid and learning agent architecture for network intrusion detection Adriana Leite , Rosario Girardi PII: DOI: Reference: S0...

2MB Sizes 0 Downloads 81 Views

Accepted Manuscript

A hybrid and learning agent architecture for network intrusion detection Adriana Leite , Rosario Girardi PII: DOI: Reference:

S0164-1212(17)30018-3 10.1016/j.jss.2017.01.028 JSS 9914

To appear in:

The Journal of Systems & Software

Received date: Revised date: Accepted date:

11 January 2016 13 December 2016 31 January 2017

Please cite this article as: Adriana Leite , Rosario Girardi , A hybrid and learning agent architecture for network intrusion detection, The Journal of Systems & Software (2017), doi: 10.1016/j.jss.2017.01.028

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT Highlights

CE

PT

ED

M

AN US

CR IP T

A hybrid learning software agent for network intrusion detection is proposed. The agent combines a reactive and a case-based reasoning performance components. A learning component allows to improve the performance by learning reactive rules. The performance and effectiveness of the agent were evaluated. Evaluation results show that the agent is well-suited for dynamic environments.

AC

• • • • •

Corresponding author Federal University of Maranhão Computer Science Department Av. dos Portugueses, 1966, Campus do Bacanga CEP: 65085-580 São Luís - MA - Brazil

ACCEPTED MANUSCRIPT •

A Hybrid and Learning Agent Architecture for Network Intrusion Detection Adriana Leite and Rosario Girardi1

CR IP T

Federal University of Maranhão - Computer Science Department {adri07lc, rosariogirardi}@gmail.com

PT

ED

M

AN US

Abstract — Learning is an effective way for automating the adaptation of systems to their environment. This ability is especially relevant in dynamic environments as computer networks where new intrusions are constantly emerging, most of them having similarities and occurring frequently. Traditional intrusion detection systems still have limitations of adaptability because they are just able to detect intrusions previously set in system design. This paper proposes HyLAA a software agent architecture that combines case-based reasoning, reactive behavior and learning. Through its learning mechanism, HyLAA can adapt itself to its environment and identify new intrusions not previously specified in system design. This is done by learning new reactive rules by observing recurrent good solutions to the same perception from the casebased reasoning system, which will be stored in the agent knowledge base. The effectiveness of HyLAA to detect intrusions using case-based reasoning behavior, the accuracy of the classifier learned by the learning component and both the performance and effectiveness of HyLAA to detect intrusions using hybrid behavior with learning and without learning were evaluated, respectively, by conducting four experiments. In the first experiment, HyLAA exhibited good effectiveness to detect intrusions. In the second experiment the classifiers learned by the learning component presented high accuracy. Both the hybrid agent behavior with learning and without learning (third and fourth experiment, respectively) presented greater effectiveness and a balance between performance and effectiveness, but only the hybrid behavior showed better effectiveness and performance as long as the agent learns.

AC

CE

Keywords – Learning Agents, Hybrid Agents, Case-based Reasoning, Ontologies, Information Security, Intrusion Detection Systems

Corresponding author Federal University of Maranhão Computer Science Department Av. dos Portugueses, 1966, Campus do Bacanga CEP: 65085-580 São Luís - MA - Brazil

ACCEPTED MANUSCRIPT

A Hybrid and LearningAgent Architecture for Network Intrusion Detection

AN US

CR IP T

Abstract — Learning is an effective way for automating the adaptation of systems to their environment. This ability is especially relevant in dynamic environments as computer networks where new intrusions are constantly emerging, most of them having similarities and occurring frequently. Traditional intrusion detection systems still have limitations of adaptability because they are just able to detect intrusions previously set in system design. This paper proposes HyLAA a software agent architecture that combines case-based reasoning, reactive behavior and learning. Through its learning mechanism, HyLAA can adapt itself to its environment and identify new intrusions not previously specified in system design. This is done by learning new reactive rules by observing recurrent good solutions to the same perception from the casebased reasoning system, which will be stored in the agent knowledge base. The effectiveness of HyLAA to detect intrusions using case-based reasoning behavior, the accuracy of the classifier learned by the learning component and both the performance and effectiveness of HyLAA to detect intrusions using hybrid behavior with learning and without learning were evaluated, respectively, by conducting four experiments. In the first experiment, HyLAA exhibited good effectiveness to detect intrusions. In the second experiment the classifiers learned by the learning component presented high accuracy. Both the hybrid agent behavior with learning and without learning (third and fourth experiment, respectively) presented greater effectiveness and a balance between performance and effectiveness, but only the hybrid behavior showed better effectiveness and performance as long as the agent learns.

M

Keywords – Learning Agents, Hybrid Agents, Case-based Reasoning, Ontologies, Information Security, Intrusion Detection Systems

ED

1. INTRODUCTION

AC

CE

PT

A software agent is a computational entity able to perceive its environment via sensors and acting in this environment through executors (Russel and Norvig, 2009). They can address complex problems such as intrusion detection in computer networks, once they have abilities such as autonomy, mobility, sociability and learning. The beliefs of a software agent are represented in a knowledge base, which includes knowledge about its environment and perception history. During the development of a software agent, it is necessary to specify its internal architecture composed of a set of components and interactions between these components, looking for satisfying functional software requirement and non-functional software requirements. A software agent architecture can be classified into three main groups: reactive, deliberative and hybrid (Russel and Norvig, 2009) (Wooldridge, 2009). A reactive architecture is ideal when an immediate action is required as the answer to a certain perception. In this type of architecture, a perception is directly mapped to an action allowing a fast decision-making process. Differently, a deliberative architecture is suitable to support more complex decisions where reasoning is performed to find the most appropriate action for a particular perception. Hybrid architectures combine reactive and deliberative behavior taking advantage of the speed of reactive behavior and the reasoning capability of the deliberative one. The agent-oriented paradigm is currently one of the most promising approaches for the development of complex systems. The agent properties such as autonomy (the ability of being able to control their own behavior and states) and learning (the capability of improving its behavior over time) turn agents an excellent software abstraction for the development of IDSs. Security mechanisms as Intrusion Detection Systems (IDSs) have been developed to address the problem of intrusions to Information Security. An IDS monitors a network or an information device in order to detect signs of intrusion (Debar, 2004) and alert the network administrator so that he/she can take 3

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN US

CR IP T

corrective actions to avoid them. Traditional IDSs still have limitations of adaptability, that is, they just detect intrusions that were previously set in system design. More flexibility is needed in order to allow IDSs to quickly react to new intrusions. An ontology is often defined as the specification of a conceptualization (Gruber, 1995). The conceptualization refers to the abstraction of a part of the world where the relevant concepts and their relationships are represented. The specification is formal and allows representation and semantic processing of knowledge. Using ontologies for intrusion detection makes possible to semantically express a taxonomy of intrusions, the relationships between the collected data, and use them to derive new information about a particular intrusion through reasoning (Undercoffer et al., 2003). An IDS can have its knowledge base structured as an application ontology that reuses concepts of the information security domain and the tasks performed by an IDS: monitor, detect and report intrusions. Ontologies are ideal for building knowledge bases due to its larger semantic expressiveness, formal description of knowledge, adaptation and integration facilities. This paper proposes a hybrid agent called HyLAA (―Case Based Reasoning Hybrid Agent‖) that combines a reactive system, responsible for reflex behaviors; a deliberative system that supports Case-Based Reasoning (CBR) and a learning system. CBR is a problem solving paradigm in which is possible to use past solutions to solve new similar problems (Aamodtand Plaza, 1994). The HyLAA knowledge base is an application ontology called ONTOID (―Application Ontology for the Development of Case-based Intrusion Detection Systems‖) (Meneses et al., 2015) representing the information security and intrusion detection systems domains. In ONTOID, each computer network intrusion is represented as a new case problem. A solution to the intrusion is represented as a case-based solution containing the corresponding preventive or corrective procedures. The learning system supports the evolution of case-based reasoning to reactive behavior, thus improving the performance of HyLAA. Four experiments to evaluate the effectiveness of HyLAA to detect intrusions using case-based reasoning behavior, the accuracy of the classifier learned by the learning component and both the performance and effectiveness of HyLAA to detect intrusions using hybrid behavior with learning and without learning were evaluated. The HyLAA agent architecture detects intrusion in a hybrid way combining reactive and deliberative behaviors. The reactive behavior is based on a signature-based approach while the deliberative one is anomaly-based. The HyLAA knowledge base is represented in an ontology making it more effective for intrusion detection than traditional intrusion detection systems due to the semantic expressiveness of ontologies. HyLAA agent architecture becomes self-adaptive to the changes in the environment through learning. This is done by automatically learning new reactive behavior by observing good recurring solutions provided by the case-based reasoning system to the same perception through supervised learning using decision trees. Thus, according to the feedback resulting from the case-based reasoning behavior, the agent turns CBR behavior into reactive one, improving its effectiveness, performance and also making it adaptable to the evolution of the environment as new intrusions are detected correctly. This feature is especially relevant in dynamic environments as computer networks where new intrusions are constantly emerging and many of them are similar and occur frequently. In addition, automatic learning of reactive behavior avoids the need of constant maintenance of the agent knowledge base. The rest of the paper is organized as follows. The second section describes main concepts about Information Security and IDSs. The third section specifies both a static and a dynamic view of the HyLAA agent architecture. The static view represents the main components of the architecture and the relationships between these components. The dynamic view shows the life cycle states of HyLAA. The fourth section describes an evaluation of the effectiveness of HyLAA to detect intrusions using case-based reasoning, the accuracy of the reactive rules learned by the learning system and the effectiveness and performance of the hybrid agent with learning and the hybrid agent without learning, respectively. The fifth section presents a comparative analysis between the HyLAA agent and other IDS approaches. Finally, the sixth section concludes the paper with a discussion of further work.

4

ACCEPTED MANUSCRIPT 2. NETWORK INTRUSION DETECTION

AN US

CR IP T

Information Security looks for protecting information from several types of intrusions and intrusions and their basic principles are availability, confidentiality, authenticity and integrity (ISO/IEC 27001, 2005). Confidentiality is the property that information cannot be made available or disclosed to individuals, entities or processes without authorization. Integrity ensures the protection of the precision and completeness of assets. Availability turns information accessible and usable upon demand by an authorized entity (ISO/IEC 27001, 2005). Another important principle that information security intends to guarantee is information authenticity. This concept is related to the property of information integration since one of the requirements for information being authentic is that third parties have not altered its content. To ensure the information security properties and prevent malicious software, computer security mechanisms were developed as encryption, digital certification, backups, anti-malware tools, firewall, and IDSs. According to (Benmoussa et al., 2014), an IDS monitors a network or a connected device in order to find the occurrence of an intrusion or malicious activity. After detecting an intrusion, the IDS can alert the network administrator so that he/she can take preventive or corrective actions or perform automatic actions such as disconnecting a particular user from the network or block certain communication ports. IDSs that monitor computer networks are called ―Network-Based Intrusion Detection System‖ (NIDS) and those that monitor only one device are called ―Host-Based Intrusion Detection System‖ (HIDS). A NIDS is usually installed in a boundary between networks and is very useful for detecting unauthorized access to the external network. A HIDS monitors the host traffic where it is installed, application logs, file access modification, operating system settings modifications, running processes and application activities. It is installed on devices connected to a network such as servers and personal computers. The main functions of an IDS are (SawantandItkar, 2014): examine the traffic of a network, identify potential events by monitoring the user and the system;



record information about the system user;



analyze configurations and system vulnerabilities;



assess the integrity of the system files;



recognize activities and typical intrusion patterns;



send alerts to the network administrator.

ED

M



CE

PT

There are two main approaches for implementing an IDS: anomaly-based and signature-based. An anomaly-based IDS identifies abnormal behavior of a computer or a network. The signature approach is based on a predefined set of intrusion signatures represented in form of detection rules (Benmoussa et al., 2014).

AC

3. A CASE-BASED HYBRID AGENT ARCHITECTURE FOR NETWORK INTRUSION DETECTION This section presents the HyLAA agent architecture, which takes advantage of the autonomy and learning agent abilities to monitor a network, detect intrusions and recommend solutions to these intrusions, working as an IDS. To guide the HyLAA development, the MADAE-Pro process (―Multi-Domain agent and Process Engineering Application‖) (Girardi and Leite, 2010) (Girardi and Leite, 2011) was applied. MADAE-Pro guides the construction of a set of models for the phases of analysis, design and implementation of a software agent. Two important MADAE-Pro models are the Concept Model and Goal Model, used respectively to represent the main concepts of the application domain and to define the multi-agent system goals. The modeling of concepts aims at performing a brainstorming of domain concepts and their relationships, representing them in a concept model. The purpose of goal modeling is to identify and

5

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN US

CR IP T

represent the functional requirements of the application, the external entities with which it cooperates and the responsibilities needed to achieve them. In Figure 1, the HyLAA Concept Model represents the main concepts of the information security domain and IDSs tasks. For example, a ―computer network‖ is susceptible to ―threats‖; ―vulnerability‖ has a ―risk‖ associated and an ―attacker‖ takes advantage of this vulnerability to perform a ―network attack‖. The basic principles of information security (availability, integrity and confidentiality) and the relationship of these principles with different classes of intrusions and IDS tasks are also specified. For example, a ―denialof-service intrusion‖ directly affects the ―availability‖ property, because the attacked system becomes inoperative for certain periods. The concepts related to the information security domain defined in this model provide the basis for creating a domain ontology and the concepts related tasks of an IDS are used to build a task ontology. Then, these two ontologies are integrated into an application ontology that will be used as an agent knowledge base. In the HyLAA Goal Model (Figure 2) the general and specific goals of HyLAA are specified as well as the responsibilities needed for achieving each specific goal and the external entities with which the agent exchanges information. For the agent to achieve its general goal (―to propose a case-based solution to information security intrusions‖), it should reach a set of specific goals by monitoring data packages that travel over the computer networks; finding a solution to these intrusions, recommending the solution to the network administrator and updating the ONTOID knowledge base. To achieve these specific goals, HyLAA must fulfill its responsibilities by analyzing packages that travel over computer networks, instantiating the ONTOID with the identified intrusions, calculating the similarity between these new identified intrusions and the ones in ONTOID and recommend solutions to the network administrator. For example, the ―System to detect threats to information security‖ have relationships ―identifies‖, ―monitors‖, ―uses‖, ―performed‖, ―provides‖ and ―analyzes‖ with ―Data package‖, ―Computer network‖, ―ONTOID‖, ―Corrective solutions‖, ―Solutions‖ and ―Monitored Package‖. Figure 3 illustrates the HyLAA a2459678gent architecture diagram. The agent architecture has two main components: the performance and the learning one. The performance component is what we have previously considered as a basic agent: it perceives and acts on the environment. The learning component is responsible for improvements in the agent behavior making possible the agent be flexible enough to respond to new previously unknown intrusions. The performance component has four subcomponents: perception interpretation, reactive system, ONTOID, and CBR system. The perception interpretation subcomponent is responsible for identifying the type of perception: a new intrusion (processed by the performance component) or feedback about the success or failure of a previously executed recommended intrusion action (processed by the learning system). ONTOID is an ontology used as the HyLAA knowledge base shared with the reactive and the CBR systems. When the perception interpretation subcomponent identifies a new perception (intrusion or normal package), it can be processed by the reactive (primarily) or CBR systems. The agent acts primarily using the reactive system because in this behavior the perception is directly mapped to an action, while the CBR system undergoes a more slow reasoning process. The ―reactive system‖ subcomponent is responsible for performing predefined behaviors. This is done by comparing the perception sentence with reactive rules in the agent knowledge base of type then . The CBR system looks for finding the most similar recommendation for a particular intrusion using case-based reasoning. The entry of the CBR system is a perception describing a new identified intrusion. From this perception, an internal representation is created, whose attribute values are compared with those of intrusions in ONTOID in order to identify a similar intrusion. When the CBR system finds a similar intrusion, this solution is reused and eventually adapted. To develop the HyLAA agent architecture first, the agent reasoning type was chosen according the application domain. The case-based reasoning was chosen because it allows to represent information that is not yet fully stable and well known. For example, in some types of intrusions are known only some of its characteristics necessary for identification. Then the supervised learning technique using decision trees was chosen because it allows the automatic generation of reactive rules preventing the effort to build them manually. 6

AN US

CR IP T

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

Figure 1. Concept Model of the HyLAA agent

Figure 2. Goal Model of the HyLAA agent

7

ACCEPTED MANUSCRIPT

CE

PT

ED

M

AN US

CR IP T

The learning system consists of the critic and training subcomponents. The critic subcomponent evaluates the agent feedback received from the perception interpretation subcomponent and informs the learning system about the success of the agent according to a fixed performance standard. The training component is responsible for making agent behavior improvements through supervised learning. It uses the feedback from the critic on the results of the agent actions and determines how the performance system should be modified to be better in the future. For this, the agent creates incrementally a training set with each pair of that meets the performance standard. When the training set has a number of examples considered sufficient for learning, the training subcomponent generates a classifier containing a set of reactive rules that is included in the ONTOID agent knowledge base and updates the behavior of the reactive system and, consequently, the overall agent performance is improved.

Figure 3. The HyLAA agent architecture

AC

Figure 4 illustrates the state diagram of HyLAA. The initial state of the agent is ―Monitoring new intrusions‖. In this state, the agent monitors the computer network waiting for new perceptions that can either be a new intrusion or the perceived result of an action performed previously on the environment, known as feedback. When the agent has the perception of a new intrusion, it enters the state ―Processing for a reactive solution‖ where it looks for a reactive rule matching the perceived intrusion. If there is a reactive solution to the new intrusion, this action is performed in the environment and the agent returns to the state ―Monitoring new intrusions‖. Otherwise, the agent enters into the state ―Processing for a case-based solution‖ where it looks for a solution based on a similar intrusion already available in ONTOID. If it finds a case-based solution, this action is performed in the environment and the agent returns to the state ―Monitoring new intrusions‖. If it does not find any solution, it also returns to state ―Monitoring new intrusions‖. When the agent has the perception of a feedback, it enters into the state ―Processing successful behaviors‖ in which the agent evaluates if the action has been successful when executed in the environment. If the evaluation result is 8

ACCEPTED MANUSCRIPT

AN US

CR IP T

positive, then the pair is saved to compose the training set that will be used for rule learning. The HyLAA agent remains in the ―Processing successful behaviors‖ state until the amount of wellevaluated actions reaches a predetermined threshold, externally defined. When this threshold is reached, the agent passes to the state ―Learning‖, where new reactive rules are learned by supervised learning and included in the ONTOID knowledge base to update the behavior of the reactive system. Then, the agent returns to the state ―Monitoring new intrusions‖.

Figure 4. The state diagram of the HyLAA agent

3.1 The Case-Based Reasoning System

PT

ED

M

CBR is one of the most popular approaches for the development of knowledge-based systems because it is a way of solving problems similarly as humans do, where the knowledge of a previous situation is reused in a new situation (Aamodtand Plaza, 1994). In this type of reasoning, the knowledge is represented by cases where each case has two parts: the problem, represented by its characteristics; and the solution, represented by a solution strategy for this problem. For example, in the IDSs domain, a case problem is a particular intrusion described through a set of attribute values in a network data package, where a case solution is an appropriate preventive or corrective recommendation for the intrusion. In CBR, when a new problem arises, it is compared with problems already existent in the CBR case base and when a similar problem is found, the solution of this problem can be reused for the new problem.

AC

CE

Figure 5 illustrates the CBR system of the HyLAA architecture and the interactions with other internal and external components. The agent is able to perceive environmental events through sensors, identifying in a particular perception a new problem to be solved which is a new intrusion in this particular domain. An internal representation of the perception is constructed as an instance named ―New Case Problem‖ (NCP) of the case problem class of ONTOID. This internal representation is used to compare and compute the similarity with already existent case problems named ―Knowledge Base Case Problems‖ (KBCP) in ONTOID, representing known intrusion cases. Section 3.1.1 details and exemplifies the NCP and KBCP concepts. A similarity measure is used by the retrieval mechanism to identify the most similar cases to the NCP. Thus, the agent should be able to select the case or a set of cases specifying problems with greater similarity to reuse the solution after possible adaptation. The adaptation mechanism uses a set of adaptation rules and domain knowledge to adapt a particular case solution. Finally, the most similar solution retrieved is executed on the environment. To learn the new solved case as a reactive rule, the learning component of HyLAA perceives the impact of this solution on the environment. If the problem was successfully solved with the applied solution in a recurrent way, the case will be learned.

9

CR IP T

ACCEPTED MANUSCRIPT

Figure 5. The CBR and the learning systems

n

AN US

To calculate the similarity between a NCP and case problems in ONTOID named KBCP the following similarity measures were used (Mendes et al., 2013):

sim_case(NCP, KBCP) = ∑w j * sim_attributej (Cncp j , Ckbcp j ) j=1

where,

(1)

M

n

∑w

j

1

ED

j1

AC

CE

PT

NCP is a new case problem; KBCP is a case already existent in the ONTOID knowledge base; j is the ordinal number associated with the jth attribute of a case problem; wj is the weight of the jth attribute; sim_attributej is the similarity measure used to calculate the similarity between the attribute values of the jth problem attribute of NCP and KBCP, as shown below:

sim_attribute (Cncp , Ckbcp ) = j j j

2 * Cncp ∩Ckbcp j j

(2)

Cncp + Ckbcp j j

Cncpj is the set formed by the NCP attribute values and its hierarchy of concepts. For example:Cncpprotocol = ―http‖ where, ―http‖is the value of the ―protocol‖ attribute of a NCP; Ckbcpj is the set formed by KBCP attribute values and its hierarchy of concepts. For example: Ckbcpprotocol = ―ftp‖ where, ―ftp‖ is the value of the ―protocol‖ attribute of a KBCP. The (1) and (2) similarity measures above were originally extracted from a similarity model, composed by a set of similarity measures, proposed in the doctoral thesis of one of the authors of this paper (Girardi, 1995) to calculate similarity between semantic cases which presented good results in the conducted experiments. Later, this model was adapted and applied to case-based reasoning in the context of the GESEC

10

ACCEPTED MANUSCRIPT group, particularly for similarity calculation between cases represented as instances of an ontology (Meneses et al., 2015). Regarding other well-known similarity measures that have been used in CBR systems, these ones were selected because, in this work, each case is represented as an instance of an ontology. Other similarity measures proposed in the literature do not support similarity between cases represented as instances of an ontology.

count duration flag

SF

1

SF

ONTOID: A Case-based ontology

M

3.1.1

Table 1. A simple example of similarity computation with a reduced amount of attributes Similarity Similarity between the values of NCP and NCP value KBCP value between NCP and KBCP KBCP attributes Formula (2) Formula (1) 0 487 385 0.66 1 0 0

AN US

Property

CR IP T

An example of similarity calculation is shown in Table 1. In the first column, part of the properties and corresponding values of a NCP are listed. The second column shows the part of the properties and values of a KBCP. The ―count‖ property represents the number of connections where source IP address and destination IP address are the same to those of the current connection in the past two seconds. The ―duration‖ property represents the connection duration in seconds and the ―flag‖ property, a connection status flag. Examples of connection status flag are S0 (―Connection attempt seen‖), S1 (―Connection established, but not terminated‖) and SF (―Normal establishment and finalization‖). The third column gives the value of similarity between the attributes of the NCP and KBCP by applying the similarity measure in (2) and considering that all attributes have the same relevance (wj = 0,33). The fourth column gives the similarity value (0.66) between the NCP and a KBCP obtained by applying the similarity measure in (1).

AC

CE

PT

ED

One of the components of the HyLAA agent is its knowledge base, representing the knowledge of the external environment, the agent perception history and the rules for mapping perceptions to actions. The HyLAA knowledge base is structured as an ontology. Ontologies are knowledge representation structures capable of expressing a set of entities in a given domain, their relationships and axioms, being used by modern knowledge-based systems as knowledge base to represent and share knowledge of a particular application domain. Compared with others knowledge representation structures, ontologies have the advantage of being reusable (Uscholdand Gruninger, 1996). An ontology can be classified according to its level of generality, such as high-level, domain, task or application ontologies (Guarino, 1998). High-level ontologies describe generic concepts that are independent of a particular domain, such as time. Domain ontologies represent explicit concepts, related to a particular domain of knowledge and the relationships between them. For example, the information security concepts can be represented in a domain ontology. Task ontologies describe activities that can be performed in one or more domains, for example, the tasks to detect intrusions and recommend corrective and preventive actions. Finally, application ontologies describe applications that specialize both domain as task ontologies, for example, an application ontology for IDS. Figure 6 illustrates the ontology hierarchy considered for the development of the ONTOID application ontology. In the HyLAA agent architecture, the knowledge base is the ONTOID ontology containing cases, adaptation rules for solutions and domain knowledge. ONTOID provides domain knowledge of information security and the tasks of an IDS.

11

CR IP T

ACCEPTED MANUSCRIPT

Figure 6. A taxonomy of ontologies and the ONTOID application ontology

PT

ED

M

AN US

Figure 7 illustrates part of this ontology. ―IntrusionCase‖ is the ontology root class which has the ―hasProblem‖ and ―hasSolution‖ non-taxonomic relationships, relating the case to the classes ―IntrusionCaseProblem‖ and ―IntrusionCaseSolution‖, respectively, representing the two parts of a case.

Figure 7. Ontology classes of the knowledge base of the HyLAA agent architecture representing network intrusion cases

AC

CE

An example of functioning of the CBR system of HyLAA is described as follows. When the agent has a new perception, it first checks if it is a new intrusion or a feedback of a previous action and then instantiates it in ONTOID. A new intrusion corresponds in case-based reasoning to a new case problem (NCP). If the perception is a new intrusion, the reactive system treats it first. However, this new intrusion could only be handled by the reactive system if it satisfies the condition of a reactive rule available in ONTOID. In other words, if it is an already known intrusion. Otherwise, the CBR system will handle the intrusion. Figure 8 shows an example of NCP (MonitoredPackage_2) that could not be handled by the reactive system because there is not a reactive rule that matches the perception. Thus, this NCP will be treated by the CBR system to find a similar problem in ONTOID, and then to reuse the associated solution to this similar problem and map it to an action.Figure 9 shows an example of a similar problem (―IntrusionProblem_1‖) to the NCP ―MonitoredPackaged2‖ (Figure 8) retrieved from ONTOID through similarity computation using the similarity measures defined in section 3.1. Thus, the corresponding solution (―IntrusionSolution_1‖) is mapped to the action ―Block directed broadcast traffic coming into network and configure hosts and routers not to respond to ICMP echo requests‖. Figure 10 illustrates the execution of the HyLAA agent for this 12

ACCEPTED MANUSCRIPT

MonitoredPackage_2 flag= S0 count= 0 duration= 0 dst_bytes= 0 dst_host_count= 255 protocol_type= ip service= ecr_i src_bytes= 520

CR IP T

example. Therefore, the agent perceives a new intrusion (NCP) (illustrated in Figure 8), detects that the KBCP related to the ―smurf‖ intrusion is the most similar case and so recommends its solution.

AN US

Figure 8. Perception of a new intrusion (NCP) instantiated in ONTOID

M

Figure 9. A similar case retrieved (KBCP) from ONTOID

ED

|?-start_HyLAA_agent. *CBR Hybrid Agent waiting new perceptions…* # Abolishing f:\ HyLAA_Agent\knowledge_base\ontoid.pl # 0.969 seconds to consult f:\ HyLAA_Agent\knowledge_base\ontoid.pl

CE

PT

New Case Problem: MonitoredPackage_2 Most similar problem: O1#IntrusionProblem_1 Intrusion type: Smurf Similarity value: 0.90 Recommended solution: *Block directed broadcast traffic coming into network *Configure hosts and routers to not respond to ICMP echo requests Figure 10. CBR action that recommends a solution to a ―Smurf‖ intrusion

AC

3.2 The Reactive System

The goal of the reactive system is to provide intelligent behavior through direct mapping of a perception to an action without any kind of reasoning. It has a knowledge base composed by a set of rules of type if ―problem‖ then ―solution‖ where a problem is a potential intrusion and the corresponding solution is a preventive or corrective recommendation to approach the intrusion. For each perception satisfying the rule condition, a corresponding rule action in the ontology knowledge base is selected and performed in the environment. For example, if the agent perceives (protocol_type = ―tcp‖) and (service = ―http‖) attribute values and there is the following rule in the knowledge base: ―if (protocol_type = ―tcp‖) and (service = ―http‖) then Intrusion is smurf‖, the action of this rule can be mapped to an agent action and executed in the environment.

13

ACCEPTED MANUSCRIPT

AN US

MonitoredPackage_1 flag= SF count= 80 duration= 0 dst_bytes= 0 dst_host_count= 255 protocol_type= icmp service= ecr_i src_bytes= 520

CR IP T

The agent acts faster using the reactive system because the perception is directly mapped to an action, while the CBR system undergoes a CBR process before performing the action in the environment. An example of the reactive system of HyLAA functioning is described as follow. Figure 11 illustrates a new instantiated intrusion containing data in a communication session. In this example, when performing the match between the new intrusion (NCP) and the reactive rule condition (Figure 12), there is an exact coincidence between the values of all attributes (protocol_type = ICMP and service = ecr_i and flag = SF), so the reactive rule action (alert ―Pod intrusion‖, ―Configure individual hosts and routers to not respond to ICMP requests or broadcasts‖) will be mapped directly to an agent action. Figure 13 illustrates the execution of the HyLAA agent when perceiving a new intrusion (shown in Figure 11), detecting it (―Pod‖ intrusion) and recommending an intrusion solution to the network administrator.

Figure 11. Perception of a new intrusion (NCP) instantiated in ONTOID

M

Figure 12. Reactive rule for a ―Pod‖ intrusion

ED

|?-start_HyLAA_agent. *CBR Hybrid Agent waiting new perceptions…*

PT

# Abolishing f:\ HyLAA_Agent\knowledge_base\ontoid.pl # 0.669 seconds to consult f:\ HyLAA_Agent\knowledge_base\ontoid.pl New Case Problem: MonitoredPackage_1 Recommended solution (reactive): *Configure individual hosts and routers to not respond to ICMP requests or broadcasts Figure 13. Reactive action to recommend solution to a ―Pod‖ intrusion

CE

3.3 The Learning System

AC

Learning turns agents the more and more effective in their behaviors making possible their adaptation to dynamic environments. Many learning techniques such as supervised learning, unsupervised learning and reinforcement learning have been applied for agent learning. Mitchell (1997) define that ―a computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E‖. In other words, a program learns if its performance improves from experience in performing a particular task. The HyLAA learning system is responsible for making improvements in the performance system, according to environment perceptions, making its behavior more effective over time. Whenever the agent has a feedback perception (the result of an action previously performed on the environment), the critic component evaluates whether the result of the action met the agent performance standard. If the action was successful, then the experience is processed in the agent training component. When the agent training set achieves a

14

ACCEPTED MANUSCRIPT

AN US

CR IP T

threshold, supervised learning is performed and a classifier is generated containing a set of reactive rules. This classifier is used to update the ONTOID knowledge base with the learned reactive rules. An example of functioning of the learning system of HyLAA is described as follows. After the execution of an action on the environment, either reactive or CBR-based, the HyLAA agent perceives a feedback with the action result which is defined by a pair where ―action‖ correspond to an action identifier and ―result‖ contains information about the execution result of this action on the environment. This feedback perception is treated by the learning system, which evaluates the level of success of a particular action according to the agent performance standard. For example, when the agent performs the recommended action <―repair network connection‖>, and perceives that the result was <―connection re-established in 45 seconds‖>, the feedback perception will be represented as [, <―connection re-established in 45 seconds‖>]. However, so far, it is unknown whether this is a good result for the agent. This assessment is made by the critic subcomponent, which compares the pair with rules that represent the agent performance standard, thus evaluating if an action on the environment was successful or not. The performance standard is a set of rules that defines if the perception feedback associated with an action is good or bad. For example, the critic subcomponent should evaluate as successful the feedback given by the pair <―repair network connection‖, ―connection re-established in 45 seconds‖> and the rules corresponding to a performance standard in Figure 14. Every time this happens, the pair is stored for training by the learning component. However, when the evaluation result is considered unsuccessful, the new case (problem and associate solution) is discarded.

M

Figure 14. Example of rules that compose a particular performance standard

ED

HyLAA continues storing the cases whose actions are considered successful by the critic subcomponent until the number of elements of the training set with positive examples reaches a certain threshold considered sufficient for learning reactive rules. When the training data set reaches this threshold, the critic subcomponent sends it to the training subcomponent. Then, the training subcomponent generates a classifier with a set of reactive rules using supervised learning.

CE

PT

Figure 15 shows an example of a set of rules generated using supervised learning-based decision trees. Finally, the training subcomponent updates the reactive system with these new learned rules. These rules turn the agent more rapid for perceptions that were previously handled through case-based reasoning. In addition, learning rules automatically eliminates the effort of creating them manually by a domain specialist. and service=http

then intrusion is land

if intrusion_category= DoS and protocol_type=ftp

and service=http

then intrusion is land

if intrusion_category= DoS and protocol_type=smtp

and service=http

then intrusion is land

if intrusion_category= DoS and protocol_type=telnet

and service=http

then intrusion is land

if intrusion_category= DoS and protocol_type=icmp

and src_bytes<=1032

then intrusion is smurf

AC

if intrusion_category= DoS and protocol_type=tcp

if intrusion_category= DoS and protocol_type=telnet and src_bytes>1032

then intrusion is pod

Figure 15. A set of reactive rules learned using supervised learning-based decision tree

15

ACCEPTED MANUSCRIPT 4. EVALUATION

AC

CE

PT

ED

M

AN US

CR IP T

This section presents an evaluation of the proposed HyLAA hybrid learning agent architecture, based on the development of four case studies. The purpose is to show that HyLAA is more effective than a reactive or deliberative agent when executed independently or than a hybrid agent that combines both behaviors without learning. Table 2 shows the organization of the four case studies, including the purpose of each experiment, the method used in its implementation and the expected results. The first experiment evaluates the effectiveness of the CBR agent component when running independently. This component corresponds to a deliberative agent with analogical reasoning without learning. The effectiveness evaluation results show the agent ability to recommend relevant solutions for a given intrusion as well as the ability to hide possible solutions that are irrelevant based on the ―recall‖ and ―precision‖ effectiveness metrics (Salton et. al, 1993) for a set of intrusions. This experiment was divided in two parts. The first part evaluates the effectiveness of the agent by considering a uniform distribution of the weights of the attributes considered in the similarity computation; the second part evaluates the impact on effectiveness when using heavier weights for the most important attributes. The second experiment evaluates the accuracy of the learning agent component, that is, the ability that the classifier learned by the learning component has to correctly classify a given intrusion using the accuracy metric (Sokolova and Lapalme, 2009). A classifier learned by the learning component consisting of a set of reactive rules is incorporated into the agent knowledge base, thus updating its reactive behavior. This experiment was organized in three parts, each one using a different training set for the learned classifier looking for measuring the impact of the training set on the accuracy of the classifier. The third experiment evaluates the effectiveness and performance of the hybrid agent as it learns. The fourth experiment evaluates the effectiveness and performance of the hybrid agent without learning, that is, the one that just combines CBR and reactive behaviors. Effectiveness evaluation is based on the ―recall and ―precision‖ metrics (Salton et. al, 1993). To evaluate agent performance, response time was used. Software response time is the amount of time spent from the point at which a user-made request is submitted until it is answered by the system (Molyneaux, 2014). For the HyLAA agent, response time is the length of time between a new perception and a solution recommendation. To measure the response time of the agent, a ―statistics‖ predicate from LPA Prolog was used that provides the time spent by the agent to process a given perception. The computer that was used to perform the experiments presented in this section is the same of the previous experiments (Intel Core i5 2.70 GHz with 8 GB of RAM) to ensure that the computer configuration has no impact on the evaluation results. This experiment was divided in two parts. The first part looks for comparing the effectiveness of the hybrid without learning agent running only the CBR component. The second part has the goal of comparing the effectiveness and performance of the hybrid agent with learning against the ones of the agent without learning. Finally, the results obtained in the different case studies are comparatively analyzed, showing the gains in terms of effectiveness and performance with and without learning (third and fourth experiment, respectively), and also, the gains in terms of effectiveness, performance, development effort and adaptability of the hybrid agent with learning compared to those of the hybrid agent without learning. The validity of an experiment is related to the level of confidence with which it was conducted. During the development of the four experiments carried out in this evaluation some guidelines looking to ensure the reliability of experiments were considered. Normally, the validity of the research focuses on internal and external aspects. The external aspects refer to the capacity of the experiment to be generalized, for example, to other domains using other data and so on. The internal aspects relate to the quality of the experiment to reflect a real result. For example, for validation of the internal aspects should be possible to replicate the experiments. A discussion of the validity in experiments and threats to validity in research design can be found at (Yu and Ohlund, 2016). During the development of the experiments showed in this section, precautions to guarantee the reliability of experiments were taken. For example, data was randomly selected and the experiments can be replicated.

16

ACCEPTED MANUSCRIPT The agent was implemented in Prolog using Chimera, a tool for building agents integrated into the LPA-Prolog environment (Steel, 2015). The agent knowledge base, the ONTOID ontology described in section 3.1.1, was originally implemented in OWL using the Protégé ontology editor (Gennari et al., 2003) and then mapped to the Prolog language using the Thea tool (Vassiliadis et al., 2005). NSL-KDD (Olusola et al., 2010) is a set of intrusion simulation data, which includes the data of a particular network connection session, corresponding to a particular type of intrusion used both as agent perception and case-problems stored in ONTOID knowledge base.

To evaluate the accuracy of the learned classifier

To evaluate the effectiveness and performance of the hybrid agent with learning

Experiment 4

To evaluate the effectiveness and performance of the hybrid agent without learning

Run the hybrid agent and evaluate its effectiveness as it learns, that is, when every new set of reactive rules is learned. (section 4.3)

Executes of the hybrid agent without learning and compare with the CBR and reactive components when executed independently (section 4.4)

ED

Experiment 3

Expected Results Good effectiveness of the CBR component, i.e., ability to recommend relevant solutions and to hide irrelevant solutions

CR IP T

Experiment 2

To evaluate the effectiveness of the CBR component in terms of the ―precision‖ and ―recall‖ measures

AN US

Experiment 1

Table 2. Organization of the HyLAA evaluation Method This experiment was divided in two parts (section 4.1) using a test set of 50 perceptions and a knowledge base instantiated with 11,000 cases of intrusions. Part 1: Uses different cut-offs points (minimum similarity values). Part 2: Uses heavier weights for the most important attributes. This experiment was divided in three parts (section 4.2) using a test set of 50 perception. Part 1: A training set with 150 examples. Part 2: A training set with 500 examples. Part 3: A training set with 1000 examples.

M

Goal

Good ability to correctly classify an intrusion (high accuracy of the learned classifier)

Greater effectiveness of the hybrid behavior with learning than the CBR or reactive ones when executed independently; Better effectiveness when the agent learns Greater effectiveness and performance of the hybrid behavior without learning than the CBR or reactive behavior when executed independently

PT

4.1 Experiment 1: CBR System Evaluation

AC

CE

This section describes the experiment used to evaluate the effectiveness of the CBR component. The objective of this study is to evaluate the CBR component ability to recommend relevant solutions for a given intrusion as well the ability to hide possible solutions that are irrelevant, based on measurements of values of ―recall‖ and ―precision‖ for a set of intrusions. For this, the CBR component performs the following actions on the environment, as illustrated in Figure 2: ―Monitor the computer network‖, where the agent monitors the network awaiting the arrival of new data packets relating to a connection session; ―Detecting attacks‖, where the agent analyzes the data packages perceived in the previous action that are related to an intrusion or a normal connection; and ―Deliver solution to identified attacks‖, in which the agent suggests a solution to the network administrator that addresses an intrusion identified in a previously performed action. Table 3 summarizes the two parts of this experiment, including the goals, method and expected results.

17

ACCEPTED MANUSCRIPT

Table 3. Experiment 1 organization

Goals

1

Part 2

To evaluate the effectiveness of the CBR component considering different minimum similarity values (cut-offs) To analyze the impact on the effectiveness of the recommendations by attributing greater weight to the most relevant attributes

Identify the most relevant attributes and assign greater weight to them. Compare the results with the ones of part 1

Expected Results Good effectiveness for some cut-offs Good effectiveness using heavier weights for the most important attributes

CR IP T

Part

Method Use different cut-off points to calculate the similarity to evaluate the effectiveness of ―recall‖ and ―precision‖ measures

The presentation of this experiment is organized into three sections. Section 4.1.1 describes the method used to organize and perform the experiment. Section 4.1.2 presents the results. Finally, Section 4.1.3 analyzes the results according to the evaluation goals.

AN US

4.1.1 Method

AC

CE

PT

ED

M

The effectiveness of the solutions recommended by the agent was evaluated in terms of the traditional evaluation measures in the area of information retrieval, ―recall‖ and ―precision‖ (Salton et. al, 1993). ―Recall‖ is the ratio of the number of retrieved relevant solutions and the total number of relevant solutions in the knowledge base, and ―precision‖ is the ratio of the number of retrieved relevant solutions and the total number of cases retrieved. To analyze the results, a graph of ―precision-recall‖, which shows the evolution of the values of ―precision‖ on the amounts of ―recall‖, was built. In this graph, the values of ―recall‖ are listed on the X axis, and the ―precision‖ values are shown on the Y axis. Usually, a good effectiveness is considered achieved when ―precision‖ is not compromised for high values of ―recall‖; that is, even if the agent increases the amount of recommended solutions to the user, they continue to be relevant for the most part. The first part of this experiment was developed to determine the cut-off point for good effectiveness by balancing the values of ―recall‖ and ―precision‖ so that high levels of ―recall‖ do not compromise the ―precision‖ of the system. A cut-off point sets the minimum similarity value necessary for at least one solution to be retrieved by the agent. It is necessary to identify the ideal cut-off point because when using cut-offs with very high values, it is not possible for the agent to retrieve a solution for a given intrusion. On the other hand, very low cut-offs may cause the agent to recommend solutions with very low similarity values, making them irrelevant. Thus, this first experiment aims at finding the optimal cut-off, that is, the largest possible cut-off point that returns solutions with high values of ―precision‖ for most of the identified intrusions. The cut-off points used in this experiment were 0.7, 0.8 and 0.9. These values were chosen based on previous tests in which many irrelevant solutions were obtained with cut-off points lower than 0.7 and many relevant solutions were not retrieved with cut-offs higher than 0.9. In the second experiment, the same cut-off points from the previous experiment were used, but larger weights were considered for the most relevant attributes to calculate similarity because the intrusion of a network has certain characteristics (attributes) that distinguish it from others intrusion types. Thus, in the second part of this experiment, the importance of attributes that identify an intrusion is determined; higher weights are assigned to attributes considered the most relevant; and the results indicating that this is preferable to assigning equal weights are verified. In this experiment, the most relevant attributes were selected according to Olusola et al. work (Olusola et al., 2010). To simulate agent perceptions, the same set of 50 instances drawn randomly from the NSL-KDD data and 11,800 instances used to populate the knowledge base representing 37 different types of intrusions

18

ACCEPTED MANUSCRIPT were used. Each example has a set of 41 attribute values that characterize an intrusion. The attributes contain information on the communication protocol used for a connection session, the amount of transferred bytes, the service used, etc. The 50 perceptions correspond to 10 types of intrusions (―neptune‖, ―guess_password‖, ―warezmaster‖, ―satan‖, ―smurf‖, ―back‖, ―portsweep‖, ―ipsweep‖, ―nmap‖, ―pod‖), with five perceptions for each type of intrusion. These intrusions were selected according to the amount of available instances in the knowledge base and the data available in the literature on the characteristics that distinguish each intrusion, i.e., the most relevant attributes. Table 4 shows the number of instances available in the ONTOID for each of the 10 types of intrusion used as a perception. Table 5 presents the most relevant attributes for each of the 10 types of intrusion used in the agent perceptions and the weight assigned to these attributes. Table 4. Set of instances of ONTOID categorized by type of intrusion

AN US

CR IP T

Intrusion name Number of instances in ONTOID neptune 1579 guess_passwd 1226 warezmaster 944 satan 727 smurf 627 back 359 portsweep 156 ipsweep 141 nmap 73 pod 41 Other (land, worm, imap, etc.) 5931

M

Table 5. Set of the most relevant attributes and their weights Relevant attributes

Intrusion name

Weight Relevant attributes

Other attributes

0,0321

0,0203

0,0370

0,0237

warezmaster

dst_bytes, duration

0,0350

0,0238

diff_srv_rate, rerror_rate, service diff_srv_rate, dst_bytes, st_host_count, dst_host_same_src_port_rate, dst_host_srv_serror_rate, logged_in, protocol_type, same_srv_rate,serror_rate,service,src_bytes dst_bytes, src_bytes

0,0333

0,0236

0,0333

0,0206

0,0350

0,0238

0,0300

0,0242

smurf

AC

back

PT

satan

CE

neptune

ED

guess_passwd

count,diff_srv_rate,dst_host_count, dst_host_same_src_port_rate, dst_host_same_srv_rate, dst_host_serror_rate, dst_host_srv_serror_rate, flag, same_srv_rate, src_bytes, srv_diff_host_rate, srv_serror_rate service,flag,dst_bytes, num_failed_logins

portsweep

srv_rerror_rate

ipsweep

dst_host_same_src_port_rate

nmap

src_bytes

pod

wrong_fragment Other intrusions

0,02439

4.1.2 Results Table 6 shows the effectiveness of the CBR component in terms of the ―precision‖ and ―recall‖ measures. It is observed that in both parts of this experiment, using the 0.7 cut-off point gave values of

19

ACCEPTED MANUSCRIPT

CR IP T

―precision‖ that were very low (an average of 21% and 24% in the first and second part of this experiment, respectively). Thus, the system has retrieved a number of irrelevant solutions, while the values of ―recall‖ were good (approximately 78% and 76% in the first and second part of this experiment, respectively). A cutoff point of 0.9 for ―precision‖ had the best result among the three cut-off points used (93% and 94% in the first and second part of this experiment, respectively), but the ―recall‖ was the lowest (29% and 30 % in the first and second part of this experiment, respectively). With the 0.8 cut-off point, it can be seen that both ―precision‖ (61% and 63% in the first and second part of this experiment, respectively) as well as ―recall‖ (63% and 60% in the first and second part of this experiment, respectively) were more balanced. In the second part of this experiment, higher values of ―precision‖ were obtained for all the cut-offs compared to the values obtained in the first experiment; in addition, higher values of ―recall‖ for 2 of the 3 cut-offs were obtained, as shown in Table 6. However, the difference between these values was small. One of the factors that contributed to this was the fact that many of the NSL-KDD intrusions have many features in common; in some cases, only one attribute differentiates one intrusion from another; then, even after the weights of its attributes are increased, the result of a similarity calculation remains very close. Figure 16 shows the ―recallprecision‖ graph for the two experiments for the 0.9 cut-off point. Table 6. Set of the most relevant attributes and their weights Exp. 1 (Uniform weight distribution) Exp. 2 (Heavier weights for more relevant attributes) Precision

Recall

Cut-off

0,7

21%

78%

0,7

0,8 0,9

61% 93%

63% 29%

Precision

Recall

24%

76%

63% 94%

60% 30%

AN US

Cut-off

CE

PT

ED

M

0,8 0,9

Figure 16. Recall-precision graph for the 0.9 cut-off point

AC

4.1.3 Discussion

When recommending solutions to intrusions in computer networks it is important to ensure high values of ―precision‖ regardless of the values of ―recall‖ since it is advantageous to recommend a few good solutions to the user, thus avoiding trial-and-error solutions that are more costly in terms of both effort and time spent, critical factors in information security. Although the values obtained in the two parts of this experiment were very similar, as shown in Table 6, both results of ―precision‖ and ―recall‖ obtained in the second experiment were superior to the former. However, in both experiments, according to the graph of Figure 16, there was a sharp drop in the ―precision‖ at certain points. This occurs when the number of relevant cases of a particular intrusion stored in

20

ACCEPTED MANUSCRIPT the knowledge base is reduced. For example, this occurs in the intrusions ―Pod‖, ―Ipsweep‖ and ―Portsweep‖, which have only 41, 141 and 156 bodies in the knowledge base, respectively, as can be seen in Table 4. Given the results obtained in the two experiments, it can be concluded that the HyLAA agent creates solutions with effective recommendations using the 0.9 cut-off because – as shown in Figure 16 – even when the value of ―recall‖ increases, ―precision‖ remains high; that is, even when the amount of recommended solutions increases, they remain relevant solutions. In addition, it is clear that assigning greater weight to the most relevant attributes did not represent a significant improvement in the values of ―precision‖ and ―recall‖. For example, as can be seen in the graph of Figure 16, using 0.9 as a cut-off point, the results of the part 2 experiment were superior to those of part 1 in only 1% of the ―precision‖ and ―recall‖ values.

CR IP T

4.2 Experiment 2: Learning System Evaluation

AN US

This second experiment is used to evaluate the accuracy of the learning system. The learned rules are expected to have greater accuracy, that is, a high rate of correct classification of examples, because they have a direct impact on the overall effectiveness of the hybrid agent that is developed, as it uses the reactive behavior primarily. Thus, the hybrid agent tries to detect intrusions reactively due to the higher performance of this type of behavior, and when there is no reactive rule, it acts in the deliberative way. Three experiments were performed, one for each training set, using 150, 500 and 1000 samples as described in Table 7 and a set of 50 perceptions for testing. This experiment was divided in three parts conducted with the purpose of analyzing the relationship between the training set size and the accuracy of the classifier. Table 7. Experiment 2 organization Method 150 examples for training and 50 Evaluate the accuracy of the samples for testing classifier with a set of 150 training Evaluate the accuracy of the classifier examples Analyze the results 500 examples for training and 50 Evaluate the accuracy of the examples for testing classifier with a set of 500 training Evaluate the accuracy of the classifier examples Analyze the results 1000 examples for training and 50 Evaluate the accuracy of the examples for testing classifier with a set of 1000 Evaluate the accuracy of the classifier training examples Analyze and compare the results

Part 2

Classifier with high accuracy

PT

Part 3

Expected Results

ED

Part 1

M

Goals

CE

In subsection 4.2.1, the evaluation method used for this new experiment is described. In subsection 4.2.2, the results obtained from the experiment are shown. Finally, in subsection 4.2.3, the results are analyzed.

AC

4.2.1 Method

To perform this experiment, the Holdout method (Kohavi, 1995) was adopted, where the set of available examples is divided in two parts: the training set and the test set. The training set is a collection of examples used to build a classifier, and the test set is used to evaluate the accuracy of the classifier. The accuracy metric (Sokolova and Lapalme, 2009) is used to evaluate the overall effectiveness of a classifier and takes into account the number of examples corresponding to an intrusion correctly classified in a given type of intrusion (―true positive‖); the number of examples that do not correspond to an intrusion (a normal connection session) correctly classified (―true negative‖); the number of examples that correspond to an intrusion incorrectly classified as non-intrusion (―false negative‖); and the number of examples that

21

ACCEPTED MANUSCRIPT correspond to an intrusion and were not classified in a given type of intrusion (―false positive‖). Thus, the accuracy is calculated by the following formula: (3) where, tp (―true positive‖), is the number of examples correctly classified in a given type of intrusion; tn (―true negative‖), is the number of examples correctly classified as non-intrusion; fp (―false positive‖), is the number of examples incorrectly classified in a given type of intrusion; fn (―false negative‖), is the number of examples incorrectly classified as non-intrusion.

CR IP T

Table 8 shows how the three parts of this experiment are organized. To construct the classifier, three different training sets were used containing 150, 500 and 1000 examples. For the test, a set of 50 perceptions (the same as the one used in experiment 1 for comparison purposes) were used. The adopted learning technique was supervised learning using decision trees and the C4.5 algorithm (Quinlan, 1986). Table 8. Method organization of the experiment 3 Training set

Learning technique

50 examples (taken from NSL-KDD, the same as in experiment 1)

150 training examples (randomly obtained from NSL-KDD) 500 training examples (randomly obtained from NSL-KDD) 1000 training examples (Randomly obtained from NSL-KDD)

Learning supervised using decision trees

Part 1

Part 2

Part 3

4.2.1.1 The learning algorithm

Learning algorithm

C4.5

M

Experiment

AN US

Perception (Test set)

PT

ED

The learning technique adopted in this experiment was inductive learning, whereby a set of generic rules examples were obtained. The examples are labeled and act as a ―teacher‖ learning algorithm for informing the class that corresponds to each example. In Figure 17, the example of the first row of the table <13, tcp, telnet, SF 118> is an intrusion of type ―guess_password‖. Each line contains an example of a particular type of intrusion. The set of all examples are used to learn classification rules.

CE

To accomplish the learning of new reactive rules, the C4.5 decision tree algorithm proposed by Ross Quinlan (Quilan, 1986) was adopted. Decision trees are supervised machine learning techniques, in which a decision tree is created for the discovery of classification standards. Thus, the classifier is a decision tree that representing a set of rules for classification.

AC

In C4.5, a tree is created from a set of labeled training examples. Each internal node of the tree corresponds to a test of the value of an attribute; the branches of the nodes are labeled with the possible results of the test, and the leaves have the labeled values of the class attribute (the attribute to predict). Each path between the root and a sheet is a standard or classification rule. In a decision tree, the nodes represent the names of attributes; arcs represent the values of these attributes, and leaves represent the class they belong to for each example. As shown in Figure 17, the training data set consists of a set of examples consisting of attributes and values and an associated class. In the example, the values of attributes are given in a communication session, and the class corresponding to a type of intrusion is characterized in that set values. The C4.5 algorithm builds a decision tree from these data. The decision tree can be represented by rules of type ―if then ‖. The C4.5 algorithm builds a decision tree in a ―top-down‖ approach that considers which attribute is most suitable to be the root of the tree node, that is, which attribute is more significant. More information about the C4.5 algorithm can be found in Quinlan work (Quinlan, 2014).

22

ACCEPTED MANUSCRIPT

AN US

CR IP T

.

Figure 17. Example of supervised learning using decision trees

M

4.2.2 Results

PT

ED

Table 9 shows the results obtained with this second experiment. In the first column, the parts of the experiment are listed. The second column indicates how many perceptions were used as a test. In the third column, the number of training examples used in each experiment is shown. The fourth column reports the number of intrusions classified correctly (―true positive‖ and ―true negative‖), and the fifth column shows the number of misclassified perceptions (―false positive‖ and ―false negative‖). In the last column, the accuracy value is shown, calculated according the formula in (3). Table 9. Accuracy values resulting from this experiment

Test Set

CE

Experiments

AC

Part 1 Part 2 Part 3

50 examples

Training Set 150 500 exemplos 1000 exemplos exemplos

Correctly Classified 37 38 39

Incorrectly Classified 13 12 11

Accuracy 74% 76% 78%

4.2.3 Discussion

The results of this experiment showed that, as expected, the higher the training set, the higher the accuracy of the learned rules. Thus, for the agent to be effective, a minimum set of examples is needed. The test set size should be determined according to the effectiveness requirements of the agent being developed. The training sets used, consisting of 150, 500 and 1000 samples, were independent because the goal of these experiments was to evaluate the variation in effectiveness according to the training set size.

23

ACCEPTED MANUSCRIPT However, in a real application, the HyLAA agent should increase the amount of training examples and constantly update its knowledge base as new reactive rules are learned. Thus, in the fourth experiment, the agent learning component is evaluated from this perspective. Due to the automatic learning of reactive rules through supervised learning, the manual effort of inserting reactive rules in the agent knowledge base is avoided as well as the need for an expert on the domain to update the knowledge base with new reactive rules. 4.3 Experiment 3: Evaluation of the hybrid agent with learning

AN US

CR IP T

The goal of this experiment was to evaluate the evolution of effectiveness and performance values of the hybrid agent as it learns. Through its learning mechanism, the hybrid agent adapts to its environment and identifies new intrusions not previously defined in the agent design phase. This is done by inducing reactive rules considering the feedback of the recommendations made by the agent through its CBR component. In Table 10, the goal, method and expected results of this experiment are presented. The agent handles 500 perceptions representing 10 different types of intrusions, using the ONTOID knowledge base instantiated with 11,000 cases corresponding to CBR intrusions without any predefined reactive rule. The effectiveness is evaluated in terms of the ―precision‖ and ―recall‖ measures, already introduced along the description of Experiment 1. Table 10. Experiment 3 organization

Method

To evaluate the evolution in effectiveness and performance of the hybrid agent behavior as it learns new reactive behavior

Using 500 perceptions, a knowledge base instantiated with 11,000 cases of intrusions and without any pre-defined reactive rule

Expected Results Most effective and best performance of the agent hybrid behavior of the agent as it learns new reactive behavior. Adaptability of the agent to its environment over time

ED

4.3.1 Method

M

Goals

AC

CE

PT

In this experiment, the effectiveness and performance of the agent hybrid behavior evolves as it learns new reactive behavior and effectiveness gains when using the last set of rules learned compared to the intermediate results. In this experiment, for every 100 new perceptions, the agent learns a new set of reactive rules using these perceptions as a training set. The reactive rules learned always replace the previous rules, but the training set is maintained and incremented every 100 newly perceived intrusions. Thus, for the first classifier, a set of 100 examples is used; for the second, a set of 200 examples and so on. Each set of 100 new perceptions used for learning is called a learning threshold. At the moment when the number of samples set by a learning threshold is reached (100 new samples), new reactive rules are learned by replacing the previous rules, and the effectiveness of the agent is evaluated in terms of ―recall‖ and ―precision‖ measures. To evaluate the agent performance, response time was used. Software response time is the amount of time spent from the point at which a user-made request is submitted until it is answered by the system (Molyneaux, 2014). For the HyLAA agent, the response time is the length of time between a new perception and a solution recommendation. To measure the response time of the agent, the ―statistics‖ predicate of LPA Prolog was used providing the time spent by the agent to process a given perception. The computer that was used to perform the experiments presented in this section is the same of the previous experiments (Intel Core i5 2.70 GHz with 8 GB of RAM) to ensure that the computer configuration has no impact on the evaluation results.

24

ACCEPTED MANUSCRIPT 4.3.2 Results

ED

M

AN US

CR IP T

In this experiment, the set of 500 perceptions used in both experiments as a test case contains examples representing ten different types of intrusions, distributed proportionally according to the number of each type of intrusion available in ONTOID. Initially, when the agent runs, it does not have knowledge supporting the reactive behavior, therefore, it detects intrusions only through case-based reasoning. The result of the effectiveness of the agent in detecting and responding to the first hundred intrusions is shown in the second column and fifth row of Table 11: 0.09 average for ―precision‖ (AP) and 0.0002 average for ―recall‖ (AR). In the sixth line the average agent response time for the first hundred intrusions (40,000 seconds) is shown. Within the first learning threshold, the agent learned 30 new reactive rules, thus becoming hybrid. The effectiveness of the hybrid agent within the first learning threshold is shown in the third column and fifth row of Table 11: 53% on average for both ―precision‖ as ―recall‖. The sixth line shows the average response time within this learning threshold 1,680 seconds. In the second learning threshold, the agent learned more 74 reactive rules that replaced the initial 30 reactive rules. The effectiveness of the agent in this threshold is shown in the fourth column and fifth row of the Table 11: an average of 68% for both ―precision‖ and ―recall‖. In the sixth line is shown the corresponding average response time of the agent within this learning threshold (865 seconds). In the third learning threshold, the agent learned 81 new reactive rules that replaced the 74 reactive rules of the learning from the second threshold. The result of the effectiveness of the hybrid agent in this threshold is shown in the fifth column and fifth row of Table 11: average of 76% of both ―precision‖ and ―recall‖. The sixth line shows the average agent response time for this threshold: 500 seconds. In the fourth threshold, the agent also learned 81 reactive rules using a training set containing 400 previous perceptions that replaced the 81 previously learned reactive rules. The effectiveness of the agent running on the 400 previous perceptions in addition to the 100 new perceptions using the set of 81 reactive rules is shown in the sixth column and fifth row of Table 11: 81% average for both ―precision‖ and ―recall‖. The sixth line shows the agent response time in detecting the 100 intrusions used in the fourth learning threshold 500 seconds. Table 11. Evaluation results of the effectiveness of the hybrid agent considering different learning thresholds

Perceptions

100

AC

CE

Training set Classifier learned Effectiveness (average “precision” and “recall”) Average response time

No learning

PT

Threshold

-

First threshold learning

Second threshold learning

Third threshold learning

Fourth threshold learning

100+100

200+100

300+100

400+100

100 30 rules

200 74 rules

300 81 rules

400 81 rules

AP

AR

AP

AR

AP

AR

AP

AR

AP

AR

0,09

0,0002

0,53

0,53

0,68

0,68

0,76

0,76

0,81

0,81

40.000 seconds

1.680 seconds

895 seconds

500 seconds

500 seconds

4.3.3 Discussion This experiment evaluated the evolution of the effectiveness and response time as new reactive behavior was learned by the agent. The agent had a very low average ―precision‖ and ―recall‖ before the agent became hybrid; of the 100 perceptions, only nine were detected correctly (Table 11). However, from

25

ACCEPTED MANUSCRIPT

CR IP T

the first learning threshold (third column), where a set of 30 rules were learned reactively, both the average ―precision‖ and ―recall‖ gradually increased. In the last threshold, it is clear that there was a great improvement of ―recall‖ and ―precision‖. The hybrid agent response time was also improved gradually, when it learned new reactive behavior. Thus, based on the results of this experiment, it was observed that the hybrid agent improved the effectiveness of its behavior and response time; new reactive rules were learned and became more adaptive to its environment. The results of the first experiment showed that the effectiveness of the agent and the response time increase gradually as it learns new reactive behavior, as it can be seen in Table 11. However, some factors such as the quality of data (inconsistencies, duplicate data, etc.) and the algorithm used can affect these results. In this experiment, it is possible to observe the relationship between the effectiveness of the agent and the training set used for learning reactive rules. For example, when using a set of 400 training examples, the agent learned 81 rules and obtained results that were superior to those of other learned classifiers using smaller training sets, as with 100, 200, and 300 examples. 4.4 Experiment 4: Evaluation of the hybrid agent without learning

M

AN US

This experiment evaluates the effectiveness and performance of the hybrid agent without learning, as shown in Table 12. The first part of this experiment evaluates the effectiveness in terms of the ―precision‖ and ―recall‖ measures and the performance of the hybrid agent without learning through response time. The agent processes the same 50 perceptions of the experiment 1 using the knowledge base ONTOID instantiated with 11,000 cases of intrusion and a set of rules inserted manually in the knowledge base. The second experiment evaluated the effectiveness of the hybrid without a learning agent using the same test data from experiment 3. In this experiment, the effectiveness is evaluated every 100 newly processed perceptions. Table 12. Experiment 4 organization

To evaluate the effectiveness and performance of the hybrid agent without learning with the same test data of Experiment 1

Running the hybrid agent in the environment with the same 50 perceptions from experiment 1 using three different sets of reactive rules capable of detecting 2, 6 and 10 different types of intrusions

CE

PT

Part 1

Method

ED

Goals

AC

To evaluate the effectiveness of the hybrid agent without learning with the same test data of the Experiment 3

Part 2

Running the hybrid agent in the environment with the same 500 perceptions from experiment 3 using three different sets of reactive rules capable of detecting 2, 6 and 10 different types of intrusions

Expected Results Greater effectiveness and better performance when increasing the amount of reactive rules for detecting various types of intrusion  Greater effectiveness when increasing the amount of reactive rules for detecting various types of intrusions  Less evolution in effectiveness over time than the agent with learning, evaluated in Experiment 3

26

ACCEPTED MANUSCRIPT 4.4.1 Method

AN US

CR IP T

In the first part of this experiment, the hybrid agent without learning was evaluated based on the same test data of Experiment 1. In addition, the knowledge base ONTOID was instantiated with 11,000 cases of intrusions using the 0.9 cut-off point, and three sets of reactive rules were manually inserted into the knowledge base. The first set of reactive rules was able to detect two types of intrusions among the 10 types of intrusions used for the set of perceptions; the second set of reactive rules was able to detect six types of intrusions, and the last set of reactive rules was able to detect all ten types of intrusions. Effectiveness was evaluated in terms of ―precision‖ and ―recall‖, and the agent response time was measured in the same manner as in Experiment 3. In the second part of this experiment, the evaluation of the hybrid agent without learning was conduced by executing the agent using the same 500 perceptions considered in Experiment 3 and an instantiated knowledge base with 11,000 cases of intrusion; 0.9 was used as cut-off point, and three sets of reactive rules were manually inserted into the knowledge base. As in the first part of this experiment, the first set of learned rules was able to detect two types of intrusions, the second 6 types of intrusion and the last 10 types of intrusion among the 10 types of intrusion used in the test set. In this experiment, the agent effectiveness was evaluated incrementally every 100, 200, 300, 400 and 500 recommendations. The goal was to analyze how effectiveness evolves over time. 4.4.2 Results

ED

M

Table 13 shows the results of the first part of this experiment. The second column indicates the number of perceptions, noting that the examples were distributed in equal amounts among the ten types of intrusions used in the first experiment. The third column tells which types of intrusions each set of reactive rules can detect. Thus, the intrusions that the reactive rules were not able to detect were handled by the CBR component. The next two columns report the ―precision‖ and ―recall‖ averages obtained by detecting the same 50 perceptions of Experiment. The last column indicates the amount of time that was necessary to detect 50 intrusions and to recommend a response to the network administrator.

PT

AC

CE

Part 1

Table 13. Results of the first part of this experiment using different sets of reactive rules and 50 perceptions Precision Recall Perceptions Reactive Rules in ONTOID Response Time Average Average Set of reactive rules able to detect 0.95 0.49 80000 seconds two different types of intrusion The same 50 Set of reactive rules able to detect perceptions of 0.98 0.85 40000 seconds six different types of intrusion Experiment 1 Set of reactive rules able to detect 1 1 5 seconds ten different types of intrusion

Tables 14 to 16 show the average ―precision‖ (AP) and average ―recall‖ (AR) for 100, 200, 300, 400 and 500 perceptions. Table 14 shows the average ―precision‖ and ―recall‖ obtained when the set of reactive rules can only detect two types of intrusions (―Apache‖ and ―guess_password‖); and other types of intrusions are detected by the CBR component. Table 15 indicates the average ―precision‖ and ―recall‖ obtained when the number of reactive rules was able to detect only six types of intrusions (―apache‖, ―guess_password‖ ―Warezmaster‖, ―satan‖, ―smurf‖ and ―back‖). Table 16 shows the average of ―precision‖ and ―recall‖ obtained when the number of reactive rules was able to detect all ten types of intrusions (―apache‖, ―guess_password‖, ―warezmaster‖, ―satan‖, "smurf", "back", "portsweep", "ipsweep", "nmap" and "pod").

27

ACCEPTED MANUSCRIPT Table 14. Results of the evaluation of the hybrid agent without learning using a set of reactive rules representing two types of intrusions

Number of Perceptions

100

Part 2

100+100

200+100

300+100

400+100

AP

AR

AP

AR

AP

AR

AP

AR

AP

AR

0.32

0.32

0.32

0.32

0.45

0.35

0.42

0.35

0.51

0.35

Table 15. Results of the evaluation of the hybrid agent without learning using a set of reactive rules representing six types of intrusions

Part 2

100 AP 0.84

100+100 AR 0.84

AP 0.90

AR 0.90

200+100 AP 0.92

300+100

AR 0.92

AP 0.93

AR 0.93

400+100 AP 0.94

CR IP T

Number of Perceptions

AR 0.94

Table 16. Results of the evaluation of the hybrid agent without learning using a set of reactive rules representing ten types of intrusions

Part 2

100 AP 0.97

100+100 AR 0.97

AP 0.97

4.4.3 Discussion

AR 0.97

200+100 AP 0.98

AR 0.98

300+100

AP 0.98

AN US

Number of Perceptions

AR 0.98

400+100

AP 0.98

AR 0.98

AC

CE

PT

ED

M

As expected, the results of the first part of this experiment (Table 13) showed that hybrid agent behavior even without learning improves both effectiveness and response time when increasing manually the number of reactive rules that can detect a greater number of different types of intrusions. Effectiveness is improved mainly in terms of ―recall‖ average: from 0.49 for a set of reactive rules able to detect two different types of intrusions to 1 for a set of reactive rules able to detect ten different types of intrusions. This occurs because, when the number of reactive rules that can detect a greater number of different types of intrusions is increased, the accuracy of the set of rules also increases; that is, the hybrid agent can detect a greater number of intrusions correctly. Response time also improves considerably: from 8000 seconds for a set of reactive rules able to detect two different types of intrusions to 5 seconds for a set of reactive rules able to detect ten different types of intrusions. This occurs because reactive behavior is quicker and executes primarily than CBR. Thus, with an increase in the number of reactive rules that can detect a greater number of different types of intrusions, a larger number of intrusion types that were previously detected by the CBR behavior when executed independently without hybrid behavior, can be faster detected reactively by the hybrid agent. Also as expected and considering that reactive rules are predefined manually in the agent design phase, the results of the second part of this experiment showed that there was less change over time in the agent without learning effectiveness both in terms of ―recall‖ and ―precision‖ values. That is, ―recall‖ and ―precision‖ values remain similar when effectiveness is measured after the processing of 100, 200, 300, 400 and 500 intrusions and this behavior pattern is independent of the amount of the set of reactive rules able to detect different types of intrusions (Tables 14, 15 and 16). However, this behavior pattern is different when compared with the hybrid agent with learning as shown in Table 11 where effectiveness is improved over time as the agent learns. For example, the hybrid agent without learning exhibited variation in effectiveness of only 1% when using the set of reactive rules able to detect all 10 types of intrusions (Table 16). Differently, the hybrid agent with learning exhibited a near to zero, very low initial effectiveness both in terms of ―recall‖ and ―precision‖ after processing the first hundred intrusions, when no reactive rule had still been learned. However, there was a gradual increase over time until 72% on average for ―precision‖ and 28

ACCEPTED MANUSCRIPT 80.98% for ―recall‖ after processing 500 intrusions having already learned a set of 81 rules able to detect 10 different types of intrusions (Table 11). In this experiment about the hybrid agent without learning, all reactive rules were manually inserted into the knowledge base in the agent design phase, thus representing a significant effort for the knowledge engineer. Differently, the hybrid agent with learning learned the rules automatically. On the other hand, the manually inserted set of reactive rules of the hybrid agent without learning may become obsolete over time. Thus, with the evolution of the agent environment, the agent knowledge base should be manually maintained which is expensive and error prone. 4.5 Evaluation conclusion

AC

CE

PT

ED

M

AN US

CR IP T

The HyLAA agent effectiveness, performance, development effort and adaptability were evaluated through the four experiments conducted. In the first experiment, the effectiveness of the CBR component agent was evaluated based on different cut-off points (minimum similarity values) and by assigning greater weights to the most relevant attributes. This experiment showed that 0.9 is the ideal cut-off point for the HyLAA agent; i.e., when recommending solutions with similarity over 90%, a high degree of accuracy is achieved by maintaining a minimum number of retrieved recommendations. Furthermore, it was observed that assigning greater weights to the more relevant attributes has little positive impact on the effectiveness of the recommendations. Thus, considering the effort required to obtain the most relevant weights and attributes for all types of intrusions, this approach is recommended only in situations where small gains in effectiveness are needed. In summary, this experiment achieved the expected result ―Good effectiveness of the CBR component, ability to recommend relevant solutions and hide irrelevant information‖ when using the 0.9 cut-off point. In the second experiment, the accuracy of the learning system in learning new reactive rules was evaluated. Using a set of 1000 examples, 78% of intrusions were correctly identified. In this experiment, it was observed a direct relationship between the training set and the accuracy of the learned rules. Thus, the accuracy of rules is directly related to the training set considering the quality of the data that make up this training set, that is, the absence of inconsistencies, duplication, etc. This experiment reached the expected result of ―Good ability to correctly detect an intrusion‖ when using a training set of 1000 examples. In the third experiment, the effectiveness of the hybrid agent with learning was evaluated. The results of this experiment showed that the effectiveness of the agent behavior gradually improved to the extent that the agent learned new reactive rules. Regarding the results of experiment 4, the hybrid agent without learning showed greater effectiveness than the hybrid agent with learning using the same test set; however, the reactive rules in the knowledge base are inserted manually with a set of initial rules capable of detecting at least two types of intrusion. However, the agent with learning was developed with no predefined rule and was reactive to the extent that the interaction with these rule environments was learned automatically. Thus, to understand the results of their actions on the environment (positive or negative), the hybrid agents with learning self-adapts to the changes that take place in this environment. This occurs as a result of that reactive behavior is faster and more efficient. Using a hybrid with learning architecture reduces the need for ongoing maintenance of the agent knowledge base, as most of the changes that occur in the environment are learned by the agent. This experiment achieved the expected result of ―Greater effectiveness of the hybrid behavior with learning than the CBR or reactive when executed independently‖ and ―Better effectiveness when the agent learns‖. In the fourth experiment, the hybrid agent without learning was evaluated. In this experiment, it was observed that the effectiveness and performance of the agent are directly related to the number of different types of intrusions that reactive rules are able to detect. This experiment produced the expected results of ―Greater effectiveness and better performance when increasing the amount of reactive rules for detecting various types of intrusion‖ and ―Less evolution in effectiveness over time of the agent with learning‖.

29

ACCEPTED MANUSCRIPT Based on the results of the four studies, it can be concluded that the hybrid agent with learning is best suited for dynamic environments making its behavior more effective and with better performance over time through learning and avoiding effort and high maintenance costs of its knowledge base by self-adapting its behavior. 5. RELATED WORK

CR IP T

Many approaches have been developed to address information security issues such as intrusion to computer networks. In this section, general work related to information security is discussed, as presented in (Undercoffer et al., 2003), (Boulahia et al., 2009), (Bul'ajoul, James & Mandeep, 2015), (Wang, Ping, & YuShih Wang, 2015), (Panda et al., 2012), (Kim, Lee and Lee, 2014) and (Rosa, 2011) and using software agents as (Sen, 2010) and (Shuang-Can et al. 2014). Undercoffer et al. (Undercoffer et al., 2003) define an ontology that specifies a model of computer intrusions. To build this ontology an empirical analysis of the characteristics and attributes of intrusions and their relationships from a data set of over four thousand classes of intrusions were conducted. This ontology was specified using the descriptive logic language DAML (―DARPA Agent Markup Language‖) and OIL (―Ontology Inference Layer‖) (Hendler, 2015). The axioms were implemented using DAMLJess KB language (Friedman-Hill, 2007) (Kopena and Regli, 2003).

AN US

In Boulahia et al. work (Boulahia et al., 2009), an ontology-based approach to specify security policies is proposed. This ontology allows mapping alerts in case of intrusion, which are used to identify the security policies to be applied on the computer network to address the intrusion. To carry out such mappings, inference rules were used. This ontology was developed in the Protégé ontology editor using OWL (Motik et al., 2012) (Mcguinness and Harmelen, 2015) and SWRL (―Semantic Web Rule Language‖) languages. It comprises all the elements of an ontology.

ED

M

Rosa (Rosa, 2011) proposes the use of ontologies to build a knowledge base for IDS intrusions, based on the behavior of these intrusions. An IDS prototype to detect known intrusions and intrusions inferred using the ontology is also described. The ontology was implemented in the OWL language and built using the Protégé tool. It is composed of classes, properties, instances and axioms.

CE

PT

In Sen work (Sen, 2010) a multi-agent architecture for distributed intrusion detection in local area networks is specified. Each agent of this architecture has five components: a perception module, responsible for the data collection of the subnet to which the agent belongs; a decision module, responsible for analyzing and reasoning about the data collected by the perception module; a communication module, responsible for the communication of agents; an action module, in charge of acting on detected, for example by sending alerts to the network administrator when an intrusion is detected; an update module, responsible for updating the knowledge base when a new intrusion is detected and confirmed by the network administrator; and a supervisor module, responsible for coordinating the interactions between all other modules.

AC

Shuang-Can et al. (Shuang-Can et al., 2014) propose a multi-agent system for intrusion detection in computer networks. Each agent consists of the following components: a management module, which is the central module, responsible for managing the other modules; a detection module, to collect and pre-process the network data; an analysis module, responsible for receiving collected data by the detection module, to analyze network traffic and detect intrusions or abnormal behaviors; a communication module, responsible for managing the communication of agents and a collaboration module allows the exchange of information between agents. The Bul'ajoul, James & Mandeep (2015) work specifies an approach to improve network intrusion detection system performance through quality of service configuration and parallel technology. This work is particularly relevant for high speed network where using an IDS system in a traditional way has several performance problems. It was evaluated using Cisco Catalyst 3560 Series switches hardware and Snort IDS

30

ACCEPTED MANUSCRIPT software using quality of service configuration and parallel technology. Good results are reported using this approach to detect network intrusions with high speed and volume, thus improving the IDS performance compared with traditional IDS approaches. Wang, Ping, & Yu-Shih Wang (2015) propose a malware detection on mobile devices using a support vector machine (SVM) classifier. Support vector machine is a supervised learning technique used for classification and regression. Good results of accuracy using the SVM classifier to detect malware on mobile devices were obtained.

CR IP T

Panda et al. (2012) propose a hybrid intelligent approach for network intrusion detection which combines supervised and unsupervised machine learning techniques to generate a classifier for network intrusion detection. This work combines different classifiers in a hybrid way. This approach was experimentally evaluated using the NSL-KDD dataset in a Java environment. The work shows good evaluation results achieving a high intrusion detection rate and a low false alarm rate. Kim, Lee and Lee (2014) propose a novel hybrid intrusion detection method integrating anomaly detection with signature-based detection. This work uses a support vector machine classifier to detect intrusion using anomaly-based technique and decision tree when using a signature-based technique. Evaluation results show that the approach have good intrusion detection rates.

AN US

Table 17 lists the main features of each approach (approach, approach type, IDS technique, supported tasks, knowledge base representation and used language) for comparison purposes. The approaches of (Undercoffer et al., 2003), (Boulahia et al., 2009) and (Rosa, 2011) define application ontologies with many domain concepts, but their task concepts are quite limited. The ontology defined in (Boulahia et al., 2009) has a poor semantic representation, in which most classes have no relationship with each other, and largely constituted by a simple taxonomy. The Sen (Sen, 2010) and ShuangCan et al. (Shuang-Can et al., 2014) works and the one proposed in this paper approaches agent technology for intrusion detection in computer networks.

ED

M

It is considered that HyLAA has advantage over the works of Undercoffer et al. (Undercoffer et al., 2003), Boulahia et al. (Boulahia et al., 2009) and Rosa (Rosa, 2011) considering the use of software agents to address the problem of intrusion detection in computer networks. Agents have features as autonomy, communication and learning that make intrusion detection more effective. In relation to the work of Sen (Sen, 2010) and Shuang-Can et al. (Shuang-Can et al., 2014), which also use software agents, the main advantage of the HyLAA agent is that it improves its behavior over time through learning.

AC

CE

PT

The Bul'ajoul, James & Mandeep (2015) work differs from the others because it uses quality of service configuration and parallel technology in the IDS implementation in high-speed networks. The Wang, Ping, & Yu-Shih Wang (2015) approach detects malwares in mobile devices using support vector machine, a machine learning technique. The learned malware patterns are represented in an ontology. The Panda et al. (2012) work uses anomaly-based detection technique to detect intrusions on computer networks. The Kim, Lee and Lee (2014) approach detects intrusion using both signature and anomaly-based detection techniques by combining support vector machine and decision tree techniques. HyLAA differs from Bul'ajoul, James & Mandeep (2015), Wang, Ping, & Yu-Shih Wang (2015), Panda et al. (2012) and Kim, Lee and Lee (2014) works because it uses an agent approach for intrusion detection using both reactive signature-based behavior and anomaly-based decision.

31

ACCEPTED MANUSCRIPT Used Language DAML and OIL

OWL and SWRL

OWL

Not defined

Not defined

Snort language

Not defined

AC

CE

PT

ED

M

AN US

CR IP T

Table 17. Comparison between approaches to information security and intrusion detection IDS Knowledge Base Approach Approach Type Supported Tasks Technique Representation Modeling Computer Ontology containing Attack: An Ontology for AnomalyApplication Detect intrusions classes, hierarchy, Intrusion Detection based Ontology and send alerts properties and non(Undercoffer, Joshi & detection taxonomic relationships Pinkston, 2003). An Ontology-based Ontology containing SignatureApproach to react to Application Identify security classes, non-taxonomic based Network Attacks Ontology policies relationships, properties, detection (Boulahia et al., 2009) instances and axioms An Ontology Intrusion Signature and Ontology containing Ontology for Detection Systems in anomalyDetect intrusions classes, properties, Application and Web Services (Rosa, based and send alerts hierarchy, instances and IDS Prototype 2011) detection axioms An Agent-Based Signature and Intrusion Detection anomalyDetect intrusions System for Agent Bayesian networks based and send alerts Local Area Networks detection (Sen, 2010) Multi-Agent Distributed Intrusion Detection Signature and System Model Based on Multi-Agent anomalyDetect intrusions Neural network BP Neural Network system based and send alerts (Shuang-Can, Chen-Jun detection & Wei-Ming, 2014) Improving network IDS intrusion detection implementation system performance approach using Signaturethrough quality of Detect intrusions quality of service based A set of rules service configuration and send alerts configuration and detection and parallel technology parallel (Bul'ajoul, James & technology Mandeep, 2015) Malware behavioural detection and vaccine Behavioural development by using a AnomalyDetect malware and detection model support vector model based output a detection Ontology and an IDS for classifier (Wang, Ping, detection log mobile malware & Yu-Shih Wang, 2015). A hybrid intelligent A hybrid Anomalyapproach for network approach for Detect intrusions Database containing based intrusion detection network intrusion and send alerts intrusion instances detection (Panda et al., 2012) detection A novel hybrid intrusion detection method Signature and integrating anomaly Intrusion anomalyRules generated by a Detect intrusions detection with misuse detection method based decision tree algorithm detection (Kim, Lee and detection Lee, 2014) Ontology containing Signature and Agent and Detect and provide classes, anomalyHyLAA Agent Application preventive and Non-taxonomic based Ontology corrective solutions relationships, property, detection hierarchy and instances.

Not defined

Not defined

OWL and Prolog

32

ACCEPTED MANUSCRIPT 6. CONCLUSION

AC

CE

PT

ED

M

AN US

CR IP T

This paper presented the HyLAA hybrid learning agent architecture that combines reactive behavior and case-based reasoning for detecting intrusions in computer networks. A static view of the agent, including the description of all components of its architecture and a dynamic view showing the states through which the agent passes during its life cycle were presented. It was also discussed a comparative analysis between HyLAA and other approaches for intrusion detection in computer networks. HyLAA development was guided by MADAE-Pro, a process giving support to the main phases of multi-agent software engineering (Girardi and Leite, 2010) (Girardi and Leite, 2011). HyLAA differs from traditional intrusion detection systems because it automatically acquires new reactive behavior by observing good recurring solutions provided by the case-based reasoning system to the same perception through supervised learning using decision trees. Thus, according to the feedback resulting from the case-based reasoning behavior, the agent turns this behavior into reactive one, improving its performance and also making it adaptable to the evolution of the environment. This feature is especially relevant in dynamic environments as computer networks where new and previously unknown intrusions are constantly emerging and many of them are similar and occur frequently. In addition, the HyLAA knowledge base is represented in an ontology making the agent more effective for intrusion detection than traditional intrusion detection systems due to the large semantic expressiveness of ontologies. The effectiveness of HyLAA to detect intrusions using case-based reasoning behavior, the accuracy of the classifier learned by the learning component and both the performance and effectiveness of HyLAA to detect intrusions using hybrid behavior with learning and without learning were evaluated, respectively, by conducting four experiments. In the first experiment, HyLAA exhibited good effectiveness when using heavier weights for the most important attributes and the 0.9 cut-off point in the similarity computation. In the second experiment, the learned classifiers presented good accuracy. Both the hybrid agent behavior with learning and without learning (third and fourth experiment, respectively) showed greater effectiveness and performance than the one reported by the CBR system when executed independently, but only the hybrid behavior had better effectiveness and performance as long as the agent learns. Thus, it can be concluded with the experimental work that the hybrid agent with learning is best suited for dynamic environments making its behavior more effective and with better performance over time through learning and avoiding effort and high maintenance costs of its knowledge base by self-adapting its behavior. In spite that these initial experimental results are promising, it is planned to conduct other experiments in other domains using different reasoning techniques and learning algorithms. For instance, a new concrete agent architecture to recommend sentence solutions supporting the decisions of judges in the Brazilian Family Law legal processes will be developed. The reactive behavior will be composed by jurisprudence cases represented as reactive rules while new family cases will be handled through practical reasoning using a BDI approach. Another potential application of the HyLAA hybrid learning agent architecture is the learning of reasoning rules thought induction of reactive rules. This approach is particularly relevant when using deductive reasoning as deliberative system because the creation of inference rules requires a large development effort. Currently, a reference architecture that specifies a generic architectural solution for the development of specific architectures of hybrid learning agents called HyLARA is being specified. It allows developing concrete hybrid agent architectures by combining different types of deliberative and reactive components. HyLARA (―Hybrid and Learning Agent Reference Architecture‖) is a generalization of the HyLAA architecture components and relationships. This reference architecture will allow the development of hybrid agents faster and more reliable by reusing good design practices specified during the development and evaluation of the HyLAA agent. Another future work is the integration of the HyLAA with MADAE-IDE (Cavalcante and Girardi, 2010). MADAE-IDE is an integrated multi-agent software development environment that automates through reuse most design activities.

33

ACCEPTED MANUSCRIPT ACKNOWLEDGMENTS This work has been supported by CAPES and FAPEMA, research-funding agencies of the Brazilian government and the Maranhão State, respectively, have supported this work. REFERENCES

AC

CE

PT

ED

M

AN US

CR IP T

Aamodt, A., Plaza, E., 1994. Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Communications, 7(1), 39-59. Benmoussa H., El Kalam, A.A.; Ouahman, A.A.,2014. Towards a new intelligent generation of intrusion detection system, Proceedings of the 4th Edition of National Security Days (JNS4),1-5. Boulahia, N. C., Cuppens, F., Vergara, J. E. L., Vazquez, E.,Guerra, J., Debar, H., 2009. An ontology-based approach to react to network intrusions. International Journal of Information and Computer Security, vol. 3, 280–305. Bul'ajoul, W, Anne, J., Mandeep P. "Improving network intrusion detection system performance through quality of service configuration and parallel technology." Journal of Computer and System Sciences 81.6 981-999,2015 Cavalcante, U., Girardi, R., 2010. An overview of the MADAE-IDE multi-agent system development environment. InInternational Conference on Information Technology: New Generations2010, 968-973. DARPA. Intrusion Detection Evaluation. Retrieved on May 19, 2015, from http://www.ll.mit.edu/ideval/docs/intrusionDB.html. Debar, H., 2004. Intrusion detection systems: introduction to intrusion detection and analysis, security and privacy in technologies. Nato Science Series III, vol. 193, IOS Press, edited by Borka Jerman Blazic, Wolfgang Schneider and Tomaz Klobular. Friedman-Hill, E. J., 2007. Jess the java expert system shell. Interface, 8206, 1–73. Retrieved June 19, 2015, from http://herzberg.ca.sandia.gov/jess/README.html. Gennari, J., Musen, M., Fergerson, R.W., Grosso, W.E., Crubezy, M., Eriksson, H., Noy, N.F., Tu, S.W., 2003. The evolution of Protégé: An environment for knowledge-based systems development. Int. J. Hum. Comput. Stud. 58, 89-123. Girardi, R. Classification and Retrieval of Software through their Description in Natural Language. Computer Science Department, University of Geneva, PhD Thesis, 1995. Girardi, R., Leite, A., 2010. The specification of requirements in the MADAE-PRO software process. iSys: Revista Brasileira de Sistemas de Informação, v. 3, 3-10. Girardi, R., Leite, A., 2011. Knowledge engineering support for agent-oriented software reuse. In: M. Ramachandran. (Org.). Knowledge Engineering for Software Development Life Cycles: Support Technologies and Applications. Hershey: IGI Global, v. I, 177-195. Gruber, T., 1995. Toward principles for the design of ontologies used for knowledge sharing. International Journal of Human-Computer Studies, 43(5-6), 907–928. http://doi.org/citeulike-article-id:230211. Guarino, N., 1998. Formal ontology in information systems. In Proceedings of the First International Conference (FOIS’98),1st ed., Trento, Italy: IOS press, 1998. Hendler, J., 2015. DARPA agent markup language ontology interface layer, Retrieved on March 1, 2015, from http://www.daml.org/2001/03/daml+oil-index. ISO/IEC 27001, 2005. Information technology - security techniques - information security management systems - requirements. Geneva: International Organization for Standardization. Kim, G., Lee, S., Kim, S. A novel hybrid intrusion detection method integrating anomaly detection with misuse detection. Expert Systems with Applications, 41(4), 1690-1700, 2014.

34

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN US

CR IP T

Kohavi, R. A study of cross-validation and bootstrap for accuracy estimation and model selection. Ijcai. Vol. 14. No. 2. 1995. Kopena, J., Regli W., 2003. DAMLJessKB: a tool for reasoning with the semantic web. In D. Fensel, K. Sycara, and J. Mylopoulos, eds. The Semantic Web - ISWC. Lecture Notes in Computer Science. Sanibel Island, USA: Springer Berlin Heidelberg, 628–643. Mcguinness, D. L., Harmelen, F. V.,2015. OWL web ontology language, Retrieved on September 1, from http://www.w3.org/TR/owl-features. Mendes, W. C., Girardi, R., Leite, A., 2013. Ontology-based architecture of an CBR Agent. In Proceedings of the 8th Iberian Conference on Information Systems and Technologies, v. I, 776-781 (In Portuguese). Meneses, R., Leite, A., Girardi, R., 2015. Application ontology for the development of case-based intrusion detection systems. In: 10ª Conferencia Ibérica de Sistemas y Tecnologías de Información, pp 1-4, Aveiro (In Portuguese). Mitchell, T. M., 1997. Machine learning. Annual Review of Computer Science. http://doi.org/10.1145/242224.242229. Molyneaux, I. The Art of Application Performance Testing: From Strategy to Tools. O'Reilly Media, Inc., 2014. Molyneaux, I., 2014. The Art of Application Performance Testing: From Strategy to Tools. O'Reilly Media,New York, NY, USA. Motik, B., Patel-Schneider, P. F., Parsia, B., Bock, C., Fokoue, A., Haase, P., and Smith, M., 2012. OWL 2 web ontology language - structural specification and functional-style syntax (second edition). Online, 1133. Olusola, A. Oladele, s. A., Abosede, D. O. Analysis of KDD'99 Intrusion detection dataset for selection of relevance features. In: Proceedings of the World Congress on Engineering and Computer Science, p. 20-22, 2010. Panda, M., Ajith, A., Manas R. A hybrid intelligent approach for network intrusion detection. Procedia Engineering 30 (2012): 1-9. Quinlan, J. R. C4. 5: Programs for Machine Learning. Elsevier, 2014. Quinlan, J. R. Induction of decision trees. Machine learning, 1(1), 81-106, 1986. Revathi, S., Malathi, A., 2013. A detailed analysis on nsl-kdd dataset using various machine learning techniques for intrusion detection. International Journal of Engineering Research and Technology (IJERT), vol. 2, 1848-1853. Rokach, L., Maimon, O., 2005. Decision Tree, Data Mining Knowledge Discovery Handbook, pp 165–192. doi:10.1007/978-0-387-09823-4_9 Rosa, T. M., 2011. An ontology for intrusion detection in web services, Master Thesis, PontificalCatholic University of Paraná (In Portuguese). Russel, S., Norvig, P., 2009. Artificial intelligence: a modern approach, 3nd ed., Prentice-Hall, 2009. Salton, G., Allan, J., Buckley, C.,1993. Approaches to passage retrieval in full text information systems. In Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, 49-58. Sawant, T. S., Itkar, S. A., 2014. A survey and comparative study of different data mining techniques for implementation of intrusion detection system. International Journal of Current Engineering and Technology, vol. 4, 1288–1291. Sen, J., 2010. An agent-based intrusion detection system for local area networks. International Journal of Communication Networks and Information Security IJCNIS, 2(2), 13, Retrieved from http://arxiv.org/abs/1011.1531.

35

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

AN US

CR IP T

Shuang-Can, Z., Chen-Jun, H. and Wei-Ming, Z., 2014. Multi-agent distributed intrusion detection system model based on bp neural network. International Journal of Security and Its Applications, v. 8, n. 2, 183-192. Sokolova, M. Lapalme, G. A systematic analysis of performance measures for classification tasks. Information Processing & Management, v. 45, n. 4, p. 427-437, 2009. Steel, B., 2015. Chimera agents for Win-prolog. Retrieved on November 1, 2015, from http://www.lpa.co.uk/chi.htm. Tavallaee, M., Bagheri, E., Lu, W., and Ghorbani, A., 2009. A detailed analysis of the KDD CUP 99 data set. IEEE Symposium on Computational Intelligence for Security and Defense Applications, CISDA 2009, (Cisda), 1–6, http://doi.org/10.1109/CISDA.2009.5356528. Undercoffer, J. Joshi, A. Pinkston, J., 2003. Modeling computer intrusions: an ontology for intrusion detection, In G. Vigna, C. Kruegel and E. Jonsson, eds. Recent Advances in Intrusion Detection, Lecture Notes in Computer Science. Springer Berlin Heidelberg, 113–135. Uschold, M., Gruninger, M.,1996. Ontologies: principles, methods and applications. The knowledge engineering review, v. 11, n. 02, 93-136. Vassiliadis, V., Wielemaker J., Mungall, C., 2009. Processing OWL2 ontologies using thea: an application of logic programming, sixth international workshop OWLED, Virginia, USA, vol. 529. Wang, P, Yu-Shih Wang. Malware behavioural detection and vaccine development by using a support vector model classifier. Journal of Computer and System Sciences 81.6 1012-1026,2015 Wooldridge, M., 2009. An introduction to multiagent systems (2nd ed.). Wiley Publishing, New York, NY, USA. Yu, C. H., Ohlund, B. Threats to validity of research design.Retrieved January, 12, 2012.

36