A fundamental technology issue occurring in many organizations today involves balancing limited log management resources with the increasing supply of log data. The high number of log sources, inconsistent content, and variable format all contribute to this issue. Log management and analysis are key processes in dealing with security issues, fraud, and operational issues. Logs can also assist with record keeping, establishing baselines, identifying trends and performing audits. However, the lack of real-time or near real-time analysis significantly reduces the effectiveness of logs.
nSnare™, developed at the National Renewable Energy Laboratory, is a software that analyses and categorizes log messages generated by various and customizable third party software packages using a custom multi-tiered rules engine. Each log message is either determined to be a false positive, or it is assigned a score based on the severity of the potential attack and confidence that the activity described by the log message is not a false positive. Log messages are then assigned to an event, which is a collection of log messages related to a single potential attack. Meta-analysis rules run on the collection of all logs within an event to correlate multiple related logs into a single known attack signature. When the sum of the scores of all logs within an event exceeds a pre-defined threshold, the event is labelled as an attack and an automated response is generated to block the attacker from connecting to any protected resources, called quarantining within the application, for a variable length of time. The length of time that the quarantine lasts for is based on the severity and frequency of the attack, along with historical information on the attacker and any third party provided threat intelligence data that the system has gathered. An additional random length of time is added to each quarantine so the algorithm cannot be easily learned and bypassed by a persistent attacker. A confidence score is generated based on a proprietary algorithm that determines the likelihood that the data which led to the attacker getting quarantined was a false positive. A threat intelligence record is created for each detected attack, with details on the attacker, a summary of what triggered the quarantine, the severity of the attack and the confidence the system has that the attack is not the result of false data, which can then be shared with external entities so that they can also protect themselves from a related attack.
nSnare™ also includes a web based front end for analyzing the current quarantined attacks, removing a quarantine in the case of a false positive, viewing previously quarantined attacks, and tracking suspicious activity that the system is evaluating. The web interface also allows the creation and modification of rules used by the rules engine, and the maintenance of a whitelist of known good systems that should never be automatically quarantined.