Open source and free log analysis and log management tools.

Open source and free log analysis and log management tools.

Maintained by Dr. Anton Chuvakin
Version 1 created 3/3/2010
Version 1.1 updated 4/15/2010
Version 1.2 updated 10/1/2010
Version 1.3 updated 3/3/2011

This page lists a few popular free open-source log management and log analysis tools. The page is a supplement to "Critical Log Review Checklist for Security Incidents" that can be found here or as PDF or DOC (feel free to modify it for your own purposes or for internal distribution - but please keep the attribution).
The log cheat sheet presents a checklist for reviewing critical system,
network and security logs when responding to a security incident. It
can also be used for routine periodic log review. It was authored by Dr. Anton Chuvakin and Lenny Zeltser.

The open source log management tools are:

1. OSSEC (ossec.net) 
   an
   open source tool for  analysis of real-time log data from Unix systems,
   Windows servers and network devices. It includes a set of useful
   default alerting rules as well as a web-based graphical user interface.
   This is THE tool to use, if you are starting up your log review program.
   It even has a book written about it.
2. Snare agent
   (intersectalliance.com/projects/index.html) and ProjectLasso remote collector (sourceforge.net/projects/lassolog)
   are used to convert Windows Event Logs into syslog, a key component of any log management infrastructure today (at
   least until Visa/W7 log aggregation tools become mainstream).
3. syslog-ng (balabit.com/network-security/syslog-ng/) is a replacement and improvement of classic syslog service - it also has a Windows version that can be used the same way as Snare
4. rsyslog (rsyslog.com)
   is another notable replacement and improvement of syslog service that
   uses traditional (rather than ng-style) format for syslog.conf
   configuration files. No Windows version, but it has an associated
   front-end called phpLogCon
5. Among the somewhat dated tools, Logwatch (logwatch.org), Lire (logreport.org) and LogSurfer (crypt.gen.nz/logsurfer) can all be used to summarize logs into readable reports.
6. sec (simple-evcorr.sourceforge.net) can be used for correlating logs, even though most people will likely find OSSEC correlation a bit easier to use
7. LogHound (ristov.users.sourceforge.net/loghound) and slct (ristov.users.sourceforge.net/slct) are more "research-grade" tools, that are still very useful for going thru a large pool of barely-structured log data.
8. Log2timeline (log2timeline.net/) is a useful tool for investigative review of logs; it can create a timeline view out of raw log data.
9. LogZilla (aka php-syslog-ng) (code.google.com/p/php-syslog-ng)
   is a simple PHP-based visual front-end for a syslog server to do searches, reports, etc

The next list is "an honorable mentions" list which inlcudes logging tools that don't quite fit the definition above:

• Splunk is neither free nor
  open source, but is has a free version usable for searching up to 500MB
  of log data per day - think of it as a smart search engine for logs.
  Splunk includes a tool to extracting parameters out of log data
• Offering both fast index searches and parsed data reports, Novell Sentinel Log Manager 25
  is not open source, but can be used for free forever as long as your
  log data volume does not exceed 25 log messages/second (25 EPS). Unlike
  splunk above, it includes log data parsing for select log formats and
  thus can be used for running reports out of the box, not just searching
• Q1Labs is also neither free nor open source, but is has a free version usable for managing up to 50EPS (roughly 2GB/day). It can be downloaded as a virtual appliance
• OSSIM  is not just for logs and also includes OSSEC; it  is an
  open source SIEM tool and can be used much the same way as commercial
  Security Information and Event Management tools are used (SIEM use cases)
• Microsoft Log Parser
  is a handy free tool to cut thru various Windows logs, not just Windows
  Event Logs. A somewhat similar tool for Windows Event log analysis is
  Mandiant Highlighter (mandiant.com/products/free_software/highlighter)
• Sguil is not a log analysis tools, but a  network security monitoring (NSM) tool, but it uses logs in its analysis.
• Loggly now offers free developer accounts (at loggly.com/signup)
  for their cloud log management service. The volume limitation is
  200MB/day and retention time limiation is 7 days. If you'd like to
  collect and search your logs without running any software, this is for
  you.

For a list of commercial log management tools go to Security Scoreboard site. A few of the commercial tools offer free trials for
up to 30 days or longer.

Back to Security Warrior Consulting by Dr. Anton Chuvakin.

Modified: 08-Mar-2011