Log Monitors in BSD UNIX

Abstract

A log monitor is a process, or daemon, which monitors log messages produced by a computer system and the programs which run on it. A properly designed log monitor can recognize unusual activity (or inactivity), alert an administrator to problems, gather statistics about system activity, and/or take automatic action to contain a threat. It can even "learn," over time, what is normal and identify message traffic that may betray an abnormal situation.

On BSD UNIX systems, most applications and daemons log events via the system logging daemon (syslogd). Because Berkeley syslogd allows log messages to be piped to a program as well as written to one or more files, implementation of a BSD log monitor often begins with the creation of a program which accepts formatted log messages via standard input, parses them, and extracts necessary information. Utilities which do not log events via syslogd (e.g. the Apache Web server) may require a separate log monitor process. In all cases, data validation, robust input handling, and careful parsing of log messages are especially important considerations in log monitor design, lest the monitor itself compromise the security of the system it is intended to monitor. Languages which facilitate string processing and pattern matching, such as Perl and SNOBOL4, are good choices for the implementation of log monitors for this reason.

Log monitors can work in tandem with firewall software to block traffic from a would-be intruder, spammer, or "mail bomber." For example, it can tally the number of outgoing e-mail messages sent by users on an internal network and detect activity which appears to constitute spamming or mail bombing. A log monitor can also recognize efforts to exploit security holes, such as attempted infection by the Code Red worm or attempts to exploit CGI scripts included with older versions of the Apache Web server. Included in this paper is source code for a log monitor that identifies and blocks attacks from Code Red, sadmin/IIS, Nimda, and similar worms. Policy issues -- including the usefulness of "amnesty" to prevent inadvertent blocking of innocent third parties -- are discussed. The work described in this paper --which is still progressing at this writing -- will hopefully culminate in the release of a general purpose log monitoring facility.

[Note: Because the camera-ready copy of this paper had to be submitted two months before the BSDCon 2002 conference to allow time for printing, and USENIX enforces strict limits on size, the version of this paper which appears online may contain more detail as well as new information from ongoing work. For the latest version, and/or to follow up the many references via HTML links, access the master copy at http://www.brettglass.com/logmonitors/paper.html on the World Wide Web.]

1  Introduction

2  What is a Log Monitor?

A stateful log monitor is a log monitor that infers the presence of a condition requiring attention by compiling data from more than one log message. It may simply note the number and/or frequency of log messages related to a particular type of activity or may generate more sophisticated cumulative statistics from those messages.

2.1  Capabilities of Log Monitors

• Detect abnormal usage patterns
• Recognize system or network abuse (e.g. spamming and mail bombing)
• Catch worms and other malware
• Detect vulnerability scans (e.g. port scans)
• Detect intruders (or attempted intrusions)
• Detect resource shortages (e.g. slow response times, out-of-memory conditions, out-of-disk conditions, inadequate swap space)
• Detect imminent or actual system failures
• Compile statistics in real time (including running averages, etc.)
• React to conditions by notifying an administrator and/or taking immediate action

2.2  Examples of Existing Log Monitors

watchfor /ANONYMOUS FTP LOGIN REFUSED FROM (\S*)/
 mail addresses=admin,subject=Attempted anonymous FTP from $1
 exec blackhole $1

It then "tails" a system log file looking for the regular expression specified in the "watchfor" line of the block. If the expression is matched, swatch takes the series of actions that follow. Possible actions include producing a beep, writing a message in a specified color to the console, sending an e-mail message including the log entry that was matched, or executing an arbitrary command. (In the example above, the $1 causes the text matched by the subexpression within the first set of parentheses to be interpolated.) swatch incorporates features to allow throttling and to recognize patterns only during certain hours of the day and/or days of the week.

2swatch extends swatch by offering the possibility of deferred as well as immediate action. 2swatch can accumulate a series of messages matching a pattern into a report that is e-mailed as a single message. This prevents mail systems from being flooded with e-mail messages that each contain a single log entry. Unfortunately, neither swatch nor 2swatch can perform stateful monitoring beyond their respective throttling and accumulation functions, though it is of course possible to implement statefulness via cleverly designed external programs or scripts. Nor can these two programs, by themselves, correlate entries from more than one log. A final drawback of these two programs is that they have been released under licensing terms (the GPL and a unique "no commercial use" license, respectively) which hamper or prohibit their reuse in commercial products.

The primary objective of the work described in this paper is to develop a generalized and portable framework which eases the creation of customized log monitors and overcomes the limitations described above. It is intended that the results of this work, which is initially being performed on FreeBSD, be released under "truly free" (i.e. MIT- or BSD-style) licensing terms so as to permit adaptation to other operating systems and commercial as well as non-commercial reuse.

2.3  Log Monitors vs. Log Analyzers

Some log analyzers are designed to "wake up" periodically and scan the latest entries in a system log, acting, if appropriate, upon what they "see." (Kai's Spamshield [6], which detects incoming spam and blocks the sender via a "blackhole" route, is an example.) If the interval between scans is sufficiently short, such log analyzers can perform some of the functions of a log monitor, detecting and handling conditions which require timely but not instantaneous attention.

3  Log Monitors and BSD syslogd

3.1  syslogd Facilities, Priorities, and Tags

 

Table 1: Standard facility codes used by Berkeley syslogd

auth authpriv cron daemon ftp

kern lpr mail mark news

syslog user uucp  local0..local7

 

Table 2: Severity levels (priorities) used by Berkeley syslogd

emerg (highest) alert crit

err warning notice

info debug (lowest)

Further confusing the issue of how to classify log messages is the fact that many recent versions of syslogd have added the ability to sort messages via strings called "tags," which are transmitted to the logging daemon along with each message. Tags usually (but not always) contain the name of the program that generated the message. (At this writing, the ability to sort messages by tag is available in OpenBSD and FreeBSD but not in NetBSD.) To facilitate logging across networks, some implementations of syslogd add yet another sorting criterion: they allow messages to be dispatched according to the name of the host from which they originated.

As if all of this weren't enough, there's yet another fly in the ointment. While Berkeley syslogd itself is able to sort messages by facility, severity, tag, and originating host, it does not record the facility and severity level in each log message, and in most implementations there is no way to make it do so. This makes it difficult for a log monitor (and, in some cases, for humans) to use these same criteria to sort messages that syslogd has aggregated into a single log file. The version of syslogd found in recent versions of FreeBSD is one of the few that provides a solution to this problem. If the daemon is started with the -v ("verbose") command line option, the facility and priority level are logged as numbers. Specifying -v twice causes them to be shown by name between angle brackets (e.g. <auth.notice>). The latter choice, while it consumes a bit more disk space, makes the logs much easier both to read and to monitor. It would be a relatively simple matter to add similar capabilities to other syslogd implementations.

3.2  Monitoring Techniques for Use With syslogd

Another technique is to open the log file every so often, collect the last n lines, and then close the file. This technique avoids problems when logs are rotated, but may cause the log monitor to miss important messages if the log has grown more quickly than expected (as can happen during an unusual situation). Repeated opening and closing of the file also creates substantial overhead.

The best technique, when it is possible to use it, is to instruct syslogd to pipe messages directly to a log monitor process. This option is not available in all implementations of syslogd; however, it is present in FreeBSD's syslogd and BalaBit's syslog-ng [11] and can be easily added to other logging daemons. Some versions of syslogd, including the Berkeley-derived Linux syslogd and Core-SDI's modular syslog (msyslog) [12], cannot pipe directly to an arbitrary program but can send output to a named pipe where an application is listening.

FreeBSD's syslogd makes piping to applications especially easy by handling nearly all of the logistics for the programmer. The lines shown in Listing 2, when placed in in /etc/syslog.conf will cause messages regarding mail to be appended to /var/mail/maillog in the usual way and also piped to a monitoring program written in Perl.
 

# Log to /var/log/maillog, as usual
mail.info    /var/log/maillog

# Also pipe the same messages to mailmon.pl
mail.info    |exec perl /usr/local/bin/mailmon.pl

Because syslogd handles the logistics of distributing each log message to the required destinations (including the log monitor), the log monitor process is unaffected by log file rotation. It merely has to be prepared to save its state and exit if syslogd restarts (see below).

One feature of syslogd which may actually defeat the purpose of a log monitor is automatic output compression. When syslogd sees two or more identical messages bound for the same destination, it outputs the first and then counts (but does not output) the duplicates. After a predetermined delay, it outputs a message of the form "Last message repeated n times" indicating the number of copies received during the delay period. If still more copies of the same message arrive, the process is repeated with a longer delay between reports. Ironically, this feature -- which cannot be turned off in any version of syslogd known to the author -- is just the opposite of what is needed for effective log monitoring. (It is precisely when an unexpected flood of repeated messages arrives that it is most useful for a log monitor to take prompt, autonomous action. But if notice of those messages are delayed, it cannot do so.) The author has submitted a patch to the FreeBSD Project which disables repeat counting on messages piped to a program. (With the patch in place, if the same messages also go to a file or terminal, compression will still occur on those outputs.) It may also be desirable to disable compression when logging to a remote host, since -- while this would cause an increase in network traffic -- it would facilitate the implementation of remote log monitors. Similar modifications should be made to other logging daemons that direct output either to programs or to named pipes, to facilitate the use of log monitors.

3.3  Processing Piped Output from FreeBSD's syslogd: Caveats and Tricks

Secure programming practices are of the utmost importance when one is creating log monitors. Piped applications started by FreeBSD's syslogd run with the same uid as syslogd itself -- normally root. Because a key function of log monitors is to take adminstrative action as a result of what they observe in a stream of log messages, they often must run as the superuser and may not be able to accomplish their intended functions if they "drop" privileges for safety. (In a capabilities-based system, it may be possible for a log monitor to drop capabilities that the author knows it will never use.)  It is therefore especially important to avoid potential buffer overflows, format string vulnerabilities, and security holes that might arise when unfiltered input is passed directly to system APIs or programs. "Tainting" and/or extremely careful validation of input is strongly recommended.

Should an application which receives piped output from syslogd terminate of its own accord, it will be restarted when there is more input for it. However, syslogd may itself need to request that a log monitor terminate. (The most common situation in which this will occur is when syslogd receives a "hangup" signal -- SIGHUP -- indicating that it must restart and reread its configuration file.) syslogd indicates its desire to shut down the log monitor by closing the pipe it has created to the log monitor's standard input. The log monitor then has a predetermined amount of time -- 60 seconds in most implementations -- to save any state it wishes to preserve and terminate. If it takes longer, syslogd will attempt to kill it by sending it a "terminate" signal -- SIGTERM. To ensure that syslogd is able to kill a log monitor that is frozen, it is advisable for a log monitor not to catch SIGTERM unless it may need a very long time to save its state.
 

4  Log Monitors and Apache

Apache can pipe log messages to external programs on all UNIX and UNIX-like platforms, making the implementation of log monitors, log rotators, and other such utilities very straightforward even on systems whose syslogd implementation does not support piped commands (See Listing 3).
 

# Piped logs can be used for log rotation as well as monitoring
CustomLog "|rotatelogs /var/log/www_log 86400" combined
ErrorLog "|rotatelogs /var/log/www_errors 86400"

# Apache can maintain more than one access log, so you can feed
# access information to a monitor and also to a log file
CustomLog "|exec perl /usr/local/bin/webmon.pl" combined

# Piping error log messages to a monitor is especially useful
# when one wants to detect abuse and/or attacks. Alas, there can
# only be one error log, so you must either rely on access logs
# for error information (usually an acceptable solution) or
# design your monitor to "tee" the messages to a file for you.
ErrorLog "|exec snobol4 -b /usr/local/bin/wormwatch.sno"

4.1  Creating Primitive Log Monitors Within Apache

# Flag requests for URIs containing common strings from Nimda-like worms 
# (including Code Red, sadmind/IIS). Note that the patterns below are regexes;
# remember to escape dots and other characters with special significance!

SetEnvIf Request_URI "/winnt/system32/cmd\.exe" worm
SetEnvIf Request_URI "/scripts/root\.exe" worm
SetEnvIf Request_URI "/MSADC/root\.exe" worm

# Don't use the following patterns if you use "upreferences" in URIs
SetEnvIf Request_URI "/\.\." worm
SetEnvIf Request_URI "\.\./" worm

# Block attackers who send the patterns above within URIs. The command below
# uses a blackhole route. It's more efficient to firewall (the command
# will vary depending upon the firewall in use) or to use SSH to add rules to
# an upstream firewall to block the attacker, but this method has the
# advantage that it is relatively independent of configuration. If several 
# commands must be executed, or if postprocessing of output is desired, it
# is best to invoke a script or compiled program rather than doing all the 
# work from within httpd.conf.
CustomLog "|exec sh" "route -nq add -host %a 127.0.0.1 -blackhole" env=worm

# Note that no input from the client is used in the shell command, so this
# set of directives is not subject to exploits via crafted strings. If strings
# from the client were used, stronger input validation would be in order.

While BSD and Apache cannot be "infected" by the worms targeted by the directives in Listing 4, a worm can nonetheless tax a Web server by consuming processes in the Apache process pool, glutting system logs with error messages, and infecting susceptible machines elsewhere on the network.

The configuration shown in Listing 4 uses Apache's SetEnvIf [17] directive (implemented by the module mod_setenvif) to perform regular expression matching on incoming URIs. It then uses the CustomLog [18] directive (implemented in mod_log_config) to do conditional logging based on the results of the match. When a worm is detected, Apache pipes a specially formatted "log message" -- actually a command -- to a shell for execution. (The "exec sh" may at first glance seem redundant. However, it is necessary to restart the shell -- which is initially invoked so as to accept a command as an argument -- so that it will accept commands via standard input instead.) The command creates a "blackhole" route on the host machine and locks out the attacker. If the Web server does double duty as the gateway between the Internet and an office LAN (as is often the case in small office/home office networks), blackholing the attacker will also protect the machines that sit "behind" the server. If there is a firewall upstream of the Web server, it may be desirable to replace the command that creates a blackhole route with one that causes the firewall to block the attacker.

While the author had great success with this simple log monitor (which he crafted during the early morning hours of 18 September, 2001 when Nimda began to spread), it clearly has many deficiencies. For example, it does not check to see whether an attack is coming from the Web server's own address (which can easily happen if the machine is doing double duty as a NAT router or dial-up server.) to prevent it from blackholing itself! Nonetheless, this example is a valuable proof of concept. It demonstrates that the ability to apply a general pattern matching facility to log messages, and then execute commands based on those messages, are sufficient to allow the creation of a useful, if not perfect, log monitor. More sophisticated tools -- such as languages with built-in pattern matching -- make it even easier to write quite sophisticated agents.

4.2  Monitoring Techniques for Use With Apache

Unlike syslogd, Apache starts piped applications as soon as it has finished reading its configuration file. This gives them time to start up before receiving the first message, improving response time at the expense of overhead. If a piped application terminates, Apache restarts it the next time a message is to be delivered to it. Like syslogd, Apache normally uses sh(1) to parse command lines and set up file redirection for piped commands.

It is important to remember, when writing log monitors for Apache, that users' log formats may vary. It is therefore best to use a custom log format that the log monitor expects -- or, alternatively, to monitor the error log, whose format cannot be customized and is therefore almost fixed. (One aspect of the error log format can be changed via configuration: the way hosts are identified. If HostNameLookups [19] is on, domain names are output instead of numerical IP addresses.) Note that Apache allows error log messages to be sent only go to one destination. (If there is more than one ErrorLog directive, each overrides the previous ones rather than supplying an additional destination.) Fortunately, Apache also records errors in the access logs, so using ErrorLog to feed messages to a log monitor still allows a human to review messages denoting errors.

4.3  An Extensible Worm Blocker/IDS for Apache

The 28 executable lines of SNOBOL4 in Listing 5 detect infection attempts from worms such as Code Red, Nimda, and sadmind/IIS and "blackhole" the attacking machine. Unlike the earlier example, however, this SNOBOL program completely parses, and validates the fields of, each Apache ErrorLog message before taking action. This eliminates any chance of a "false positive," which might occur if a regular expression in the earlier example happens to match part of a legitimate request.
 

* An Extensible worm blocker/IDS for Apache in SNOBOL4
* Copyright (c) 2001 by Brett Glass
* Licensing terms are at the end of this file.
*
* This program accepts the piped error output from the
* Apache Web server and spots lines indicating an attack
* from Nimda.A or similar worms, including Code Red,
* sadmind/IIS, and Nimda.E. It can then firewall or
* blackhole the attacking host. Add it to your Apache
* configuration by inserting a line such as
*
* ErrorLog "|exec snobol4 -b /usr/local/bin/wormblock.sno"
*
* Also, make sure that HostNameLookups is off so that the 
* log messages contain numeric IP addresses.
*
* This program is designed to be easily extensible to catch 
* a wide variety of potential exploits.
*
* An Apache error log message generated by the Nimda worm
* might look like this (wrapped for readability):
*
* [Thu Nov  1 12:46:07 2001] [error] [client 12.98.224.154]
*  File does not exist: /usr/local/www/data/textorics/scripts/
* ..%5c../winnt/system32/cmd.exe
*
* Build up SNOBOL patterns for Apache ErrorLog messages.
* Use the pattern "WS" for whitespace (tabs or blanks)
        WS = SPAN(' ' CHAR(9))
        DIGITS = '0123456789'
        WEEKDAY = 'Mon' | 'Tue' | 'Wed' | 'Thu' | 'Fri' | 'Sat' | 'Sun'
        MONTH = 'Jan' | 'Feb' | 'Mar' | 'Apr' | 'May' | 'Jun' |
+        'Jul' | 'Aug' | 'Sep' | 'Oct' | 'Nov' | 'Dec'
        DAYOFMONTH = (SPAN(DIGITS) $ NUMBER) *LE(NUMBER,31) *GE(NUMBER,1)
        HOUR = (SPAN(DIGITS) $ NUMBER) *LE(NUMBER,23)
        MINUTE = (SPAN(DIGITS) $ NUMBER) *LE(NUMBER,59)
        SECOND = MINUTE
        DAYTIME = HOUR ':' MINUTE ':' SECOND
        YEAR = SPAN(DIGITS)
        DATEANDTIME = '[' WEEKDAY WS MONTH WS DAYOFMONTH WS DAYTIME WS YEAR ']'
        OCTET = (SPAN(DIGITS) $ NUMBER) *LE(NUMBER,255)
        IPADDRESS = OCTET '.' OCTET '.' OCTET '.' OCTET
        ERRSTR = '[error]'
        CLIENTINFO = '[client' WS (IPADDRESS . CLIENTIP) ']'
        FILEERR = 'File does not exist:'
        FILENOTFOUNDERROR = DATEANDTIME WS ERRSTR WS CLIENTINFO WS
+        FILEERR WS REM . PATH
        DANGEROUSPATH = '/winnt/system32/cmd.exe' | '/scripts/root.exe' |
+        '/MSADC/root.exe' | "/.." | "../"

LOOP    LOGLINE = INPUT
* Anchor the matching of the error message for efficiency
        &ANCHOR = 1
* We're using unevaluated expressions ("thunks") and so must do full scans
        &FULLSCAN = 1
        LOGLINE FILENOTFOUNDERROR :F(LOOP)
* Scan the path, using an unanchored match, for strings betraying a worm
        &ANCHOR = 0
        &FULLSCAN = 0
        PATH DANGEROUSPATH :F(LOOP)
        HOST(1,'logger -t wormblock -pauth.notice Nimda or similar attack detected!'
+        'Blocking IP address ' CLIENTIP)
        HOST(1,'route -nq add -host ' CLIENTIP ' 127.0.0.1 -blackhole')
        :(LOOP)

* Note that a blackhole route is a brute force blocking method. It
* allows the first SYN to arrive but blocks the outgoing SYN-ACK,
* causing the TCP three-way handshake to fail. Its advantage is
* that it works on nearly any system regardless of configuration.

* If you're running a firewall, you can replace the route command
* above with ones that add firewall rules. Here are samples for
* FreeBSD's ipfw:
*       HOST(1,'/sbin/ipfw -q add deny all from any to ' CLIENTIP)
*       HOST(1,'/sbin/ipfw -q add deny all from ' CLIENTIP ' to any')
* The commands for ipf and pf are similar.

* If you want to block attacks at a different machine (say, the
* firewall that guards your entire network), you can use SSH to send
* the firewall similar commands. The exact commands required will
* depend upon your network and firewall configurations.

END

* Copyright (c) 2001 by Brett Glass

* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:

* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.

* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.

The example in Listing 5 was written for Philip Budne's free Macro SNOBOL4 for UNIX [22], which compiles on most BSD UNIX implementations and is present in the NetBSD and FreeBSD ports collections. It is readily portable to other implementations of the language including Catspaw SPITBOL [23]. The author has found SNOBOL4 to be extremely useful for log monitoring -- even more so than Perl -- because its extremely powerful recursive pattern matching allows it to completely parse a log message by executing a single line of code. SNOBOL4 also allows more extensive input validation to be done within a pattern than can easily be done within a Perl regular expression. For example, in the patterns DAYOFMONTH, HOURS, MINUTES, and SECONDS patterns, the GE() and LE() predicates are applied to strings of digits during pattern matching to ensure that the numbers they represent are within allowable limits. SNOBOL's pattern matching engine can backtrack (or indicate failure) if these conditions are not met.

4.4  Refinements to the Initial Design

Fortunately, it is relatively simple to make refinements to the log monitor shown above to handle these problems. A "do not block" list can be added to ensure that the machine does not block itself. By requiring two or more hits from an IP address before blocking it, the monitor can reduce the chances of an accidental block or of a block caused by a maliciously distributed link. Use of the MAPS Dial-up List (DUL) [24] to recognize dial-ups, plus an "amnesty" policy, can protect against long term blocking of dial-ups, though at the expense of a few more hits from worms.

Other refinements suggested during previous presentations of this work include:

• The ability to notify an administrator of the current block list (and/or "repeat offenders") so that s/he can notify administrators by phone or e-mail;
• The ability to mail or page an administrator when an attack is detected from a host on the "do not block" list (or under other conditions);
• The ability to gather statistics (such as the number of hits received from a particular Web address or subnet per hour) and automatically notice anomalies;
• The ability to place the log monitor on a separate machine, so as to preserve both copies of logs and information about attacks or malfunctions in the event of catastrophic system failure or tampering; and
• The ability to view a display and/or reports detailing the log monitor's actions.

5  Creating a Generalized Logging and Log Monitoring Facility

• Compatibility with the "legacy" logging facilities and facility/severity codes of current UNIX implementations;
• The ability to apply pre-written message parsing templates to messages (akin to the "distillation" process used by Lire [25] but performed in real time) so that rules can refer to message field by name;
• The ability to identify and report messages which were not parsed (possibly indicating an obsolete template and/or a software problem);
• The ability to access all information associated with a log message and the process that generated it -- including the identity of the program, effective user and group ids, facility and severity codes, point of origin (if not on the local system), etc.;
• Accumulation of statistics (e.g. number of e-mail messages received from a specific user or IP address) for use in rules;
• The ability to correlate log messages and statistics produced by different applications, e.g. a POP server and an SMTP server;
• The ability to generate one or more periodically refreshed displays (e.g. bar graphs) based on log statistics;
• The ability to query external databases such as DNS blacklists;
• The ability to maintain, save, and restore internal databases (e.g. of blocked hosts and times at which they were blocked) and report their contents at runtime;
• The ability to "fire" rules at specific times or intervals as well as in response to messages;
• The ability to send log messages to, and accept them on or from, arbitrary UDP or TCP ports;
• The ability to log to another machine via an encrypted connection (e.g. through SSH or SSL);
• Stronger authentication than that implemented in current versions of syslogd (most of which use source IP address and port number);
• Flexible notification facilities, including the ability to send notices via e-mail, pager, IRC, and instant messaging systems;
• The ability to issue commands to firewalls, routers, bridges, managed hubs, and remote power controllers; and
• The ability to allow or deny users access to facilities (e.g. by changing group memberships, changing a user's login shell to /etc/nologin, or removing and restoring passwords).

6  Conclusions

While a handful of "plug and play" log monitors now exist, none contain the features necessary to allow them to perform sophisticated stateful monitoring. The author's goal is to fill this gap by implementing a collection of complex log monitors and then creating a generalized facility which can subsume all of their functions.

To ease the implementation of log monitors, the logging facilities in different UNIX implementations -- which have diverged in subtle ways and often hide useful information from administrators and intelligent agents alike -- should be updated or replaced with a more modern scheme that is backward-compatible with what exists today. It will then be much easier to implement a generalized log monitoring facility that runs on a wide variety of platforms.

Of course, no log monitoring system can completely replace the insight or talents of a human administrator. As Bruce Schneier, founder of Counterpane Network Security, writes in a white paper posted at http://www.counterpane.com/msm.html:

Network attacks can be much more subtle than a broken window. Much depends on context. Software can filter the tens of megabytes of audit information a medium-sized network can generate in a day, but software is too easy for an attacker to fool. Intelligent alert requires people. People to analyze what the software finds suspicious. People to delve deeper into suspicious events, determining what is really going on. People to separate false alarms from real attacks. People who understand context. [26]

7  Acknowledgments

References

[2] Pacific Institute for Computer Security (PICS) research group, San Diego Supercomputer Center (SDSC). 2swatch. Software at URL:  ftp://ftp.sdsc.edu/pub/sdsc/security/PICS/2swatch/.

[3] Paul Vixie and contributors. cron. FreeBSD 5.0-current version documented at URL: http://www.FreeBSD.org/cgi/man.cgi?query=cron&apropos=0&sektion=8&manpath=FreeBSD+5.0-current&format=html.

[4] Paul Traina and Brian Somers. periodic. FreeBSD 5.0-current version documented at URL: http://www.FreeBSD.org/cgi/man.cgi?query=periodic&apropos=0&sektion=8&manpath=FreeBSD+5.0-current&format=html.

[5] Tom Boutell. Wusage. Software and documentation at URL:  http://www.boutell.com/wusage/.

[6] Kai Schlichting. Kai's Spamshield. Software and documentation at URL:  http://spamshield.conti.nu/.

[7] Eric Allman, The University of Califonia at Berkeley, The FreeBSD Project and contributors. syslogd. FreeBSD 5.0-current version documented at URL: http://www.FreeBSD.org/cgi/man.cgi?query=syslogd&apropos=0&sektion=8&manpath=FreeBSD+5.0-current&format=html.

[8] Eric Allman, University of California at Berkeley and contributors. syslogd. 4.4BSD Lite2 version documented at URL: http://www.FreeBSD.org/cgi/man.cgi?query=syslogd&apropos=0&sektion=8&manpath=4.4BSD+Lite2&format=html.

[9] Eric Allman, The University of Califonia at Berkeley, The FreeBSD Project and contributors. syslogd.conf. FreeBSD 5.0-current version documented at URL: http://www.FreeBSD.org/cgi/man.cgi?query=syslog.conf&apropos=0&sektion=5&manpath=FreeBSD+5.0-current&format=html.

[10] Matija Grabnar. File::Tail. Software at URL:  http://www.cpan.org/modules/by-module/File/File-Tail-0.98.tar.gz.

[11] BalaBit IT Ltd. syslog-ng. Software and documentation at URL: http://www.balabit.hu/en/downloads/syslog-ng/.

[12] Core-SDI. msyslog. Software and documentation at URL: http://community.corest.com/pub/msyslog/.

[13] The Apache Software Foundation. Apache HTTPD Server Project. Software and documentation at URL: http://www.apache.org/.

[14] Computer Emergency Response Team (CERT). Advisory CA-2001-26: Nimda Worm. At URL: http://www.cert.org/advisories/CA-2001-26.html.

[15] Computer Emergency Response Team (CERT). Advisory CA-2001-19: "Code Red" Worm Exploiting Buffer Overflow In IIS Indexing Service DLL. At URL: http://www.cert.org/advisories/CA-2001-19.html.

[16] Computer Emergency Response Team (CERT). Advisory CA-2001-11: sadmind/IIS Worm. At URL: http://www.cert.org/advisories/CA-2001-11.html.

[17] The Apache Software Foundation. mod_setenvif. Documentation at URL:  http://httpd.apache.org/docs/mod/mod_setenvif.html#setenvif.

[18] The Apache Software Foundation. mod_log_config. Documentation at URL: http://httpd.apache.org/docs/mod/mod_log_config.html#customlog

[19] The Apache Software Foundation. Apache Core Features. HostNameLookups directive. At URL: http://httpd.apache.org/docs/mod/core.html#hostnamelookups.

[20] R. E. Griswold, J. F. Poage, I. P. Polonsky. The SNOBOL4 Programming Language, 2nd Edition. Bell Telephone Laboratories/Prentice-Hall, 1971.

[21] Phil Budne. Phil's SNOBOL Resources Page. At URL: http://people.ne.mediaone.net/philbudne/snobol.html.

[22] Phil Budne. Macro Implementation of SNOBOL4 in C (C-MAINBOL). Software and documentation at URL:  http://people.ne.mediaone.net/philbudne/src.html#snobol

[23] Mark Emmer. Catspaw SPITBOL. Information at URL:  ftp://ftp.snobol4.com/specshet.pdf

[24] Mail Abuse Prevention System (mail-abuse.org). MAPS DUL Introduction. At URL: http://www.mail-abuse.org/dul/intro.htm

[25] LogReport Foundation. Report Production Line. At URL: http://www.logreport.org/documentation/about=architecture.

[26] Bruce Schneier. Managed Security Monitoring: Network Security for the 21st Century . At URL: http://www.counterpane.com/msm.html.