It’s a fact of life, logs grow. Just like your lawn, the logs need processed every so often to keep them manageable, and to help you find any snakes in the grass. What you do with your old logs will reflect on your local policies, and possibly even laws eg. logs may need to be kept for several years, or on the other hand, logs perhaps may not be kept for longer than a few years due to privacy laws; you should seek advice from your lawyer or industry.
In Linux distributions, logs are usually rotated using a
tool called logrotate. Have a look at the
default Ubuntu configuration of logrotate by
looking in the file /etc/logrotate.conf. Note
that it includes all the files in
/etc/logrotate.d/, to enable
package-maintainers to easily install rules for having
logrotate rotate its logs, simply by dropping a
file into the logrotate.d/ directory. Here is
what the /etc/logrotate.conf file looks
like. The file has been edited to save space.
Specifying global defaults weekly Rotate logs weekly rotate 4 Keep 4 weeks worth of backlogs create Create new (empty) log files after rotating old ones #compress Uncomment to compress rotated files A directory so packages can install log rotation policies. include /etc/logrotate.d/ /var/log/wtmp { Stores login history. missingok Don’t complain if wtmp file is missing. monthly create 0664 root utmp perm user group rotate 1 }
The above is a simple example; we could do a lot more. We’ve
selected the condition to rotate based on time
(weekly and monthly), but we
could also choose to rotate based on file size. You can also
specify commands to be run before and after rotating the log
file. Here is a fictitious entry which shows some of the other
useful entries. This could be a file in
/etc/logrotate.d/ Remember that the following
is an example, so I don’t expect you to input
it.
You can specify multiple files /var/log/foo/access /var/log/foo/errors { size=100k Rotate when it reaches a certain size sharedscripts Run post and prerotate only once for all files in this set postrotate You can run a list of commands after rotation killall -HUP food food would be the daemon for the foo service endscript End the list using endscript }
logrotate gets run by
cron, using
/etc/cron.daily/logrotate. The scripts in
cron.daily get run early each morning,
typically about 6am. You can alter this by editing
/etc/crontab.
You can force logrotate to rotate
(ie. ignoring the selection tags such as
weekly or size) the files
by using the -f argument to
logrotate. Have a look, using ls
-l, in the /var/log directory, and
then run the command logrotate -f
/etc/logrotate.conf. Have a look to see what
changed. Repeat a few times, and describe what happens.
Although Ubuntu doesn’t compress them by default,
logrotate is able to compress logs using
gzip, and thus get an extension of
.gz. Compressed logs can be viewed using the
zless or zcat program, and
grepped using zgrep (we’ll cover
grep later in the lab.