4. Pruning Services

Most operating systems seem to come loaded with services enabled out of the box; most of which you don’t need and therefore ought not to be running. Ubuntu, on the other hand, has a good policy of not shipping with any network services reachable from outside the machine by default, though naturally if you ask for a particular network service to be installed, it will be reachable.

In order to give us something interesting to look at in this section, you will first need to run the following “mystery commands” on Server1, which will install and configure some services for us to look at:

# apt-get install openbsd-inetd netcat-openbsd openssh-server
# sed -i -e 's/^#daytime/daytime/' /etc/inetd.conf
# /etc/init.d/openbsd-inetd restart

What that does, in short, is to install some software, which includes an “Internet Super-Server”[43], which we shall be using shortly; alter its configuration file a little to enable a particular service by uncommenting a line; and restart it, affecting the change. We also install the OpenBSD version of the useful netcat utility (the “TCP Swiss Army Knife”), as it again has much better IPv6 support than other versions of netcat[44]. Finally, we also installed the OpenSSH SSH service. We can now run lsof and see what is listening on the network:

# lsof -i
COMMAND    PID USER   FD TYPE … NODE NAME
dhclient3  294 root   4u IPv4 … UDP *:68
sshd       902 root   3u IPv4 … TCP *:ssh (LISTEN)
sshd       902 root   4u IPv6 … TCP *:ssh (LISTEN)
inetd     1275 root   4u IPv6 … TCP *:daytime (LISTEN)

Note

If you don’t see any lines output, check that you have run lsof with root privilege. Otherwise, it will only report on your own processes, of which you will most likely have none that are using network sockets.

dhclient3 is the DHCP client for the NAT network attachment (we’ll learn more about DHCP in a later lab). Likewise, we’ll look at that sshd entry a little later, but right now we want to have a look at the daytime entry, and find out more about it.

You’ll notice that inetd is handling the connections for the daytime service. The configuration file for inetd is /etc/inetd.conf. Before we disable anything in /etc/inetd.conf, we should know that a service is represented as a pair of protocol/port, where protocol is either tcp or udp, and port is the port number. This is a common way of specifying port numbers of services in documentation. To find the protocol and port number of a service, you can use the -Pni options to lsof. You can google protocol/port to find out information about the service.

It is useful also to look at what interfaces services are listening on, for that tells us much about where a service is being offered. For the daytime entry above, a * indicates it is listening on all IPv6 interfaces (furthermore, in a common dual-stack feature, because there is no IPv4 service on the same port number, IPv6 will also likely get IPv4 requests as well). If you see a particular hostname (or IP address if you are using the -n option to lsof), then it is only accepting connections on that interface. Commonly, services that need to be running, but don’t need to be remotely accessable by default will listen on 127.0.0.1 or ::1. Such an address is commonly shown as localhost, ip6-localhost or the canonical hostname of the machine, depending on the contents of /etc/hosts, which differs on different distributions.

inetd is configured via the file /etc/inetd.conf. An entry in that file is disabled either by commenting it out or removing it entirely. To make it easier for package installation scripts to be run, various configuration files can be managed using system-provided scripts, such as update-inetd[45]. This is easier than having to have each inetd-related package having to edit the file itself, separating policy and implementation. In this lab, we disable the “daytime” service manually by editing inetd.conf.

Once you have modified inetd.conf, you need to tell inetd that it has changed. This is generally done by the services startup script, such as by using the command /etc/init.d/openbsd-inetd reload.

Note

In summary, to disable or enable a service from inetd, you generally want to use a pattern of commands such as the following:

  1. vim

    Use vim or nano to edit inetd.conf and comment or uncomment the corresponding line of the service (judging by protocol/port of the service).

  2. pstree

    Use this to verify that the right number of daemon processes is running. In the case of inetd, there should only be one inetd process in a quiescent system (meaning the system is not serving any requests.)

  3. sudo /etc/init.d/openbsd-inetd stop

    Use this to shut down the service.

  4. sudo killall inetd

    You can use this to kill off any remaining inetd processes so that processes get a second chance to terminate gracefully. Give a little time for them to be shut down first though.

  5. pstree

    Has it gone yet? If not, use sudo killall -KILL inetd to kill it forcefully. Check that it has really died this time with another pstree.

  6. sudo /etc/init.d/openbsd-inetd start

    This will start the service again.

  7. pstree and lsof -ni, to verify that the process is running, and listening as expected.

  8. Check your system logs, typically in /var/log/syslog. You can use the tail command to view the end of this, but beware that viewing logs using tail exposes a security weakness[46]

Following this general procedure throughout the rest of the course will greatly help you in diagnosing problems, and save yourself a lot of time in the future.

Now that we’ve disabled the “daytime” service, let’s just ensure that it is no longer available. Here, We’re using a particular invocation of lsof that just tells us about things on the “daytime” port:

# lsof -Pni:daytime

Because there is no output reported (and yes, we are running this with root privilege), we can infer that nothing is listening on the “daytime” port.

What about that SSH service?

Here is what is currently listening:

# lsof -Pni
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
dhclient3 …
sshd      9723 root    3u  IPv4  23591      0t0  TCP *:22 (LISTEN)
sshd      9723 root    4u  IPv6  23593      0t0  TCP *:22 (LISTEN)

Let’s assume that we don’t need the SSH service; how best to disable it? Unlike the “daytime” service, which was run by inetd, the SSH service is run by the sshd process. We have a number of options available to disable the service, listed in no particular order of suitability:

  1. Remove the links from under /etc/rc*.d/Snnname that start cause the associated script in /etc/init.d/name to be run.

    This is generally the best way to disable a background service either permanently or have it not running by default. It can still be run by hand by calling /etc/init.d/name start etc.

    There is generally a tool to help you manage the contents of /etc/rc*.d/. On Debian-based systems, you can use the update-rc.d script[47].

  2. Uninstall the package, retaining its configuration files. This can be done using apt-get autoremove packagename. This is useful if you think you may want to keep the configuration files and data files for a later time. It also means the program cannot be (accidentally or otherwise) run. Note that we have used autoremove instead of remove, which removes any other packages that were installed to satisfy a dependency that are no-longer needed.

    This is generally the best way to remove a service if it doesn’t need to be run anymore. One particular risk is that the package could accidentally be installed at a later date if it gets installed due to installation of other packages via package dependency.

  3. Uninstall the package, purging its configuration files. This can be done using apt-get autoremove --purge packagename. This is useful when you want to permanently remove a package, or you don’t want stale (possibly broken) configuration files hanging around, which might confuse issues later if you decide to re-install the service. Your original configuration files should be in backup and ideally in version control before the uninstallation.

In our case, we don’t want the SSH service installed at all, so we shall practice removing the package (in a later lab, we shall install it when we need it).

To make things a little more real for a beginning administrator, let’s assume that we don’t know what package contains this sshd process. Where is this sshd process anyway?

$ ps -eo command | grep sshd
/usr/sbin/sshd
…

Okay, so it seems that the sshd mentioned in the lsof output above is actually /usr/sbin/sshd. What package contains that file?

$ dpkg -S /usr/sbin/sshd
openssh-server: /usr/sbin/sshd

Now we know the package name, we should be able to remove it:

# apt-get autoremove openssh-server

Verifying that it is no-longer running:

# lsof -Pni:22
No output, therefore nothing listening on port 22 (ssh)


[43] This particular version comes from OpenBSD, and supports IPv6 much better than the other available version: Inetutils-inetd.

[44] As an example, the netcat6 package doesn’t allow you to specify an IPv6 address to connect to, only a hostname.

[45] This would replace the call to sed we used earlier, and would be expected to be more robust.

[46] Discussed in Hacking Linux Exposed. You should instead create an alias in your ~/.bashrc, such as alias vlog='sudo less --follow-name +G +F'.

[47] On Redhat systems, the equivalent is chkconfig.