admin.jugamanusia.com

FreeBSD Handbook - Chapter 29 Network Servers

January 6th, 2009

No comments

Warning: file_get_contents(http://ecs.amazonaws.com/onca/xml?Service=AWSECommerceService&AWSAccessKeyId=1TJ8QTQ6ZFCVAJ3X1T02&AssociateTag=ii0c3-20&Operation=ItemSearch&SearchIndex=Books&ResponseGroup=Small,Images&Keywords=account) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 400 Bad Request in /home/manusia2/public_html/wp-content/plugins/amazonfeed/php/amazonfeed.class.php on line 271

Reorganized by Murray Stokely.

29.1 Synopsis

This chapter will cover some of the more frequently used network services on UNIX systems. We will cover how to install, configure, test, and maintain many different types of network services. Example configuration files are included throughout this chapter for you to benefit from.

After reading this chapter, you will know:

How to manage the inetd daemon.
How to set up a network file system.
How to set up a network information server for sharing user accounts.
How to set up automatic network settings using DHCP.
How to set up a domain name server.
How to set up the Apache HTTP Server.
How to set up a File Transfer Protocol (FTP) Server.
How to set up a file and print server for Windows clients using Samba.
How to synchronize the time and date, and set up a time server, with the NTP protocol.
How to configure the standard logging daemon, syslogd, to accept logs from remote hosts.

Before reading this chapter, you should:

Understand the basics of the /etc/rc scripts.
Be familiar with basic network terminology.
Know how to install additional third-party software (Chapter 4).

29.2 The inetd “Super-Server”

Contributed by Chern Lee. Updated for FreeBSD 6.1-RELEASE by The FreeBSD Documentation Project.

29.2.1 Overview

inetd(8) is sometimes referred to as the “Internet Super-Server” because it manages connections for several services. When a connection is received by inetd, it determines which program the connection is destined for, spawns the particular process and delegates the socket to it (the program is invoked with the service socket as its standard input, output and error descriptors). Running inetd for servers that are not heavily used can reduce the overall system load, when compared to running each daemon individually in stand-alone mode.

Primarily, inetd is used to spawn other daemons, but several trivial protocols are handled directly, such as chargen, auth, and daytime.

This section will cover the basics in configuring inetd through its command-line options and its configuration file, /etc/inetd.conf.

29.2.2 Settings

inetd is initialized through the rc(8) system. The inetd_enable option is set to NO by default, but may be turned on by sysinstall during installation, depending on the configuration chosen by the user. Placing:

inetd_enable="YES"

inetd_enable="NO"

into /etc/rc.conf will enable or disable inetd starting at boot time. The command:

# /etc/rc.d/inetd rcvar

can be run to display the current effective setting.

Additionally, different command-line options can be passed to inetd via the inetd_flags option.

29.2.3 Command-Line Options

Like most server daemons, inetd has a number of options that it can be passed in order to modify its behaviour. The full list of options reads:

inetd [-d] [-l] [-w] [-W] [-c maximum] [-C rate] [-a address | hostname] [-p filename] [-R rate] [-s maximum] [configuration file]

Options can be passed to inetd using the inetd_flags option in /etc/rc.conf. By default, inetd_flags is set to -wW -C 60, which turns on TCP wrapping for inetd’s services, and prevents any single IP address from requesting any service more than 60 times in any given minute.

Novice users may be pleased to note that these parameters usually do not need to be modified, although we mention the rate-limiting options below as they be useful should you find that you are receiving an excessive amount of connections. A full list of options can be found in the inetd(8) manual.

-c maximum

Specify the default maximum number of simultaneous invocations of each service; the default is unlimited. May be overridden on a per-service basis with the max-child parameter.

-C rate

Specify the default maximum number of times a service can be invoked from a single IP address in one minute; the default is unlimited. May be overridden on a per-service basis with the max-connections-per-ip-per-minute parameter.

-R rate

Specify the maximum number of times a service can be invoked in one minute; the default is 256. A rate of 0 allows an unlimited number of invocations.

-s maximum

Specify the maximum number of times a service can be invoked from a single IP address at any one time; the default is unlimited. May be overridden on a per-service basis with the max-child-per-ip parameter.

29.2.4 `inetd.conf`

Configuration of inetd is done via the file /etc/inetd.conf.

When a modification is made to /etc/inetd.conf, inetd can be forced to re-read its configuration file by running the command:

Example 29-1. Reloading the inetd configuration file

# /etc/rc.d/inetd reload

Each line of the configuration file specifies an individual daemon. Comments in the file are preceded by a “#”. The format of each entry in /etc/inetd.conf is as follows:

service-name
socket-type
protocol
{wait|nowait}[/max-child[/max-connections-per-ip-per-minute[/max-child-per-ip]]]
user[:group][/login-class]
server-program
server-program-arguments

An example entry for the ftpd(8) daemon using IPv4 might read:

ftp     stream  tcp     nowait  root    /usr/libexec/ftpd       ftpd -l

service-name

This is the service name of the particular daemon. It must correspond to a service listed in /etc/services. This determines which port inetd must listen to. If a new service is being created, it must be placed in /etc/services first.

socket-type

Either stream, dgram, raw, or seqpacket. stream must be used for connection-based, TCP daemons, while dgram is used for daemons utilizing the UDP transport protocol.

protocol

One of the following:

Protocol

Explanation

tcp, tcp4

TCP IPv4

udp, udp4

UDP IPv4

tcp6

TCP IPv6

udp6

UDP IPv6

tcp46

Both TCP IPv4 and v6

udp46

Both UDP IPv4 and v6

{wait|nowait}[/max-child[/max-connections-per-ip-per-minute[/max-child-per-ip]]]

wait|nowait indicates whether the daemon invoked from inetd is able to handle its own socket or not. dgram socket types must use the wait option, while stream socket daemons, which are usually multi-threaded, should use nowait. wait usually hands off multiple sockets to a single daemon, while nowait spawns a child daemon for each new socket.

The maximum number of child daemons inetd may spawn can be set using the max-child option. If a limit of ten instances of a particular daemon is needed, a /10 would be placed after nowait. Specifying /0 allows an unlimited number of children

In addition to max-child, two other options which limit the maximum connections from a single place to a particular daemon can be enabled. max-connections-per-ip-per-minute limits the number of connections from any particular IP address per minutes, e.g. a value of ten would limit any particular IP address connecting to a particular service to ten attempts per minute. max-child-per-ip limits the number of children that can be started on behalf on any single IP address at any moment. These options are useful to prevent intentional or unintentional excessive resource consumption and Denial of Service (DoS) attacks to a machine.

In this field, either of wait or nowait is mandatory. max-child, max-connections-per-ip-per-minute and max-child-per-ip are optional.

A stream-type multi-threaded daemon without any max-child, max-connections-per-ip-per-minute or max-child-per-ip limits would simply be: nowait.

The same daemon with a maximum limit of ten daemons would read: nowait/10.

The same setup with a limit of twenty connections per IP address per minute and a maximum total limit of ten child daemons would read: nowait/10/20.

These options are utilized by the default settings of the fingerd(8) daemon, as seen here:

finger stream  tcp     nowait/3/10 nobody /usr/libexec/fingerd fingerd -s

Finally, an example of this field with a maximum of 100 children in total, with a maximum of 5 for any one IP address would read: nowait/100/0/5.

user

This is the username that the particular daemon should run as. Most commonly, daemons run as the root user. For security purposes, it is common to find some servers running as the daemon user, or the least privileged nobody user.

server-program

The full path of the daemon to be executed when a connection is received. If the daemon is a service provided by inetd internally, then internal should be used.

server-program-arguments

This works in conjunction with server-program by specifying the arguments, starting with argv[0], passed to the daemon on invocation. If mydaemon -d is the command line, mydaemon -d would be the value of server-program-arguments. Again, if the daemon is an internal service, use internal here.

29.2.5 Security

Depending on the choices made at install time, many of inetd’s services may be enabled by default. If there is no apparent need for a particular daemon, consider disabling it. Place a “#” in front of the daemon in question in /etc/inetd.conf, and then reload the inetd configuration. Some daemons, such as fingerd, may not be desired at all because they provide information that may be useful to an attacker.

Some daemons are not security-conscious and have long, or non-existent, timeouts for connection attempts. This allows an attacker to slowly send connections to a particular daemon, thus saturating available resources. It may be a good idea to place max-connections-per-ip-per-minute, max-child or max-child-per-ip limitations on certain daemons if you find that you have too many connections.

By default, TCP wrapping is turned on. Consult the hosts_access(5) manual page for more information on placing TCP restrictions on various inetd invoked daemons.

29.2.6 Miscellaneous

daytime, time, echo, discard, chargen, and auth are all internally provided services of inetd.

The auth service provides identity network services, and is configurable to a certain degree, whilst the others are simply on or off.

Consult the inetd(8) manual page for more in-depth information.

29.3 Network File System (NFS)

Reorganized and enhanced by Tom Rhodes. Written by Bill Swingle.

Among the many different file systems that FreeBSD supports is the Network File System, also known as NFS. NFS allows a system to share directories and files with others over a network. By using NFS, users and programs can access files on remote systems almost as if they were local files.

Some of the most notable benefits that NFS can provide are:

Local workstations use less disk space because commonly used data can be stored on a single machine and still remain accessible to others over the network.
There is no need for users to have separate home directories on every network machine. Home directories could be set up on the NFS server and made available throughout the network.
Storage devices such as floppy disks, CDROM drives, and Zip® drives can be used by other machines on the network. This may reduce the number of removable media drives throughout the network.

29.3.1 How NFS Works

NFS consists of at least two main parts: a server and one or more clients. The client remotely accesses the data that is stored on the server machine. In order for this to function properly a few processes have to be configured and running.

The server has to be running the following daemons:

Daemon

Description

nfsd

The NFS daemon which services requests from the NFS clients.

mountd

The NFS mount daemon which carries out the requests that nfsd(8) passes on to it.

rpcbind

This daemon allows NFS clients to discover which port the NFS server is using.

The client can also run a daemon, known as nfsiod. The nfsiod daemon services the requests from the NFS server. This is optional, and improves performance, but is not required for normal and correct operation. See the nfsiod(8) manual page for more information.

29.3.2 Configuring NFS

NFS configuration is a relatively straightforward process. The processes that need to be running can all start at boot time with a few modifications to your /etc/rc.conf file.

On the NFS server, make sure that the following options are configured in the /etc/rc.conf file:

rpcbind_enable="YES"
nfs_server_enable="YES"
mountd_flags="-r"

mountd runs automatically whenever the NFS server is enabled.

On the client, make sure this option is present in /etc/rc.conf:

nfs_client_enable="YES"

The /etc/exports file specifies which file systems NFS should export (sometimes referred to as “share”). Each line in /etc/exports specifies a file system to be exported and which machines have access to that file system. Along with what machines have access to that file system, access options may also be specified. There are many such options that can be used in this file but only a few will be mentioned here. You can easily discover other options by reading over the exports(5) manual page.

Here are a few example /etc/exports entries:

The following examples give an idea of how to export file systems, although the settings may be different depending on your environment and network configuration. For instance, to export the /cdrom directory to three example machines that have the same domain name as the server (hence the lack of a domain name for each) or have entries in your /etc/hosts file. The -ro flag makes the exported file system read-only. With this flag, the remote system will not be able to write any changes to the exported file system.

/cdrom -ro host1 host2 host3

The following line exports /home to three hosts by IP address. This is a useful setup if you have a private network without a DNS server configured. Optionally the /etc/hosts file could be configured for internal hostnames; please review hosts(5) for more information. The -alldirs flag allows the subdirectories to be mount points. In other words, it will not mount the subdirectories but permit the client to mount only the directories that are required or needed.

/home  -alldirs  10.0.0.2 10.0.0.3 10.0.0.4

The following line exports /a so that two clients from different domains may access the file system. The -maproot=root flag allows the root user on the remote system to write data on the exported file system as root. If the -maproot=root flag is not specified, then even if a user has root access on the remote system, he will not be able to modify files on the exported file system.

/a  -maproot=root  host.example.com box.example.org

In order for a client to access an exported file system, the client must have permission to do so. Make sure the client is listed in your /etc/exports file.

In /etc/exports, each line represents the export information for one file system to one host. A remote host can only be specified once per file system, and may only have one default entry. For example, assume that /usr is a single file system. The following /etc/exports would be invalid:

# Invalid when /usr is one file system
/usr/src   client
/usr/ports client

One file system, /usr, has two lines specifying exports to the same host, client. The correct format for this situation is:

/usr/src /usr/ports  client

The properties of one file system exported to a given host must all occur on one line. Lines without a client specified are treated as a single host. This limits how you can export file systems, but for most people this is not an issue.

The following is an example of a valid export list, where /usr and /exports are local file systems:

# Export src and ports to client01 and client02, but only
# client01 has root privileges on it
/usr/src /usr/ports -maproot=root    client01
/usr/src /usr/ports               client02
# The client machines have root and can mount anywhere
# on /exports. Anyone in the world can mount /exports/obj read-only
/exports -alldirs -maproot=root      client01 client02
/exports/obj -ro

The mountd daemon must be forced to recheck the /etc/exports file whenever it has been modified, so the changes can take effect. This can be accomplished either by sending a HUP signal to the running daemon:

# kill -HUP `cat /var/run/mountd.pid`

or by invoking the mountd rc(8) script with the appropriate parameter:

# /etc/rc.d/mountd onereload

Please refer to Section 11.7 for more information about using rc scripts.

Alternatively, a reboot will make FreeBSD set everything up properly. A reboot is not necessary though. Executing the following commands as root should start everything up.

On the NFS server:

# rpcbind
# nfsd -u -t -n 4
# mountd -r

On the NFS client:

# nfsiod -n 4

Now everything should be ready to actually mount a remote file system. In these examples the server’s name will be server and the client’s name will be client. If you only want to temporarily mount a remote file system or would rather test the configuration, just execute a command like this as root on the client:

# mount server:/home /mnt

This will mount the /home directory on the server at /mnt on the client. If everything is set up correctly you should be able to enter /mnt on the client and see all the files that are on the server.

If you want to automatically mount a remote file system each time the computer boots, add the file system to the /etc/fstab file. Here is an example:

server:/home   /mnt    nfs rw  0   0

The fstab(5) manual page lists all the available options.

29.3.3 Locking

Some applications (e.g. mutt) require file locking to operate correctly. In the case of NFS, rpc.lockd can be used for file locking. To enable it, add the following to the /etc/rc.conf file on both client and server (it is assumed that the NFS client and server are configured already):

rpc_lockd_enable="YES"
rpc_statd_enable="YES"

Start the application by using:

# /etc/rc.d/lockd start
# /etc/rc.d/statd start

If real locking between the NFS clients and NFS server is not required, it is possible to let the NFS client do locking locally by passing -L to mount_nfs(8). Refer to the mount_nfs(8) manual page for further details.

29.3.4 Practical Uses

NFS has many practical uses. Some of the more common ones are listed below:

Set several machines to share a CDROM or other media among them. This is cheaper and often a more convenient method to install software on multiple machines.
On large networks, it might be more convenient to configure a central NFS server in which to store all the user home directories. These home directories can then be exported to the network so that users would always have the same home directory, regardless of which workstation they log in to.
Several machines could have a common /usr/ports/distfiles directory. That way, when you need to install a port on several machines, you can quickly access the source without downloading it on each machine.

29.3.5 Automatic Mounts with amd

Contributed by Wylie Stilwell. Rewritten by Chern Lee.

amd(8) (the automatic mounter daemon) automatically mounts a remote file system whenever a file or directory within that file system is accessed. Filesystems that are inactive for a period of time will also be automatically unmounted by amd. Using amd provides a simple alternative to permanent mounts, as permanent mounts are usually listed in /etc/fstab.

amd operates by attaching itself as an NFS server to the /host and /net directories. When a file is accessed within one of these directories, amd looks up the corresponding remote mount and automatically mounts it. /net is used to mount an exported file system from an IP address, while /host is used to mount an export from a remote hostname.

An access to a file within /host/foobar/usr would tell amd to attempt to mount the /usr export on the host foobar.

Example 29-2. Mounting an Export with amd

You can view the available mounts of a remote host with the showmount command. For example, to view the mounts of a host named foobar, you can use:

% showmount -e foobar
Exports list on foobar:
/usr                               10.10.10.0
/a                                 10.10.10.0
% cd /host/foobar/usr

As seen in the example, the showmount shows /usr as an export. When changing directories to /host/foobar/usr, amd attempts to resolve the hostname foobar and automatically mount the desired export.

amd can be started by the startup scripts by placing the following lines in /etc/rc.conf:

amd_enable="YES"

Additionally, custom flags can be passed to amd from the amd_flags option. By default, amd_flags is set to:

amd_flags="-a /.amd_mnt -l syslog /host /etc/amd.map /net /etc/amd.map"

The /etc/amd.map file defines the default options that exports are mounted with. The /etc/amd.conf file defines some of the more advanced features of amd.

Consult the amd(8) and amd.conf(5) manual pages for more information.

29.3.6 Problems Integrating with Other Systems

Contributed by John Lind.

Certain Ethernet adapters for ISA PC systems have limitations which can lead to serious network problems, particularly with NFS. This difficulty is not specific to FreeBSD, but FreeBSD systems are affected by it.

The problem nearly always occurs when (FreeBSD) PC systems are networked with high-performance workstations, such as those made by Silicon Graphics, Inc., and Sun Microsystems, Inc. The NFS mount will work fine, and some operations may succeed, but suddenly the server will seem to become unresponsive to the client, even though requests to and from other systems continue to be processed. This happens to the client system, whether the client is the FreeBSD system or the workstation. On many systems, there is no way to shut down the client gracefully once this problem has manifested itself. The only solution is often to reset the client, because the NFS situation cannot be resolved.

Though the “correct” solution is to get a higher performance and capacity Ethernet adapter for the FreeBSD system, there is a simple workaround that will allow satisfactory operation. If the FreeBSD system is the server, include the option -w=1024 on the mount from the client. If the FreeBSD system is the client, then mount the NFS file system with the option -r=1024. These options may be specified using the fourth field of the fstab entry on the client for automatic mounts, or by using the -o parameter of the mount(8) command for manual mounts.

It should be noted that there is a different problem, sometimes mistaken for this one, when the NFS servers and clients are on different networks. If that is the case, make certain that your routers are routing the necessary UDP information, or you will not get anywhere, no matter what else you are doing.

In the following examples, fastws is the host (interface) name of a high-performance workstation, and freebox is the host (interface) name of a FreeBSD system with a lower-performance Ethernet adapter. Also, /sharedfs will be the exported NFS file system (see exports(5)), and /project will be the mount point on the client for the exported file system. In all cases, note that additional options, such as hard or soft and bg may be desirable in your application.

Examples for the FreeBSD system (freebox) as the client in /etc/fstab on freebox:

fastws:/sharedfs /project nfs rw,-r=1024 0 0

As a manual mount command on freebox:

# mount -t nfs -o -r=1024 fastws:/sharedfs /project

Examples for the FreeBSD system as the server in /etc/fstab on fastws:

freebox:/sharedfs /project nfs rw,-w=1024 0 0

As a manual mount command on fastws:

# mount -t nfs -o -w=1024 freebox:/sharedfs /project

Nearly any 16-bit Ethernet adapter will allow operation without the above restrictions on the read or write size.

For anyone who cares, here is what happens when the failure occurs, which also explains why it is unrecoverable. NFS typically works with a “block” size of 8 K (though it may do fragments of smaller sizes). Since the maximum Ethernet packet is around 1500 bytes, the NFS “block” gets split into multiple Ethernet packets, even though it is still a single unit to the upper-level code, and must be received, assembled, and acknowledged as a unit. The high-performance workstations can pump out the packets which comprise the NFS unit one right after the other, just as close together as the standard allows. On the smaller, lower capacity cards, the later packets overrun the earlier packets of the same unit before they can be transferred to the host and the unit as a whole cannot be reconstructed or acknowledged. As a result, the workstation will time out and try again, but it will try again with the entire 8 K unit, and the process will be repeated, ad infinitum.

By keeping the unit size below the Ethernet packet size limitation, we ensure that any complete Ethernet packet received can be acknowledged individually, avoiding the deadlock situation.

Overruns may still occur when a high-performance workstations is slamming data out to a PC system, but with the better cards, such overruns are not guaranteed on NFS “units”. When an overrun occurs, the units affected will be retransmitted, and there will be a fair chance that they will be received, assembled, and acknowledged.

29.4 Network Information System (NIS/YP)

Written by Bill Swingle. Enhanced by Eric Ogren and Udo Erdelhoff.

29.4.1 What Is It?

NIS, which stands for Network Information Services, was developed by Sun Microsystems to centralize administration of UNIX (originally SunOS) systems. It has now essentially become an industry standard; all major UNIX like systems (Solaris, HP-UX, AIX®, Linux, NetBSD, OpenBSD, FreeBSD, etc) support NIS.

NIS was formerly known as Yellow Pages, but because of trademark issues, Sun changed the name. The old term (and yp) is still often seen and used.

It is a RPC-based client/server system that allows a group of machines within an NIS domain to share a common set of configuration files. This permits a system administrator to set up NIS client systems with only minimal configuration data and add, remove or modify configuration data from a single location.

It is similar to the Windows NT® domain system; although the internal implementation of the two are not at all similar, the basic functionality can be compared.

29.4.2 Terms/Processes You Should Know

There are several terms and several important user processes that you will come across when attempting to implement NIS on FreeBSD, whether you are trying to create an NIS server or act as an NIS client:

Term

Description

NIS domainname

An NIS master server and all of its clients (including its slave servers) have a NIS domainname. Similar to an Windows NT domain name, the NIS domainname does not have anything to do with DNS.

rpcbind

Must be running in order to enable RPC (Remote Procedure Call, a network protocol used by NIS). If rpcbind is not running, it will be impossible to run an NIS server, or to act as an NIS client.

ypbind

“Binds” an NIS client to its NIS server. It will take the NIS domainname from the system, and using RPC, connect to the server. ypbind is the core of client-server communication in an NIS environment; if ypbind dies on a client machine, it will not be able to access the NIS server.

ypserv

Should only be running on NIS servers; this is the NIS server process itself. If ypserv(8) dies, then the server will no longer be able to respond to NIS requests (hopefully, there is a slave server to take over for it). There are some implementations of NIS (but not the FreeBSD one), that do not try to reconnect to another server if the server it used before dies. Often, the only thing that helps in this case is to restart the server process (or even the whole server) or the ypbind process on the client.

rpc.yppasswdd

Another process that should only be running on NIS master servers; this is a daemon that will allow NIS clients to change their NIS passwords. If this daemon is not running, users will have to login to the NIS master server and change their passwords there.

29.4.3 How Does It Work?

There are three types of hosts in an NIS environment: master servers, slave servers, and clients. Servers act as a central repository for host configuration information. Master servers hold the authoritative copy of this information, while slave servers mirror this information for redundancy. Clients rely on the servers to provide this information to them.

Information in many files can be shared in this manner. The master.passwd, group, and hosts files are commonly shared via NIS. Whenever a process on a client needs information that would normally be found in these files locally, it makes a query to the NIS server that it is bound to instead.

29.4.3.1 Machine Types

A NIS master server. This server, analogous to a Windows NT primary domain controller, maintains the files used by all of the NIS clients. The passwd, group, and other various files used by the NIS clients live on the master server.

Note: It is possible for one machine to be an NIS master server for more than one NIS domain. However, this will not be covered in this introduction, which assumes a relatively small-scale NIS environment.
NIS slave servers. Similar to the Windows NT backup domain controllers, NIS slave servers maintain copies of the NIS master’s data files. NIS slave servers provide the redundancy, which is needed in important environments. They also help to balance the load of the master server: NIS Clients always attach to the NIS server whose response they get first, and this includes slave-server-replies.
NIS clients. NIS clients, like most Windows NT workstations, authenticate against the NIS server (or the Windows NT domain controller in the Windows NT workstations case) to log on.

29.4.4 Using NIS/YP

This section will deal with setting up a sample NIS environment.

29.4.4.1 Planning

Let us assume that you are the administrator of a small university lab. This lab, which consists of 15 FreeBSD machines, currently has no centralized point of administration; each machine has its own /etc/passwd and /etc/master.passwd. These files are kept in sync with each other only through manual intervention; currently, when you add a user to the lab, you must run adduser on all 15 machines. Clearly, this has to change, so you have decided to convert the lab to use NIS, using two of the machines as servers.

Therefore, the configuration of the lab now looks something like:

Machine name

IP address

Machine role

ellington

10.0.0.2

NIS master

coltrane

10.0.0.3

NIS slave

basie

10.0.0.4

Faculty workstation

bird

10.0.0.5

Client machine

cli[1-11]

10.0.0.[6-17]

Other client machines

If you are setting up a NIS scheme for the first time, it is a good idea to think through how you want to go about it. No matter what the size of your network, there are a few decisions that need to be made.

29.4.4.1.1 Choosing a NIS Domain Name

This might not be the “domainname” that you are used to. It is more accurately called the “NIS domainname”. When a client broadcasts its requests for info, it includes the name of the NIS domain that it is part of. This is how multiple servers on one network can tell which server should answer which request. Think of the NIS domainname as the name for a group of hosts that are related in some way.

Some organizations choose to use their Internet domainname for their NIS domainname. This is not recommended as it can cause confusion when trying to debug network problems. The NIS domainname should be unique within your network and it is helpful if it describes the group of machines it represents. For example, the Art department at Acme Inc. might be in the “acme-art” NIS domain. For this example, assume you have chosen the name test-domain.

However, some operating systems (notably SunOS) use their NIS domain name as their Internet domain name. If one or more machines on your network have this restriction, you must use the Internet domain name as your NIS domain name.

29.4.4.1.2 Physical Server Requirements

There are several things to keep in mind when choosing a machine to use as a NIS server. One of the unfortunate things about NIS is the level of dependency the clients have on the server. If a client cannot contact the server for its NIS domain, very often the machine becomes unusable. The lack of user and group information causes most systems to temporarily freeze up. With this in mind you should make sure to choose a machine that will not be prone to being rebooted regularly, or one that might be used for development. The NIS server should ideally be a stand alone machine whose sole purpose in life is to be an NIS server. If you have a network that is not very heavily used, it is acceptable to put the NIS server on a machine running other services, just keep in mind that if the NIS server becomes unavailable, it will affect all of your NIS clients adversely.

29.4.4.2 NIS Servers

The canonical copies of all NIS information are stored on a single machine called the NIS master server. The databases used to store the information are called NIS maps. In FreeBSD, these maps are stored in /var/yp/[domainname] where [domainname] is the name of the NIS domain being served. A single NIS server can support several domains at once, therefore it is possible to have several such directories, one for each supported domain. Each domain will have its own independent set of maps.

NIS master and slave servers handle all NIS requests with the ypserv daemon. ypserv is responsible for receiving incoming requests from NIS clients, translating the requested domain and map name to a path to the corresponding database file and transmitting data from the database back to the client.

29.4.4.2.1 Setting Up a NIS Master Server

Setting up a master NIS server can be relatively straight forward, depending on your needs. FreeBSD comes with support for NIS out-of-the-box. All you need is to add the following lines to /etc/rc.conf, and FreeBSD will do the rest for you.

```
nisdomainname="test-domain"
```
This line will set the NIS domainname to test-domain upon network setup (e.g. after reboot).
```
nis_server_enable="YES"
```
This will tell FreeBSD to start up the NIS server processes when the networking is next brought up.
```
nis_yppasswdd_enable="YES"
```
This will enable the rpc.yppasswdd daemon which, as mentioned above, will allow users to change their NIS password from a client machine.

Note: Depending on your NIS setup, you may need to add further entries. See the section about NIS servers that are also NIS clients, below, for details.

Now, all you have to do is to run the command /etc/netstart as superuser. It will set up everything for you, using the values you defined in /etc/rc.conf.

29.4.4.2.2 Initializing the NIS Maps

The NIS maps are database files, that are kept in the /var/yp directory. They are generated from configuration files in the /etc directory of the NIS master, with one exception: the /etc/master.passwd file. This is for a good reason, you do not want to propagate passwords to your root and other administrative accounts to all the servers in the NIS domain. Therefore, before we initialize the NIS maps, you should:

# cp /etc/master.passwd /var/yp/master.passwd
# cd /var/yp
# vi master.passwd

You should remove all entries regarding system accounts (bin, tty, kmem, games, etc), as well as any accounts that you do not want to be propagated to the NIS clients (for example root and any other UID 0 (superuser) accounts).

Note: Make sure the /var/yp/master.passwd is neither group nor world readable (mode 600)! Use the chmod command, if appropriate.

When you have finished, it is time to initialize the NIS maps! FreeBSD includes a script named ypinit to do this for you (see its manual page for more information). Note that this script is available on most UNIX Operating Systems, but not on all. On Digital UNIX/Compaq Tru64 UNIX it is called ypsetup. Because we are generating maps for an NIS master, we are going to pass the -m option to ypinit. To generate the NIS maps, assuming you already performed the steps above, run:

ellington# ypinit -m test-domain
Server Type: MASTER Domain: test-domain
Creating an YP server will require that you answer a few questions.
Questions will all be asked at the beginning of the procedure.
Do you want this procedure to quit on non-fatal errors? [y/n: n] n
Ok, please remember to go back and redo manually whatever fails.
If you don't, something might not work.
At this point, we have to construct a list of this domains YP servers.
rod.darktech.org is already known as master server.
Please continue to add any slave servers, one per line. When you are
done with the list, type a <control D>.
master server   :  ellington
next host to add:  coltrane
next host to add:  ^D
The current list of NIS servers looks like this:
ellington
coltrane
Is this correct?  [y/n: y] y

[..output from map generation..]

NIS Map update completed.
ellington has been setup as an YP master server without any errors.

ypinit should have created /var/yp/Makefile from /var/yp/Makefile.dist. When created, this file assumes that you are operating in a single server NIS environment with only FreeBSD machines. Since test-domain has a slave server as well, you must edit /var/yp/Makefile:

ellington# vi /var/yp/Makefile

You should comment out the line that says

NOPUSH = "True"

(if it is not commented out already).

29.4.4.2.3 Setting up a NIS Slave Server

Setting up an NIS slave server is even more simple than setting up the master. Log on to the slave server and edit the file /etc/rc.conf as you did before. The only difference is that we now must use the -s option when running ypinit. The -s option requires the name of the NIS master be passed to it as well, so our command line looks like:

coltrane# ypinit -s ellington test-domain

Server Type: SLAVE Domain: test-domain Master: ellington

Creating an YP server will require that you answer a few questions.
Questions will all be asked at the beginning of the procedure.

Do you want this procedure to quit on non-fatal errors? [y/n: n]  n

Ok, please remember to go back and redo manually whatever fails.
If you don't, something might not work.
There will be no further questions. The remainder of the procedure
should take a few minutes, to copy the databases from ellington.
Transferring netgroup...
ypxfr: Exiting: Map successfully transferred
Transferring netgroup.byuser...
ypxfr: Exiting: Map successfully transferred
Transferring netgroup.byhost...
ypxfr: Exiting: Map successfully transferred
Transferring master.passwd.byuid...
ypxfr: Exiting: Map successfully transferred
Transferring passwd.byuid...
ypxfr: Exiting: Map successfully transferred
Transferring passwd.byname...
ypxfr: Exiting: Map successfully transferred
Transferring group.bygid...
ypxfr: Exiting: Map successfully transferred
Transferring group.byname...
ypxfr: Exiting: Map successfully transferred
Transferring services.byname...
ypxfr: Exiting: Map successfully transferred
Transferring rpc.bynumber...
ypxfr: Exiting: Map successfully transferred
Transferring rpc.byname...
ypxfr: Exiting: Map successfully transferred
Transferring protocols.byname...
ypxfr: Exiting: Map successfully transferred
Transferring master.passwd.byname...
ypxfr: Exiting: Map successfully transferred
Transferring networks.byname...
ypxfr: Exiting: Map successfully transferred
Transferring networks.byaddr...
ypxfr: Exiting: Map successfully transferred
Transferring netid.byname...
ypxfr: Exiting: Map successfully transferred
Transferring hosts.byaddr...
ypxfr: Exiting: Map successfully transferred
Transferring protocols.bynumber...
ypxfr: Exiting: Map successfully transferred
Transferring ypservers...
ypxfr: Exiting: Map successfully transferred
Transferring hosts.byname...
ypxfr: Exiting: Map successfully transferred

coltrane has been setup as an YP slave server without any errors.
Don't forget to update map ypservers on ellington.

You should now have a directory called /var/yp/test-domain. Copies of the NIS master server’s maps should be in this directory. You will need to make sure that these stay updated. The following /etc/crontab entries on your slave servers should do the job:

20      *       *       *       *       root   /usr/libexec/ypxfr passwd.byname
21      *       *       *       *       root   /usr/libexec/ypxfr passwd.byuid

These two lines force the slave to sync its maps with the maps on the master server. Although these entries are not mandatory, since the master server attempts to ensure any changes to its NIS maps are communicated to its slaves and because password information is vital to systems depending on the server, it is a good idea to force the updates. This is more important on busy networks where map updates might not always complete.

Now, run the command /etc/netstart on the slave server as well, which again starts the NIS server.

29.4.4.3 NIS Clients

An NIS client establishes what is called a binding to a particular NIS server using the ypbind daemon. ypbind checks the system’s default domain (as set by the domainname command), and begins broadcasting RPC requests on the local network. These requests specify the name of the domain for which ypbind is attempting to establish a binding. If a server that has been configured to serve the requested domain receives one of the broadcasts, it will respond to ypbind, which will record the server’s address. If there are several servers available (a master and several slaves, for example), ypbind will use the address of the first one to respond. From that point on, the client system will direct all of its NIS requests to that server. ypbind will occasionally “ping” the server to make sure it is still up and running. If it fails to receive a reply to one of its pings within a reasonable amount of time, ypbind will mark the domain as unbound and begin broadcasting again in the hopes of locating another server.

29.4.4.3.1 Setting Up a NIS Client

Setting up a FreeBSD machine to be a NIS client is fairly straightforward.

Edit the file /etc/rc.conf and add the following lines in order to set the NIS domainname and start ypbind upon network startup:
```
nisdomainname="test-domain"
nis_client_enable="YES"
```
To import all possible password entries from the NIS server, remove all user accounts from your /etc/master.passwd file and use vipw to add the following line to the end of the file:
```
+:::::::::
```
Note: This line will afford anyone with a valid account in the NIS server’s password maps an account. There are many ways to configure your NIS client by changing this line. See the netgroups section below for more information. For more detailed reading see O’Reilly’s book on Managing NFS and NIS.

Note: You should keep at least one local account (i.e. not imported via NIS) in your /etc/master.passwd and this account should also be a member of the group wheel. If there is something wrong with NIS, this account can be used to log in remotely, become root, and fix things.
To import all possible group entries from the NIS server, add this line to your /etc/group file:
```
+:*::
```

After completing these steps, you should be able to run ypcat passwd and see the NIS server’s passwd map.

29.4.5 NIS Security

In general, any remote user can issue an RPC to ypserv(8) and retrieve the contents of your NIS maps, provided the remote user knows your domainname. To prevent such unauthorized transactions, ypserv(8) supports a feature called “securenets” which can be used to restrict access to a given set of hosts. At startup, ypserv(8) will attempt to load the securenets information from a file called /var/yp/securenets.

Note: This path varies depending on the path specified with the -p option. This file contains entries that consist of a network specification and a network mask separated by white space. Lines starting with “#” are considered to be comments. A sample securenets file might look like this:

# allow connections from local host -- mandatory
127.0.0.1     255.255.255.255
# allow connections from any host
# on the 192.168.128.0 network
192.168.128.0 255.255.255.0
# allow connections from any host
# between 10.0.0.0 to 10.0.15.255
# this includes the machines in the testlab
10.0.0.0      255.255.240.0

If ypserv(8) receives a request from an address that matches one of these rules, it will process the request normally. If the address fails to match a rule, the request will be ignored and a warning message will be logged. If the /var/yp/securenets file does not exist, ypserv will allow connections from any host.

The ypserv program also has support for Wietse Venema’s TCP Wrapper package. This allows the administrator to use the TCP Wrapper configuration files for access control instead of /var/yp/securenets.

Note: While both of these access control mechanisms provide some security, they, like the privileged port test, are vulnerable to “IP spoofing” attacks. All NIS-related traffic should be blocked at your firewall.

Servers using /var/yp/securenets may fail to serve legitimate NIS clients with archaic TCP/IP implementations. Some of these implementations set all host bits to zero when doing broadcasts and/or fail to observe the subnet mask when calculating the broadcast address. While some of these problems can be fixed by changing the client configuration, other problems may force the retirement of the client systems in question or the abandonment of /var/yp/securenets.

Using /var/yp/securenets on a server with such an archaic implementation of TCP/IP is a really bad idea and will lead to loss of NIS functionality for large parts of your network.

The use of the TCP Wrapper package increases the latency of your NIS server. The additional delay may be long enough to cause timeouts in client programs, especially in busy networks or with slow NIS servers. If one or more of your client systems suffers from these symptoms, you should convert the client systems in question into NIS slave servers and force them to bind to themselves.

29.4.6 Barring Some Users from Logging On

In our lab, there is a machine basie that is supposed to be a faculty only workstation. We do not want to take this machine out of the NIS domain, yet the passwd file on the master NIS server contains accounts for both faculty and students. What can we do?

There is a way to bar specific users from logging on to a machine, even if they are present in the NIS database. To do this, all you must do is add -username to the end of the /etc/master.passwd file on the client machine, where username is the username of the user you wish to bar from logging in. This should preferably be done using vipw, since vipw will sanity check your changes to /etc/master.passwd, as well as automatically rebuild the password database when you finish editing. For example, if we wanted to bar user bill from logging on to basie we would:

basie# vipw
[add -bill to the end, exit]
vipw: rebuilding the database...
vipw: done

basie# cat /etc/master.passwd

root:[password]:0:0::0:0:The super-user:/root:/bin/csh
toor:[password]:0:0::0:0:The other super-user:/root:/bin/sh
daemon:*:1:1::0:0:Owner of many system processes:/root:/sbin/nologin
operator:*:2:5::0:0:System &:/:/sbin/nologin
bin:*:3:7::0:0:Binaries Commands and Source,,,:/:/sbin/nologin
tty:*:4:65533::0:0:Tty Sandbox:/:/sbin/nologin
kmem:*:5:65533::0:0:KMem Sandbox:/:/sbin/nologin
games:*:7:13::0:0:Games pseudo-user:/usr/games:/sbin/nologin
news:*:8:8::0:0:News Subsystem:/:/sbin/nologin
man:*:9:9::0:0:Mister Man Pages:/usr/share/man:/sbin/nologin
bind:*:53:53::0:0:Bind Sandbox:/:/sbin/nologin
uucp:*:66:66::0:0:UUCP pseudo-user:/var/spool/uucppublic:/usr/libexec/uucp/uucico
xten:*:67:67::0:0:X-10 daemon:/usr/local/xten:/sbin/nologin
pop:*:68:6::0:0:Post Office Owner:/nonexistent:/sbin/nologin
nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/sbin/nologin
+:::::::::
-bill

basie#

29.4.7 Using Netgroups

Contributed by Udo Erdelhoff.

The method shown in the previous section works reasonably well if you need special rules for a very small number of users and/or machines. On larger networks, you will forget to bar some users from logging onto sensitive machines, or you may even have to modify each machine separately, thus losing the main benefit of NIS: centralized administration.

The NIS developers’ solution for this problem is called netgroups. Their purpose and semantics can be compared to the normal groups used by UNIX file systems. The main differences are the lack of a numeric ID and the ability to define a netgroup by including both user accounts and other netgroups.

Netgroups were developed to handle large, complex networks with hundreds of users and machines. On one hand, this is a Good Thing if you are forced to deal with such a situation. On the other hand, this complexity makes it almost impossible to explain netgroups with really simple examples. The example used in the remainder of this section demonstrates this problem.

Let us assume that your successful introduction of NIS in your laboratory caught your superiors’ interest. Your next job is to extend your NIS domain to cover some of the other machines on campus. The two tables contain the names of the new users and new machines as well as brief descriptions of them.

User Name(s)

Description

alpha, beta

Normal employees of the IT department

charlie, delta

The new apprentices of the IT department

echo, foxtrott, golf, …

Ordinary employees

able, baker, …

The current interns

Machine Name(s)

Description

war, death, famine, pollution

Your most important servers. Only the IT employees are allowed to log onto these machines.

pride, greed, envy, wrath, lust, sloth

Less important servers. All members of the IT department are allowed to login onto these machines.

one, two, three, four, …

Ordinary workstations. Only the real employees are allowed to use these machines.

trashcan

A very old machine without any critical data. Even the intern is allowed to use this box.

If you tried to implement these restrictions by separately blocking each user, you would have to add one -user line to each system’s passwd for each user who is not allowed to login onto that system. If you forget just one entry, you could be in trouble. It may be feasible to do this correctly during the initial setup, however you will eventually forget to add the lines for new users during day-to-day operations. After all, Murphy was an optimist.

Handling this situation with netgroups offers several advantages. Each user need not be handled separately; you assign a user to one or more netgroups and allow or forbid logins for all members of the netgroup. If you add a new machine, you will only have to define login restrictions for netgroups. If a new user is added, you will only have to add the user to one or more netgroups. Those changes are independent of each other: no more “for each combination of user and machine do…” If your NIS setup is planned carefully, you will only have to modify exactly one central configuration file to grant or deny access to machines.

The first step is the initialization of the NIS map netgroup. FreeBSD’s ypinit(8) does not create this map by default, but its NIS implementation will support it once it has been created. To create an empty map, simply type

ellington# vi /var/yp/netgroup

and start adding content. For our example, we need at least four netgroups: IT employees, IT apprentices, normal employees and interns.

IT_EMP  (,alpha,test-domain)    (,beta,test-domain)
IT_APP  (,charlie,test-domain)  (,delta,test-domain)
USERS   (,echo,test-domain)     (,foxtrott,test-domain) \
        (,golf,test-domain)
INTERNS (,able,test-domain)     (,baker,test-domain)

IT_EMP, IT_APP etc. are the names of the netgroups. Each bracketed group adds one or more user accounts to it. The three fields inside a group are:

The name of the host(s) where the following items are valid. If you do not specify a hostname, the entry is valid on all hosts. If you do specify a hostname, you will enter a realm of darkness, horror and utter confusion.
The name of the account that belongs to this netgroup.
The NIS domain for the account. You can import accounts from other NIS domains into your netgroup if you are one of the unlucky fellows with more than one NIS domain.

Each of these fields can contain wildcards. See netgroup(5) for details.

Note: Netgroup names longer than 8 characters should not be used, especially if you have machines running other operating systems within your NIS domain. The names are case sensitive; using capital letters for your netgroup names is an easy way to distinguish between user, machine and netgroup names.

Some NIS clients (other than FreeBSD) cannot handle netgroups with a large number of entries. For example, some older versions of SunOS start to cause trouble if a netgroup contains more than 15 entries. You can circumvent this limit by creating several sub-netgroups with 15 users or less and a real netgroup that consists of the sub-netgroups:
BIGGRP1  (,joe1,domain)  (,joe2,domain)  (,joe3,domain) [...]
BIGGRP2  (,joe16,domain)  (,joe17,domain) [...]
BIGGRP3  (,joe31,domain)  (,joe32,domain)
BIGGROUP  BIGGRP1 BIGGRP2 BIGGRP3
You can repeat this process if you need more than 225 users within a single netgroup.

Activating and distributing your new NIS map is easy:

ellington# cd /var/yp
ellington# make

This will generate the three NIS maps netgroup, netgroup.byhost and netgroup.byuser. Use ypcat(1) to check if your new NIS maps are available:

ellington% ypcat -k netgroup
ellington% ypcat -k netgroup.byhost
ellington% ypcat -k netgroup.byuser

The output of the first command should resemble the contents of /var/yp/netgroup. The second command will not produce output if you have not specified host-specific netgroups. The third command can be used to get the list of netgroups for a user.

The client setup is quite simple. To configure the server war, you only have to start vipw(8) and replace the line

+:::::::::

with

+@IT_EMP:::::::::

Now, only the data for the users defined in the netgroup IT_EMP is imported into war’s password database and only these users are allowed to login.

Unfortunately, this limitation also applies to the ~ function of the shell and all routines converting between user names and numerical user IDs. In other words, cd ~user will not work, ls -l will show the numerical ID instead of the username and find . -user joe -print will fail with “No such user”. To fix this, you will have to import all user entries without allowing them to login onto your servers.

This can be achieved by adding another line to /etc/master.passwd. This line should contain:

+:::::::::/sbin/nologin, meaning “Import all entries but replace the shell with /sbin/nologin in the imported entries”. You can replace any field in the passwd entry by placing a default value in your /etc/master.passwd.

Warning: Make sure that the line +:::::::::/sbin/nologin is placed after +@IT_EMP:::::::::. Otherwise, all user accounts imported from NIS will have /sbin/nologin as their login shell.

After this change, you will only have to change one NIS map if a new employee joins the IT department. You could use a similar approach for the less important servers by replacing the old +::::::::: in their local version of /etc/master.passwd with something like this:

+@IT_EMP:::::::::
+@IT_APP:::::::::
+:::::::::/sbin/nologin

The corresponding lines for the normal workstations could be:

+@IT_EMP:::::::::
+@USERS:::::::::
+:::::::::/sbin/nologin

And everything would be fine until there is a policy change a few weeks later: The IT department starts hiring interns. The IT interns are allowed to use the normal workstations and the less important servers; and the IT apprentices are allowed to login onto the main servers. You add a new netgroup IT_INTERN, add the new IT interns to this netgroup and start to change the configuration on each and every machine… As the old saying goes: “Errors in centralized planning lead to global mess”.

NIS’ ability to create netgroups from other netgroups can be used to prevent situations like these. One possibility is the creation of role-based netgroups. For example, you could create a netgroup called BIGSRV to define the login restrictions for the important servers, another netgroup called SMALLSRV for the less important servers and a third netgroup called USERBOX for the normal workstations. Each of these netgroups contains the netgroups that are allowed to login onto these machines. The new entries for your NIS map netgroup should look like this:

BIGSRV    IT_EMP  IT_APP
SMALLSRV  IT_EMP  IT_APP  ITINTERN
USERBOX   IT_EMP  ITINTERN USERS

This method of defining login restrictions works reasonably well if you can define groups of machines with identical restrictions. Unfortunately, this is the exception and not the rule. Most of the time, you will need the ability to define login restrictions on a per-machine basis.

Machine-specific netgroup definitions are the other possibility to deal with the policy change outlined above. In this scenario, the /etc/master.passwd of each box contains two lines starting with “+”. The first of them adds a netgroup with the accounts allowed to login onto this machine, the second one adds all other accounts with /sbin/nologin as shell. It is a good idea to use the “ALL-CAPS” version of the machine name as the name of the netgroup. In other words, the lines should look like this:

+@BOXNAME:::::::::
+:::::::::/sbin/nologin

Once you have completed this task for all your machines, you will not have to modify the local versions of /etc/master.passwd ever again. All further changes can be handled by modifying the NIS map. Here is an example of a possible netgroup map for this scenario with some additional goodies:

# Define groups of users first
IT_EMP    (,alpha,test-domain)    (,beta,test-domain)
IT_APP    (,charlie,test-domain)  (,delta,test-domain)
DEPT1     (,echo,test-domain)     (,foxtrott,test-domain)
DEPT2     (,golf,test-domain)     (,hotel,test-domain)
DEPT3     (,india,test-domain)    (,juliet,test-domain)
ITINTERN  (,kilo,test-domain)     (,lima,test-domain)
D_INTERNS (,able,test-domain)     (,baker,test-domain)
#
# Now, define some groups based on roles
USERS     DEPT1   DEPT2     DEPT3
BIGSRV    IT_EMP  IT_APP
SMALLSRV  IT_EMP  IT_APP    ITINTERN
USERBOX   IT_EMP  ITINTERN  USERS
#
# And a groups for a special tasks
# Allow echo and golf to access our anti-virus-machine
SECURITY  IT_EMP  (,echo,test-domain)  (,golf,test-domain)
#
# machine-based netgroups
# Our main servers
WAR       BIGSRV
FAMINE    BIGSRV
# User india needs access to this server
POLLUTION  BIGSRV  (,india,test-domain)
#
# This one is really important and needs more access restrictions
DEATH     IT_EMP
#
# The anti-virus-machine mentioned above
ONE       SECURITY
#
# Restrict a machine to a single user
TWO       (,hotel,test-domain)
# [...more groups to follow]

If you are using some kind of database to manage your user accounts, you should be able to create the first part of the map with your database’s report tools. This way, new users will automatically have access to the boxes.

One last word of caution: It may not always be advisable to use machine-based netgroups. If you are deploying a couple of dozen or even hundreds of identical machines for student labs, you should use role-based netgroups instead of machine-based netgroups to keep the size of the NIS map within reasonable limits.

29.4.8 Important Things to Remember

There are still a couple of things that you will need to do differently now that you are in an NIS environment.

Every time you wish to add a user to the lab, you must add it to the master NIS server only, and you must remember to rebuild the NIS maps. If you forget to do this, the new user will not be able to login anywhere except on the NIS master. For example, if we needed to add a new user jsmith to the lab, we would:
```
# pw useradd jsmith
# cd /var/yp
# make test-domain
```
You could also run adduser jsmith instead of pw useradd jsmith.
Keep the administration accounts out of the NIS maps. You do not want to be propagating administrative accounts and passwords to machines that will have users that should not have access to those accounts.
Keep the NIS master and slave secure, and minimize their downtime. If somebody either hacks or simply turns off these machines, they have effectively rendered many people without the ability to login to the lab.

This is the chief weakness of any centralized administration system. If you do not protect your NIS servers, you will have a lot of angry users!

29.4.9 NIS v1 Compatibility

FreeBSD’s ypserv has some support for serving NIS v1 clients. FreeBSD’s NIS implementation only uses the NIS v2 protocol, however other implementations include support for the v1 protocol for backwards compatibility with older systems. The ypbind daemons supplied with these systems will try to establish a binding to an NIS v1 server even though they may never actually need it (and they may persist in broadcasting in search of one even after they receive a response from a v2 server). Note that while support for normal client calls is provided, this version of ypserv does not handle v1 map transfer requests; consequently, it cannot be used as a master or slave in conjunction with older NIS servers that only support the v1 protocol. Fortunately, there probably are not any such servers still in use today.

29.4.10 NIS Servers That Are Also NIS Clients

Care must be taken when running ypserv in a multi-server domain where the server machines are also NIS clients. It is generally a good idea to force the servers to bind to themselves rather than allowing them to broadcast bind requests and possibly become bound to each other. Strange failure modes can result if one server goes down and others are dependent upon it. Eventually all the clients will time out and attempt to bind to other servers, but the delay involved can be considerable and the failure mode is still present since the servers might bind to each other all over again.

You can force a host to bind to a particular server by running ypbind with the -S flag. If you do not want to do this manually each time you reboot your NIS server, you can add the following lines to your /etc/rc.conf:

nis_client_enable="YES"    # run client stuff as well
nis_client_flags="-S NIS domain,server"

See ypbind(8) for further information.

29.4.11 Password Formats

One of the most common issues that people run into when trying to implement NIS is password format compatibility. If your NIS server is using DES encrypted passwords, it will only support clients that are also using DES. For example, if you have Solaris NIS clients in your network, then you will almost certainly need to use DES encrypted passwords.

To check which format your servers and clients are using, look at /etc/login.conf. If the host is configured to use DES encrypted passwords, then the default class will contain an entry like this:

default:\
    :passwd_format=des:\
    :copyright=/etc/COPYRIGHT:\
    [Further entries elided]

Other possible values for the passwd_format capability include blf and md5 (for Blowfish and MD5 encrypted passwords, respectively).

If you have made changes to /etc/login.conf, you will also need to rebuild the login capability database, which is achieved by running the following command as root:

# cap_mkdb /etc/login.conf

Note: The format of passwords already in /etc/master.passwd will not be updated until a user changes his password for the first time after the login capability database is rebuilt.

Next, in order to ensure that passwords are encrypted with the format that you have chosen, you should also check that the crypt_default in /etc/auth.conf gives precedence to your chosen password format. To do this, place the format that you have chosen first in the list. For example, when using DES encrypted passwords, the entry would be:

crypt_default  =   des blf md5

Having followed the above steps on each of the FreeBSD based NIS servers and clients, you can be sure that they all agree on which password format is used within your network. If you have trouble authenticating on an NIS client, this is a pretty good place to start looking for possible problems. Remember: if you want to deploy an NIS server for a heterogenous network, you will probably have to use DES on all systems because it is the lowest common standard.

29.5 Automatic Network Configuration (DHCP)

Written by Greg Sutter.

29.5.1 What Is DHCP?

DHCP, the Dynamic Host Configuration Protocol, describes the means by which a system can connect to a network and obtain the necessary information for communication upon that network. FreeBSD versions prior to 6.0 use the ISC (Internet Software Consortium) DHCP client (dhclient(8)) implementation. Later versions use the OpenBSD dhclient taken from OpenBSD 3.7. All information here regarding dhclient is for use with either of the ISC or OpenBSD DHCP clients. The DHCP server is the one included in the ISC distribution.

29.5.2 What This Section Covers

This section describes both the client-side components of the ISC and OpenBSD DHCP client and server-side components of the ISC DHCP system. The client-side program, dhclient, comes integrated within FreeBSD, and the server-side portion is available from the net/isc-dhcp3-server port. The dhclient(8), dhcp-options(5), and dhclient.conf(5) manual pages, in addition to the references below, are useful resources.

29.5.3 How It Works

When dhclient, the DHCP client, is executed on the client machine, it begins broadcasting requests for configuration information. By default, these requests are on UDP port 68. The server replies on UDP 67, giving the client an IP address and other relevant network information such as netmask, router, and DNS servers. All of this information comes in the form of a DHCP “lease” and is only valid for a certain time (configured by the DHCP server maintainer). In this manner, stale IP addresses for clients no longer connected to the network can be automatically reclaimed.

DHCP clients can obtain a great deal of information from the server. An exhaustive list may be found in dhcp-options(5).

29.5.4 FreeBSD Integration

FreeBSD fully integrates the ISC or OpenBSD DHCP client, dhclient (according to the FreeBSD version you run). DHCP client support is provided within both the installer and the base system, obviating the need for detailed knowledge of network configurations on any network that runs a DHCP server. dhclient has been included in all FreeBSD distributions since 3.2.

DHCP is supported by sysinstall. When configuring a network interface within sysinstall, the second question asked is: “Do you want to try DHCP configuration of the interface?”. Answering affirmatively will execute dhclient, and if successful, will fill in the network configuration information automatically.

There are two things you must do to have your system use DHCP upon startup:

Make sure that the bpf device is compiled into your kernel. To do this, add device bpf to your kernel configuration file, and rebuild the kernel. For more information about building kernels, see Chapter 8.

The bpf device is already part of the GENERIC kernel that is supplied with FreeBSD, so if you do not have a custom kernel, you should not need to create one in order to get DHCP working.

Note: For those who are particularly security conscious, you should be warned that bpf is also the device that allows packet sniffers to work correctly (although they still have to be run as root). bpf is required to use DHCP, but if you are very sensitive about security, you probably should not add bpf to your kernel in the expectation that at some point in the future you will be using DHCP.
Edit your /etc/rc.conf to include the following:
```
ifconfig_fxp0="DHCP"
```
Note: Be sure to replace fxp0 with the designation for the interface that you wish to dynamically configure, as described in Section 11.8.

If you are using a different location for dhclient, or if you wish to pass additional flags to dhclient, also include the following (editing as necessary):
```
dhclient_program="/sbin/dhclient"
dhclient_flags=""
```

The DHCP server, dhcpd, is included as part of the net/isc-dhcp3-server port in the ports collection. This port contains the ISC DHCP server and documentation.

29.5.5 Files

/etc/dhclient.conf

dhclient requires a configuration file, /etc/dhclient.conf. Typically the file contains only comments, the defaults being reasonably sane. This configuration file is described by the dhclient.conf(5) manual page.
/sbin/dhclient

dhclient is statically linked and resides in /sbin. The dhclient(8) manual page gives more information about dhclient.
/sbin/dhclient-script

dhclient-script is the FreeBSD-specific DHCP client configuration script. It is described in dhclient-script(8), but should not need any user modification to function properly.
/var/db/dhclient.leases

The DHCP client keeps a database of valid leases in this file, which is written as a log. dhclient.leases(5) gives a slightly longer description.

29.5.6 Further Reading

The DHCP protocol is fully described in RFC 2131. An informational resource has also been set up at http://www.dhcp.org/.

29.5.7 Installing and Configuring a DHCP Server

29.5.7.1 What This Section Covers

This section provides information on how to configure a FreeBSD system to act as a DHCP server using the ISC (Internet Software Consortium) implementation of the DHCP server.

The server is not provided as part of FreeBSD, and so you will need to install the net/isc-dhcp3-server port to provide this service. See Chapter 4 for more information on using the Ports Collection.

29.5.7.2 DHCP Server Installation

In order to configure your FreeBSD system as a DHCP server, you will need to ensure that the bpf(4) device is compiled into your kernel. To do this, add device bpf to your kernel configuration file, and rebuild the kernel. For more information about building kernels, see Chapter 8.

The bpf device is already part of the GENERIC kernel that is supplied with FreeBSD, so you do not need to create a custom kernel in order to get DHCP working.

Note: Those who are particularly security conscious should note that bpf is also the device that allows packet sniffers to work correctly (although such programs still need privileged access). bpf is required to use DHCP, but if you are very sensitive about security, you probably should not include bpf in your kernel purely because you expect to use DHCP at some point in the future.

The next thing that you will need to do is edit the sample dhcpd.conf which was installed by the net/isc-dhcp3-server port. By default, this will be /usr/local/etc/dhcpd.conf.sample, and you should copy this to /usr/local/etc/dhcpd.conf before proceeding to make changes.

29.5.7.3 Configuring the DHCP Server

dhcpd.conf is comprised of declarations regarding subnets and hosts, and is perhaps most easily explained using an example :

option domain-name "example.com";
option domain-name-servers 192.168.4.100;
option subnet-mask 255.255.255.0;

default-lease-time 3600;
max-lease-time 86400;
ddns-update-style none;

subnet 192.168.4.0 netmask 255.255.255.0 {
  range 192.168.4.129 192.168.4.254;
  option routers 192.168.4.1;
}

host mailhost {
  hardware ethernet 02:03:04:05:06:07;
  fixed-address mailhost.example.com;
}

: This option specifies the domain that will be provided to clients as the default search domain. See resolv.conf(5) for more information on what this means.
: This option specifies a comma separated list of DNS servers that the client should use.
: The netmask that will be provided to clients.
: A client may request a specific length of time that a lease will be valid. Otherwise the server will assign a lease with this expiry value (in seconds).
: This is the maximum length of time that the server will lease for. Should a client request a longer lease, a lease will be issued, although it will only be valid for max-lease-time seconds.
: This option specifies whether the DHCP server should attempt to update DNS when a lease is accepted or released. In the ISC implementation, this option is required.
: This denotes which IP addresses should be used in the pool reserved for allocating to clients. IP addresses between, and including, the ones stated are handed out to clients.
: Declares the default gateway that will be provided to clients.
: The hardware MAC address of a host (so that the DHCP server can recognize a host when it makes a request).
: Specifies that the host should always be given the same IP address. Note that using a hostname is correct here, since the DHCP server will resolve the hostname itself before returning the lease information.

Once you have finished writing your dhcpd.conf, you should enable the DHCP server in /etc/rc.conf, i.e. by adding:

dhcpd_enable="YES"
dhcpd_ifaces="dc0"

Replace the dc0 interface name with the interface (or interfaces, separated by whitespace) that your DHCP server should listen on for DHCP client requests.

Then, you can proceed to start the server by issuing the following command:

# /usr/local/etc/rc.d/isc-dhcpd.sh start

Should you need to make changes to the configuration of your server in the future, it is important to note that sending a SIGHUP signal to dhcpd does not result in the configuration being reloaded, as it does with most daemons. You will need to send a SIGTERM signal to stop the process, and then restart it using the command above.

29.5.7.4 Files

/usr/local/sbin/dhcpd

dhcpd is statically linked and resides in /usr/local/sbin. The dhcpd(8) manual page installed with the port gives more information about dhcpd.
/usr/local/etc/dhcpd.conf

dhcpd requires a configuration file, /usr/local/etc/dhcpd.conf before it will start providing service to clients. This file needs to contain all the information that should be provided to clients that are being serviced, along with information regarding the operation of the server. This configuration file is described by the dhcpd.conf(5) manual page installed by the port.
/var/db/dhcpd.leases

The DHCP server keeps a database of leases it has issued in this file, which is written as a log. The manual page dhcpd.leases(5), installed by the port gives a slightly longer description.
/usr/local/sbin/dhcrelay

dhcrelay is used in advanced environments where one DHCP server forwards a request from a client to another DHCP server on a separate network. If you require this functionality, then install the net/isc-dhcp3-relay port. The dhcrelay(8) manual page provided with the port contains more detail.

29.6 Domain Name System (DNS)

Contributed by Chern Lee, Tom Rhodes, and Daniel Gerzo.

29.6.1 Overview

FreeBSD utilizes, by default, a version of BIND (Berkeley Internet Name Domain), which is the most common implementation of the DNS protocol. DNS is the protocol through which names are mapped to IP addresses, and vice versa. For example, a query for www.FreeBSD.org will receive a reply with the IP address of The FreeBSD Project’s web server, whereas, a query for ftp.FreeBSD.org will return the IP address of the corresponding FTP machine. Likewise, the opposite can happen. A query for an IP address can resolve its hostname. It is not necessary to run a name server to perform DNS lookups on a system.

FreeBSD currently comes with BIND9 DNS server software by default. Our installation provides enhanced security features, a new file system layout and automated chroot(8) configuration.

DNS is coordinated across the Internet through a somewhat complex system of authoritative root, Top Level Domain (TLD), and other smaller-scale name servers which host and cache individual domain information.

Currently, BIND is maintained by the Internet Software Consortium http://www.isc.org/.

29.6.2 Terminology

To understand this document, some terms related to DNS must be understood.

Term

Definition

Forward DNS

Mapping of hostnames to IP addresses.

Origin

Refers to the domain covered in a particular zone file.

named, BIND, name server

Common names for the BIND name server package within FreeBSD.

Resolver

A system process through which a machine queries a name server for zone information.

Reverse DNS

The opposite of forward DNS; mapping of IP addresses to hostnames.

Root zone

The beginning of the Internet zone hierarchy. All zones fall under the root zone, similar to how all files in a file system fall under the root directory.

Zone

An individual domain, subdomain, or portion of the DNS administered by the same authority.

Examples of zones:

. is the root zone.
org. is a Top Level Domain (TLD) under the root zone.
example.org. is a zone under the org. TLD.
1.168.192.in-addr.arpa is a zone referencing all IP addresses which fall under the 192.168.1.* IP space.

As one can see, the more specific part of a hostname appears to its left. For example, example.org. is more specific than org., as org. is more specific than the root zone. The layout of each part of a hostname is much like a file system: the /dev directory falls within the root, and so on.

29.6.3 Reasons to Run a Name Server

Name servers usually come in two forms: an authoritative name server, and a caching name server.

An authoritative name server is needed when:

One wants to serve DNS information to the world, replying authoritatively to queries.
A domain, such as example.org, is registered and IP addresses need to be assigned to hostnames under it.
An IP address block requires reverse DNS entries (IP to hostname).
A backup or second name server, called a slave, will reply to queries.

A caching name server is needed when:

A local DNS server may cache and respond more quickly than querying an outside name server.

When one queries for www.FreeBSD.org, the resolver usually queries the uplink ISP’s name server, and retrieves the reply. With a local, caching DNS server, the query only has to be made once to the outside world by the caching DNS server. Every additional query will not have to look to the outside of the local network, since the information is cached locally.

29.6.4 How It Works

In FreeBSD, the BIND daemon is called named for obvious reasons.

File

Description

named(8)

The BIND daemon.

rndc(8)

Name server control utility.

/etc/namedb

Directory where BIND zone information resides.

/etc/namedb/named.conf

Configuration file of the daemon.

Depending on how a given zone is configured on the server, the files related to that zone can be found in the master, slave, or dynamic subdirectories of the /etc/namedb directory. These files contain the DNS information that will be given out by the name server in response to queries.

29.6.5 Starting BIND

Since BIND is installed by default, configuring it all is relatively simple.

The default named configuration is that of a basic resolving name server, ran in a chroot(8) environment. To start the server one time with this configuration, use the following command:

# /etc/rc.d/named forcestart

To ensure the named daemon is started at boot each time, put the following line into the /etc/rc.conf:

named_enable="YES"

There are obviously many configuration options for /etc/namedb/named.conf that are beyond the scope of this document. However, if you are interested in the startup options for named on FreeBSD, take a look at the named_* flags in /etc/defaults/rc.conf and consult the rc.conf(5) manual page. The Section 11.7 section is also a good read.

29.6.6 Configuration Files

Configuration files for named currently reside in /etc/namedb directory and will need modification before use, unless all that is needed is a simple resolver. This is where most of the configuration will be performed.

29.6.6.1 Using `make-localhost`

To configure a master zone for the localhost visit the /etc/namedb directory and run the following command:

# sh make-localhost

If all went well, a new file should exist in the master subdirectory. The filenames should be localhost.rev for the local domain name and localhost-v6.rev for IPv6 configurations. As the default configuration file, required information will be present in the named.conf file.

29.6.6.2 `/etc/namedb/named.conf`

// $FreeBSD$
//
// Refer to the named.conf(5) and named(8) man pages, and the documentation
// in /usr/share/doc/bind9 for more details.
//
// If you are going to set up an authoritative server, make sure you
// understand the hairy details of how DNS works.  Even with
// simple mistakes, you can break connectivity for affected parties,
// or cause huge amounts of useless Internet traffic.

options {
    directory   "/etc/namedb";
    pid-file    "/var/run/named/pid";
    dump-file   "/var/dump/named_dump.db";
    statistics-file "/var/stats/named.stats";

// If named is being used only as a local resolver, this is a safe default.
// For named to be accessible to the network, comment this option, specify
// the proper IP address, or delete this option.
    listen-on   { 127.0.0.1; };

// If you have IPv6 enabled on this system, uncomment this option for
// use as a local resolver.  To give access to the network, specify
// an IPv6 address, or the keyword "any".
//  listen-on-v6    { ::1; };

// In addition to the "forwarders" clause, you can force your name
// server to never initiate queries of its own, but always ask its
// forwarders only, by enabling the following line:
//
//  forward only;

// If you've got a DNS server around at your upstream provider, enter
// its IP address here, and enable the line below.  This will make you
// benefit from its cache, thus reduce overall DNS traffic in the Internet.
/*
    forwarders {
        127.0.0.1;
    };
*/

Just as the comment says, to benefit from an uplink’s cache, forwarders can be enabled here. Under normal circumstances, a name server will recursively query the Internet looking at certain name servers until it finds the answer it is looking for. Having this enabled will have it query the uplink’s name server (or name server provided) first, taking advantage of its cache. If the uplink name server in question is a heavily trafficked, fast name server, enabling this may be worthwhile.

Warning: 127.0.0.1 will not work here. Change this IP address to a name server at your uplink.

   /*
     * If there is a firewall between you and nameservers you want
     * to talk to, you might need to uncomment the query-source
     * directive below.  Previous versions of BIND always asked
     * questions using port 53, but BIND versions 8 and later
     * use a pseudo-random unprivileged UDP port by default.
     */
     // query-source address * port 53;
};

// If you enable a local name server, don't forget to enter 127.0.0.1
// first in your /etc/resolv.conf so this server will be queried.
// Also, make sure to enable it in /etc/rc.conf.

zone "." {
    type hint;
    file "named.root";
};

zone "0.0.127.IN-ADDR.ARPA" {
    type master;
    file "master/localhost.rev";
};

// RFC 3152
zone "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.IP6.ARPA" {
    type master;
    file "master/localhost-v6.rev";
};

// NB: Do not use the IP addresses below, they are faked, and only
// serve demonstration/documentation purposes!
//
// Example slave zone config entries.  It can be convenient to become
// a slave at least for the zone your own domain is in.  Ask
// your network administrator for the IP address of the responsible
// primary.
//
// Never forget to include the reverse lookup (IN-ADDR.ARPA) zone!
// (This is named after the first bytes of the IP address, in reverse
// order, with ".IN-ADDR.ARPA" appended.)
//
// Before starting to set up a primary zone, make sure you fully
// understand how DNS and BIND works.  There are sometimes
// non-obvious pitfalls.  Setting up a slave zone is simpler.
//
// NB: Don't blindly enable the examples below.   Use actual names
// and addresses instead.

/* An example master zone
zone "example.net" {
    type master;
    file "master/example.net";
};
*/

/* An example dynamic zone
key "exampleorgkey" {
    algorithm hmac-md5;
    secret "sf87HJqjkqh8ac87a02lla==";
};
zone "example.org" {
    type master;
    allow-update {
        key "exampleorgkey";
    };
    file "dynamic/example.org";
};
*/

/* Examples of forward and reverse slave zones
zone "example.com" {
    type slave;
    file "slave/example.com";
    masters {
        192.168.1.1;
    };
};
zone "1.168.192.in-addr.arpa" {
    type slave;
    file "slave/1.168.192.in-addr.arpa";
    masters {
        192.168.1.1;
    };
};
*/

In named.conf, these are examples of slave entries for a forward and reverse zone.

For each new zone served, a new zone entry must be added to named.conf.

For example, the simplest zone entry for example.org can look like:

zone "example.org" {
    type master;
    file "master/example.org";
};

The zone is a master, as indicated by the type statement, holding its zone information in /etc/namedb/master/example.org indicated by the file statement.

zone "example.org" {
    type slave;
    file "slave/example.org";
};

In the slave case, the zone information is transferred from the master name server for the particular zone, and saved in the file specified. If and when the master server dies or is unreachable, the slave name server will have the transferred zone information and will be able to serve it.

29.6.6.3 Zone Files

An example master zone file for example.org (existing within /etc/namedb/master/example.org) is as follows:

$TTL 3600        ; 1 hour
example.org.    IN      SOA      ns1.example.org. admin.example.org. (
                                2006051501      ; Serial
                                10800           ; Refresh
                                3600            ; Retry
                                604800          ; Expire
                                86400           ; Minimum TTL
                        )

; DNS Servers
                IN      NS      ns1.example.org.
                IN      NS      ns2.example.org.

; MX Records
                IN      MX 10   mx.example.org.
                IN      MX 20   mail.example.org.

                IN      A       192.168.1.1

; Machine Names
localhost       IN      A       127.0.0.1
ns1             IN      A       192.168.1.2
ns2             IN      A       192.168.1.3
mx              IN      A       192.168.1.4
mail            IN      A       192.168.1.5

; Aliases
www             IN      CNAME   @

Note that every hostname ending in a “.” is an exact hostname, whereas everything without a trailing “.” is referenced to the origin. For example, www is translated into www.origin. In our fictitious zone file, our origin is example.org., so www would translate to www.example.org.

The format of a zone file follows:

recordname      IN recordtype   value

The most commonly used DNS records:

SOA

start of zone authority

NS

an authoritative name server

A

a host address

CNAME

the canonical name for an alias

MX

mail exchanger

PTR

a domain name pointer (used in reverse DNS)

example.org. IN SOA ns1.example.org. admin.example.org. (
                        2006051501      ; Serial
                        10800           ; Refresh after 3 hours
                        3600            ; Retry after 1 hour
                        604800          ; Expire after 1 week
                        86400 )         ; Minimum TTL of 1 day

example.org.

the domain name, also the origin for this zone file.

ns1.example.org.

the primary/authoritative name server for this zone.

admin.example.org.

the responsible person for this zone, email address with “@” replaced. (<admin@example.org> becomes admin.example.org)

2006051501

the serial number of the file. This must be incremented each time the zone file is modified. Nowadays, many admins prefer a yyyymmddrr format for the serial number. 2006051501 would mean last modified 05/15/2006, the latter 01 being the first time the zone file has been modified this day. The serial number is important as it alerts slave name servers for a zone when it is updated.

       IN NS           ns1.example.org.

This is an NS entry. Every name server that is going to reply authoritatively for the zone must have one of these entries.

localhost       IN      A       127.0.0.1
ns1             IN      A       192.168.1.2
ns2             IN      A       192.168.1.3
mx              IN      A       192.168.1.4
mail            IN      A       192.168.1.5

The A record indicates machine names. As seen above, ns1.example.org would resolve to 192.168.1.2.

                IN      A       192.168.1.1

This line assigns IP address 192.168.1.1 to the current origin, in this case example.org.

www             IN CNAME        @

The canonical name record is usually used for giving aliases to a machine. In the example, www is aliased to the “master” machine which name equals to domain name example.org (192.168.1.1). CNAMEs can be used to provide alias hostnames, or round robin one hostname among multiple machines.

               IN MX   10      mail.example.org.

The MX record indicates which mail servers are responsible for handling incoming mail for the zone. mail.example.org is the hostname of the mail server, and 10 being the priority of that mail server.

One can have several mail servers, with priorities of 10, 20 and so on. A mail server attempting to deliver to example.org would first try the highest priority MX (the record with the lowest priority number), then the second highest, etc, until the mail can be properly delivered.

For in-addr.arpa zone files (reverse DNS), the same format is used, except with PTR entries instead of A or CNAME.

$TTL 3600

1.168.192.in-addr.arpa. IN SOA ns1.example.org. admin.example.org. (
                        2006051501      ; Serial
                        10800           ; Refresh
                        3600            ; Retry
                        604800          ; Expire
                        3600 )          ; Minimum

        IN      NS      ns1.example.org.
        IN      NS      ns2.example.org.

1       IN      PTR     example.org.
2       IN      PTR     ns1.example.org.
3       IN      PTR     ns2.example.org.
4       IN      PTR     mx.example.org.
5       IN      PTR     mail.example.org.

This file gives the proper IP address to hostname mappings of our above fictitious domain.

29.6.7 Caching Name Server

A caching name server is a name server that is not authoritative for any zones. It simply asks queries of its own, and remembers them for later use. To set one up, just configure the name server as usual, omitting any inclusions of zones.

29.6.8 Security

Although BIND is the most common implementation of DNS, there is always the issue of security. Possible and exploitable security holes are sometimes found.

While FreeBSD automatically drops named into a chroot(8) environment; there are several other security mechanisms in place which could help to lure off possible DNS service attacks.

It is always good idea to read CERT’s security advisories and to subscribe to the FreeBSD security notifications mailing list to stay up to date with the current Internet and FreeBSD security issues.

Tip: If a problem arises, keeping sources up to date and having a fresh build of named would not hurt.

29.6.9 Further Reading

BIND/named manual pages: rndc(8) named(8) named.conf(5)

Official ISC BIND Page
Official ISC BIND Forum
BIND9 FAQ
O’Reilly DNS and BIND 5th Edition
RFC1034 - Domain Names - Concepts and Facilities
RFC1035 - Domain Names - Implementation and Specification

29.7 Apache HTTP Server

Contributed by Murray Stokely.

29.7.1 Overview

FreeBSD is used to run some of the busiest web sites in the world. The majority of web servers on the Internet are using the Apache HTTP Server. Apache software packages should be included on your FreeBSD installation media. If you did not install Apache when you first installed FreeBSD, then you can install it from the www/apache13 or www/apache22 port.

Once Apache has been installed successfully, it must be configured.

Note: This section covers version 1.3.X of the Apache HTTP Server as that is the most widely used version for FreeBSD. Apache 2.X introduces many new technologies but they are not discussed here. For more information about Apache 2.X, please see http://httpd.apache.org/.

29.7.2 Configuration

The main Apache HTTP Server configuration file is installed as /usr/local/etc/apache/httpd.conf on FreeBSD. This file is a typical UNIX text configuration file with comment lines beginning with the # character. A comprehensive description of all possible configuration options is outside the scope of this book, so only the most frequently modified directives will be described here.

ServerRoot "/usr/local"

This specifies the default directory hierarchy for the Apache installation. Binaries are stored in the bin and sbin subdirectories of the server root, and configuration files are stored in etc/apache.

ServerAdmin you@your.address

The address to which problems with the server should be emailed. This address appears on some server-generated pages, such as error documents.

ServerName www.example.com

ServerName allows you to set a host name which is sent back to clients for your server if it is different to the one that the host is configured with (i.e., use www instead of the host’s real name).

DocumentRoot "/usr/local/www/data"

DocumentRoot: The directory out of which you will serve your documents. By default, all requests are taken from this directory, but symbolic links and aliases may be used to point to other locations.

It is always a good idea to make backup copies of your Apache configuration file before making changes. Once you are satisfied with your initial configuration you are ready to start running Apache.

29.7.3 Running Apache

Apache does not run from the inetd super server as many other network servers do. It is configured to run standalone for better performance for incoming HTTP requests from client web browsers. A shell script wrapper is included to make starting, stopping, and restarting the server as simple as possible. To start up Apache for the first time, just run:

# /usr/local/sbin/apachectl start

You can stop the server at any time by typing:

# /usr/local/sbin/apachectl stop

After making changes to the configuration file for any reason, you will need to restart the server:

# /usr/local/sbin/apachectl restart

To restart Apache without aborting current connections, run:

# /usr/local/sbin/apachectl graceful

Additional information available at apachectl(8) manual page.

To launch Apache at system startup, add the following line to /etc/rc.conf:

apache_enable="YES"

or for Apache 2.2:

apache22_enable="YES"

If you would like to supply additional command line options for the Apache httpd program started at system boot, you may specify them with an additional line in rc.conf:

apache_flags=""

Now that the web server is running, you can view your web site by pointing a web browser to http://localhost/. The default web page that is displayed is /usr/local/www/data/index.html.

29.7.4 Virtual Hosting

Apache supports two different types of Virtual Hosting. The first method is Name-based Virtual Hosting. Name-based virtual hosting uses the clients HTTP/1.1 headers to figure out the hostname. This allows many different domains to share the same IP address.

To setup Apache to use Name-based Virtual Hosting add an entry like the following to your httpd.conf:

NameVirtualHost *

If your webserver was named www.domain.tld and you wanted to setup a virtual domain for www.someotherdomain.tld then you would add the following entries to httpd.conf:

<VirtualHost *>
ServerName www.domain.tld
DocumentRoot /www/domain.tld
</VirtualHost>

<VirtualHost *>
ServerName www.someotherdomain.tld
DocumentRoot /www/someotherdomain.tld
</VirtualHost>

Replace the addresses with the addresses you want to use and the path to the documents with what you are using.

For more information about setting up virtual hosts, please consult the official Apache documentation at: http://httpd.apache.org/docs/vhosts/.

29.7.5 Apache Modules

There are many different Apache modules available to add functionality to the basic server. The FreeBSD Ports Collection provides an easy way to install Apache together with some of the more popular add-on modules.

29.7.5.1 mod_ssl

The mod_ssl module uses the OpenSSL library to provide strong cryptography via the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) protocols. This module provides everything necessary to request a signed certificate from a trusted certificate signing authority so that you can run a secure web server on FreeBSD.

If you have not yet installed Apache, then a version of Apache 1.3.X that includes mod_ssl may be installed with the www/apache13-modssl port. SSL support is also available for Apache 2.X in the www/apache22 port, where it is enabled by default.

29.7.5.2 Language Bindings

There are Apache modules for most major scripting languages. These modules typically make it possible to write Apache modules entirely in a scripting language. They are also often used as a persistent interpreter embedded into the server that avoids the overhead of starting an external interpreter and the startup-time penalty for dynamic websites, as described in the next section.

29.7.6 Dynamic Websites

In the last decade, more businesses have turned to the Internet in order to enhance their revenue and increase exposure. This has also increased the need for interactive web content. While some companies, such as Microsoft, have introduced solutions into their proprietary products, the open source community answered the call. Modern options for dynamic web content include Django, Ruby on Rails, mod_perl, and mod_php.

29.7.6.1 Django

Django is a BSD licensed framework designed to allow developers to write high performance, elegant web applications quickly. It provides an object-relational mapper so that data types are developed as Python objects, and a rich dynamic database-access API is provided for those objects without the developer ever having to write SQL. It also provides an extensible template system so that the logic of the application is separated from the HTML presentation.

Django depends on mod_python, Apache, and an SQL database engine of your choice. The FreeBSD Port will install all of these pre-requisites for you with the appropriate flags.

Example 29-3. Installing Django with Apache2, mod_python3, and PostgreSQL

# cd /usr/ports/www/py-django; make all install clean -DWITH_MOD_PYTHON3 -DWITH_POSTGRESQL

Once Django and these pre-requisites are installed, you will need to create a Django project directory and then configure Apache to use the embedded Python interpreter to call your application for specific URLs on your site.

Example 29-4. Apache Configuration for Django/mod_python

You will need to add a line to the apache httpd.conf file to configure Apache to pass requests for certain URLs to your web application:

<Location "/">
    SetHandler python-program
    PythonPath "['/dir/to/your/django/packages/'] + sys.path"
    PythonHandler django.core.handlers.modpython
    SetEnv DJANGO_SETTINGS_MODULE mysite.settings
    PythonAutoReload On
    PythonDebug On
</Location>

29.7.6.2 Ruby on Rails

Ruby on Rails is another open source web framework that provides a full development stack and is optimized to make web developers more productive and capable of writing powerful applications quickly. It can be installed easily from the ports system.

# cd /usr/ports/www/rubygem-rails; make all install clean

29.7.6.3 mod_perl

The Apache/Perl integration project brings together the full power of the Perl programming language and the Apache HTTP Server. With the mod_perl module it is possible to write Apache modules entirely in Perl. In addition, the persistent interpreter embedded in the server avoids the overhead of starting an external interpreter and the penalty of Perl start-up time.

mod_perl is available a few different ways. To use mod_perl remember that mod_perl 1.0 only works with Apache 1.3 and mod_perl 2.0 only works with Apache 2.X. mod_perl 1.0 is available in www/mod_perl and a statically compiled version is available in www/apache13-modperl. mod_perl 2.0 is available in www/mod_perl2.

29.7.6.4 mod_php

Written by Tom Rhodes.

PHP, also known as “PHP: Hypertext Preprocessor” is a general-purpose scripting language that is especially suited for Web development. Capable of being embedded into HTML its syntax draws upon C, Java, and Perl with the intention of allowing web developers to write dynamically generated webpages quickly.

To gain support for PHP5 for the Apache web server, begin by installing the lang/php5 port.

If the lang/php5 port is being installed for the first time, available OPTIONS will be displayed automatically. If a menu is not displayed, i.e. because the lang/php5 port has been installed some time in the past, it is always possible to bring the options dialog up again by running:

# make config

in the port directory.

In the options dialog, check the APACHE option to build mod_php5 as a loadable module for the Apache web server.

Note: A lot of sites are still using PHP4 for various reasons (i.e. compatibility issues or already deployed web applications). If the mod_php4 is needed instead of mod_php5, then please use the lang/php4 port. The lang/php4 port supports many of the configuration and build-time options of the lang/php5 port.

This will install and configure the modules required to support dynamic PHP applications. Check to ensure the following sections have been added to /usr/local/etc/apache/httpd.conf:

LoadModule php5_module        libexec/apache/libphp5.so

AddModule mod_php5.c
    <IfModule mod_php5.c>
        DirectoryIndex index.php index.html
    </IfModule>
    <IfModule mod_php5.c>
        AddType application/x-httpd-php .php
        AddType application/x-httpd-php-source .phps
    </IfModule>

Once completed, a simple call to the apachectl command for a graceful restart is needed to load the PHP module:

# apachectl graceful

For future upgrades of PHP, the make config command will not be required; the selected OPTIONS are saved automatically by the FreeBSD Ports framework.

The PHP support in FreeBSD is extremely modular so the base install is very limited. It is very easy to add support using the lang/php5-extensions port. This port provides a menu driven interface to PHP extension installation. Alternatively, individual extensions can be installed using the appropriate port.

For instance, to add support for the MySQL database server to PHP5, simply install the databases/php5-mysql port.

After installing an extension, the Apache server must be reloaded to pick up the new configuration changes:

# apachectl graceful

29.8 File Transfer Protocol (FTP)

Contributed by Murray Stokely.

29.8.1 Overview

The File Transfer Protocol (FTP) provides users with a simple way to transfer files to and from an FTP server. FreeBSD includes FTP server software, ftpd, in the base system. This makes setting up and administering an FTP server on FreeBSD very straightforward.

29.8.2 Configuration

The most important configuration step is deciding which accounts will be allowed access to the FTP server. A normal FreeBSD system has a number of system accounts used for various daemons, but unknown users should not be allowed to log in with these accounts. The /etc/ftpusers file is a list of users disallowed any FTP access. By default, it includes the aforementioned system accounts, but it is possible to add specific users here that should not be allowed access to FTP.

You may want to restrict the access of some users without preventing them completely from using FTP. This can be accomplished with the /etc/ftpchroot file. This file lists users and groups subject to FTP access restrictions. The ftpchroot(5) manual page has all of the details so it will not be described in detail here.

If you would like to enable anonymous FTP access to your server, then you must create a user named ftp on your FreeBSD system. Users will then be able to log on to your FTP server with a username of ftp or anonymous and with any password (by convention an email address for the user should be used as the password). The FTP server will call chroot(2) when an anonymous user logs in, to restrict access to only the home directory of the ftp user.

There are two text files that specify welcome messages to be displayed to FTP clients. The contents of the file /etc/ftpwelcome will be displayed to users before they reach the login prompt. After a successful login, the contents of the file /etc/ftpmotd will be displayed. Note that the path to this file is relative to the login environment, so the file ~ftp/etc/ftpmotd would be displayed for anonymous users.

Once the FTP server has been configured properly, it must be enabled in /etc/inetd.conf. All that is required here is to remove the comment symbol “#” from in front of the existing ftpd line :

ftp    stream  tcp nowait  root    /usr/libexec/ftpd   ftpd -l

As explained in Example 29-1, the inetd configuration must be reloaded after this configuration file is changed. Please refer to Section 29.2.2 for details on enabling inetd on your system.

Alternatively, ftpd can also be started as a stand-alone server. In this case, it is sufficient to set the appropriate variable in /etc/rc.conf:

ftpd_enable="YES"

After setting the above variable, the stand-alone server will be started at the next reboot, or it can be started manually by executing the following command as root:

# /etc/rc.d/ftpd start

You can now log on to your FTP server by typing:

% ftp localhost

29.8.3 Maintaining

The ftpd daemon uses syslog(3) to log messages. By default, the system log daemon will put messages related to FTP in the /var/log/xferlog file. The location of the FTP log can be modified by changing the following line in /etc/syslog.conf:

ftp.info      /var/log/xferlog

Be aware of the potential problems involved with running an anonymous FTP server. In particular, you should think twice about allowing anonymous users to upload files. You may find that your FTP site becomes a forum for the trade of unlicensed commercial software or worse. If you do need to allow anonymous FTP uploads, then you should set up the permissions so that these files can not be read by other anonymous users until they have been reviewed.

29.9 File and Print Services for Microsoft Windows clients (Samba)

Contributed by Murray Stokely.

29.9.1 Overview

Samba is a popular open source software package that provides file and print services for Microsoft Windows clients. Such clients can connect to and use FreeBSD filespace as if it was a local disk drive, or FreeBSD printers as if they were local printers.

Samba software packages should be included on your FreeBSD installation media. If you did not install Samba when you first installed FreeBSD, then you can install it from the net/samba3 port or package.

29.9.2 Configuration

A default Samba configuration file is installed as /usr/local/share/examples/samba/smb.conf.default. This file must be copied to /usr/local/etc/smb.conf and customized before Samba can be used.

The smb.conf file contains runtime configuration information for Samba, such as definitions of the printers and “file system shares” that you would like to share with Windows clients. The Samba package includes a web based tool called swat which provides a simple way of configuring the smb.conf file.

29.9.2.1 Using the Samba Web Administration Tool (SWAT)

The Samba Web Administration Tool (SWAT) runs as a daemon from inetd. Therefore, the following line in /etc/inetd.conf should be uncommented before swat can be used to configure Samba:

swat   stream  tcp     nowait/400      root    /usr/local/sbin/swat    swat

As explained in Example 29-1, the inetd configuration must be reloaded after this configuration file is changed.

Once swat has been enabled in inetd.conf, you can use a browser to connect to http://localhost:901. You will first have to log on with the system root account.

Once you have successfully logged on to the main Samba configuration page, you can browse the system documentation, or begin by clicking on the Globals tab. The Globals section corresponds to the variables that are set in the [global] section of /usr/local/etc/smb.conf.

29.9.2.2 Global Settings

Whether you are using swat or editing /usr/local/etc/smb.conf directly, the first directives you are likely to encounter when configuring Samba are:

workgroup

NT Domain-Name or Workgroup-Name for the computers that will be accessing this server.

netbios name

This sets the NetBIOS name by which a Samba server is known. By default it is the same as the first component of the host’s DNS name.

server string

This sets the string that will be displayed with the net view command and some other networking tools that seek to display descriptive text about the server.

29.9.2.3 Security Settings

Two of the most important settings in /usr/local/etc/smb.conf are the security model chosen, and the backend password format for client users. The following directives control these options:

security

The two most common options here are security = share and security = user. If your clients use usernames that are the same as their usernames on your FreeBSD machine then you will want to use user level security. This is the default security policy and it requires clients to first log on before they can access shared resources.

In share level security, client do not need to log onto the server with a valid username and password before attempting to connect to a shared resource. This was the default security model for older versions of Samba.

passdb backend

Samba has several different backend authentication models. You can authenticate clients with LDAP, NIS+, a SQL database, or a modified password file. The default authentication method is smbpasswd, and that is all that will be covered here.

Assuming that the default smbpasswd backend is used, the /usr/local/private/smbpasswd file must be created to allow Samba to authenticate clients. If you would like to give your UNIX user accounts access from Windows clients, use the following command:

# smbpasswd -a username

Note: Since Samba 3.0.23c, the actual directory for authentication files is /usr/local/etc/samba. The recommended backend is now tdbsam, and the following command should be used to add user accounts:
# pdbedit -a -u username

Please see the Official Samba HOWTO for additional information about configuration options. With the basics outlined here, you should have everything you need to start running Samba.

29.9.3 Starting Samba

The net/samba3 port adds a new startup script, which can be used to control Samba. To enable this script, so that it can be used for example to start, stop or restart Samba, add the following line to the /etc/rc.conf file:

samba_enable="YES"

Or, for fine grain control:

nmbd_enable="YES"

smbd_enable="YES"

Note: This will also configure Samba to automatically start at system boot time.

It is possible then to start Samba at any time by typing:

# /usr/local/etc/rc.d/samba start
Starting SAMBA: removing stale tdbs :
Starting nmbd.
Starting smbd.

Please refer to Section 11.7 for more information about using rc scripts.

Samba actually consists of three separate daemons. You should see that both the nmbd and smbd daemons are started by the samba script. If you enabled winbind name resolution services in smb.conf, then you will also see that the winbindd daemon is started.

You can stop Samba at any time by typing :

# /usr/local/etc/rc.d/samba stop

Samba is a complex software suite with functionality that allows broad integration with Microsoft Windows networks. For more information about functionality beyond the basic installation described here, please see http://www.samba.org.

29.10 Clock Synchronization with NTP

Contributed by Tom Hukins.

29.10.1 Overview

Over time, a computer’s clock is prone to drift. The Network Time Protocol (NTP) is one way to ensure your clock stays accurate.

Many Internet services rely on, or greatly benefit from, computers’ clocks being accurate. For example, a web server may receive requests to send a file if it has been modified since a certain time. In a local area network environment, it is essential that computers sharing files from the same file server have synchronized clocks so that file timestamps stay consistent. Services such as cron(8) also rely on an accurate system clock to run commands at the specified times.

FreeBSD ships with the ntpd(8) NTP server which can be used to query other NTP servers to set the clock on your machine or provide time services to others.

29.10.2 Choosing Appropriate NTP Servers

In order to synchronize your clock, you will need to find one or more NTP servers to use. Your network administrator or ISP may have set up an NTP server for this purpose–check their documentation to see if this is the case. There is an online list of publicly accessible NTP servers which you can use to find an NTP server near to you. Make sure you are aware of the policy for any servers you choose, and ask for permission if required.

Choosing several unconnected NTP servers is a good idea in case one of the servers you are using becomes unreachable or its clock is unreliable. ntpd(8) uses the responses it receives from other servers intelligently–it will favor unreliable servers less than reliable ones.

29.10.3 Configuring Your Machine

29.10.3.1 Basic Configuration

If you only wish to synchronize your clock when the machine boots up, you can use ntpdate(8). This may be appropriate for some desktop machines which are frequently rebooted and only require infrequent synchronization, but most machines should run ntpd(8).

Using ntpdate(8) at boot time is also a good idea for machines that run ntpd(8). The ntpd(8) program changes the clock gradually, whereas ntpdate(8) sets the clock, no matter how great the difference between a machine’s current clock setting and the correct time.

To enable ntpdate(8) at boot time, add ntpdate_enable="YES" to /etc/rc.conf. You will also need to specify all servers you wish to synchronize with and any flags to be passed to ntpdate(8) in ntpdate_flags.

29.10.3.2 General Configuration

NTP is configured by the /etc/ntp.conf file in the format described in ntp.conf(5). Here is a simple example:

server ntplocal.example.com prefer
server timeserver.example.org
server ntp2a.example.net

driftfile /var/db/ntp.drift

The server option specifies which servers are to be used, with one server listed on each line. If a server is specified with the prefer argument, as with ntplocal.example.com, that server is preferred over other servers. A response from a preferred server will be discarded if it differs significantly from other servers’ responses, otherwise it will be used without any consideration to other responses. The prefer argument is normally used for NTP servers that are known to be highly accurate, such as those with special time monitoring hardware.

The driftfile option specifies which file is used to store the system clock’s frequency offset. The ntpd(8) program uses this to automatically compensate for the clock’s natural drift, allowing it to maintain a reasonably correct setting even if it is cut off from all external time sources for a period of time.

The driftfile option specifies which file is used to store information about previous responses from the NTP servers you are using. This file contains internal information for NTP. It should not be modified by any other process.

29.10.3.3 Controlling Access to Your Server

By default, your NTP server will be accessible to all hosts on the Internet. The restrict option in /etc/ntp.conf allows you to control which machines can access your server.

If you want to deny all machines from accessing your NTP server, add the following line to /etc/ntp.conf:

restrict default ignore

Note: This will also prevent access from your server to any servers listed in your local configuration. If you need to synchronise your NTP server with an external NTP server you should allow the specific server. See the ntp.conf(5) manual for more information.

If you only want to allow machines within your own network to synchronize their clocks with your server, but ensure they are not allowed to configure the server or used as peers to synchronize against, add

restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap

instead, where 192.168.1.0 is an IP address on your network and 255.255.255.0 is your network’s netmask.

/etc/ntp.conf can contain multiple restrict options. For more details, see the Access Control Support subsection of ntp.conf(5).

29.10.4 Running the NTP Server

To ensure the NTP server is started at boot time, add the line ntpd_enable="YES" to /etc/rc.conf. If you wish to pass additional flags to ntpd(8), edit the ntpd_flags parameter in /etc/rc.conf.

To start the server without rebooting your machine, run ntpd being sure to specify any additional parameters from ntpd_flags in /etc/rc.conf. For example:

# ntpd -p /var/run/ntpd.pid

29.10.5 Using ntpd with a Temporary Internet Connection

The ntpd(8) program does not need a permanent connection to the Internet to function properly. However, if you have a temporary connection that is configured to dial out on demand, it is a good idea to prevent NTP traffic from triggering a dial out or keeping the connection alive. If you are using user PPP, you can use filter directives in /etc/ppp/ppp.conf. For example:

 set filter dial 0 deny udp src eq 123
 # Prevent NTP traffic from initiating dial out
 set filter dial 1 permit 0 0
 set filter alive 0 deny udp src eq 123
 # Prevent incoming NTP traffic from keeping the connection open
 set filter alive 1 deny udp dst eq 123
 # Prevent outgoing NTP traffic from keeping the connection open
 set filter alive 2 permit 0/0 0/0

For more details see the PACKET FILTERING section in ppp(8) and the examples in /usr/share/examples/ppp/.

Note: Some Internet access providers block low-numbered ports, preventing NTP from functioning since replies never reach your machine.

29.10.6 Further Information

Documentation for the NTP server can be found in /usr/share/doc/ntp/ in HTML format.

29.11 Remote Host Logging with `syslogd`

Contributed by Tom Rhodes.

Interacting with system logs is a crucial aspect of both security and system administration. Monitoring the log files of multiple hosts can get very unwieldy when these hosts are distributed across medium or large networks, or when they are parts of various different types of networks. In these cases, configuring remote logging may make the whole process a lot more comfortable.

Centralized logging to a specific logging host can reduce some of the administrative burden of log file administration. Log file aggregation, merging and rotation can be configured in one location, using the native tools of FreeBSD, such as syslogd(8) and newsyslog(8). In the following example configuration, host A, named logserv.example.com, will collect logging information for the local network. Host B, named logclient.example.com will pass logging information to the server system. In live configurations, both hosts require proper forward and reverse DNS or entries in /etc/hosts. Otherwise, data will be rejected by the server.

29.11.1 Log Server Configuration

Log servers are machines configured to accept logging information from remote hosts. In most cases this is to ease configuration, in other cases it may just be a better administration move. Regardless of reason, there are a few requirements before continuing.

A properly configured logging server has met the following minimal requirements:

The firewall ruleset allows for UDP to be passed on port 514 on both the client and server;
syslogd has been configured to accept remote messages from client machines;
The syslogd server and all client machines must have valid entries for both forward and reverse DNS, or be properly configured in /etc/hosts.

To configure the log server, the client must be listed in /etc/syslog.conf, and the logging facility must be specified:

+logclient.example.com
*.*     /var/log/logclient.log

Note: More information on various supported and available facilities may be found in the syslog.conf(5) manual page.

Once added, all facility messages will be logged to the file specified previously, /var/log/logclient.log.

The server machine must also have the following listing placed inside /etc/rc.conf:

syslogd_enable="YES"
syslogd_flags="-a logclient.example.com -vv"

The first option will enable the syslogd daemon on boot up, and the second option allows data from the specified client to be accepted on this server. The latter part, using -vv, will increase the verbosity of logged messages. This is extremely useful for tweaking facilities as administrators are able to see what type of messages are being logged under which facility.

Multiple -a options may be specified to allow logging from multiple clients. IP addresses and whole netblocks may also be specified, see the syslog(3) manual page for a full list of possible options.

Finally, the log file should be created. The method used does not matter, but touch(1) works great for situations such as this:

# touch /var/log/logclient.log

At this point, the syslogd daemon should be restarted and verified:

# /etc/rc.d/syslogd restart
# pgrep syslog

If a PID is returned, the server has been restarted successfully, and client configuration may begin. If the server has not restarted, consult the /var/log/messages log for any output.

29.11.2 Log Client Configuration

A logging client is a machine which sends log information to a logging server in addition to keeping local copies.

Similar to log servers, clients must also meet a few minimum requirements:

syslogd(8) must be configured to send messages of specific types to a log server, which must accept them;
The firewall must allow UDP packets through on port 514;
Both forward and reverse DNS must be configured or have proper entries in the /etc/hosts.

Client configuration is a bit more relaxed when compared to that of the servers. The client machine must have the following listing placed inside /etc/rc.conf:

syslogd_enable="YES"
syslogd_flags="-s -vv"

As before, these entries will enable the syslogd daemon on boot up, and increases the verbosity of logged messages. The -s option prevents logs from being accepted by this client from other hosts.

Facilities describe the system part for which a message is generated. For an example, ftp and ipfw are both facilities. When log messages are generated for those two services, they will normally include those two utilities in any log messages. Facilities are accompanied with a priority or level, which is used to mark how important a log message is. The most common will be the warning and info. Please refer to the syslog(3) manual page for a full list of available facilities and priorities.

The logging server must be defined in the client’s /etc/syslog.conf. In this instance, the @ symbol is used to send logging data to a remote server and would look similar to the following entry:

*.*        @logserv.example.com

Once added, syslogd must be restarted for the changes to take effect:

# /etc/rc.d/syslogd restart

To test that log messages are being sent across the network, use logger(1) on the client to send a message to syslogd:

# logger "Test message from logclient"

This message should now exist both in /var/log/messages on the client, and /var/log/logclient.log on the log server.

29.11.3 Debugging Log Servers

In certain cases, debugging may be required if messages are not being received on the log server. There are several reasons this may occur; however, the most common two are network connection issues and DNS issues. To test these cases, ensure both hosts are able to reach one another using the hostname specified in /etc/rc.conf. If this appears to be working properly, an alternation to the syslogd_flags option in /etc/rc.conf will be required.

In the following example, /var/log/logclient.log is empty, and the /var/log/messages files indicate no reason for the failure. To increase debugging output, change the syslogd_flags option to look like the following example, and issue a restart:

syslogd_flags="-d -a logclien.example.com -vv"

# /etc/rc.d/syslogd restart

Debugging data similar to the following will flash on the screen immediately after the restart:

logmsg: pri 56, flags 4, from logserv.example.com, msg syslogd: restart
syslogd: restarted
logmsg: pri 6, flags 4, from logserv.example.com, msg syslogd: kernel boot file is /boot/kernel/kernel
Logging to FILE /var/log/messages
syslogd: kernel boot file is /boot/kernel/kernel
cvthname(192.168.1.10)
validate: dgram from IP 192.168.1.10, port 514, name logclient.example.com;
rejected in rule 0 due to name mismatch.

It appears obvious the messages are being rejected due to a name mismatch. After reviewing the configuration bit by bit, it appears a typo in the following /etc/rc.conf line has an issue:

syslogd_flags="-d -a logclien.example.com -vv"

The line should contain logclient, not logclien. After the proper alterations are made, a restart is issued with expected results:

# /etc/rc.d/syslogd restart
logmsg: pri 56, flags 4, from logserv.example.com, msg syslogd: restart
syslogd: restarted
logmsg: pri 6, flags 4, from logserv.example.com, msg syslogd: kernel boot file is /boot/kernel/kernel
syslogd: kernel boot file is /boot/kernel/kernel
logmsg: pri 166, flags 17, from logserv.example.com,
msg Dec 10 20:55:02 <syslog.err> logserv.example.com syslogd: exiting on signal 2
cvthname(192.168.1.10)
validate: dgram from IP 192.168.1.10, port 514, name logclient.example.com;
accepted in rule 0.
logmsg: pri 15, flags 0, from logclient.example.com, msg Dec 11 02:01:28 trhodes: Test message 2
Logging to FILE /var/log/logclient.log
Logging to FILE /var/log/messages

At this point, the messages are being properly received and placed in the correct file.

29.11.4 Security Considerations

As with any network service, security requirements should be considered before implementing this configuration. At times, log files may contain sensitive data about services enabled on the local host, user accounts, and configuration data. Network data sent from the client to the server will not be encrypted nor password protected. If a need for encryption exists, it might be possible to use security/stunnel, which will transmit data over an encrypted tunnel.

Local security is also an issue. Log files are not encrypted during use or after log rotation. Local users may access these files to gain additional insight on system configuration. In those cases, setting proper permissions on these files will be critical. The newsyslog(8) utility supports setting permissions on newly created and rotated log files. Setting log files to mode 600 should prevent any unwanted snooping by local users.

Tags: account, analog, Apache, backup, bsd, cron, database, domain, domain name, email, forwarder, freebsd, FreeBSD Handbook, ftp, inetd, manage, mysql, netbsd, openbsd, password, php, pop, postgresql, raw, software, ssl, virus

FreeBSD Handbook account, analog, Apache, backup, bsd, cron, database, domain, domain name, email, forwarder, freebsd, ftp, inetd, manage, mysql, netbsd, openbsd, password, php, pop, postgresql, raw, software, ssl, virus

FreeBSD Handbook - Chapter 28 Electronic Mail

January 6th, 2009

1 comment

Original work by Bill Lloyd. Rewritten by Jim Mock.

28.1 Synopsis

“Electronic Mail”, better known as email, is one of the most widely used forms of communication today. This chapter provides a basic introduction to running a mail server on FreeBSD, as well as an introduction to sending and receiving email using FreeBSD; however, it is not a complete reference and in fact many important considerations are omitted. For more complete coverage of the subject, the reader is referred to the many excellent books listed in Appendix B.

After reading this chapter, you will know:

What software components are involved in sending and receiving electronic mail.
Where basic sendmail configuration files are located in FreeBSD.
The difference between remote and local mailboxes.
How to block spammers from illegally using your mail server as a relay.
How to install and configure an alternate Mail Transfer Agent on your system, replacing sendmail.
How to troubleshoot common mail server problems.
How to use SMTP with UUCP.
How to set up the system to send mail only.
How to use mail with a dialup connection.
How to configure SMTP Authentication for added security.
How to install and use a Mail User Agent, such as mutt to send and receive email.
How to download your mail from a remote POP or IMAP server.
How to automatically apply filters and rules to incoming email.

Before reading this chapter, you should:

Properly set up your network connection (Chapter 31).
Properly set up the DNS information for your mail host (Chapter 29).
Know how to install additional third-party software (Chapter 4).

28.2 Using Electronic Mail

There are five major parts involved in an email exchange. They are: the user program, the server daemon, DNS, a remote or local mailbox, and of course, the mailhost itself.

28.2.1 The User Program

This includes command line programs such as mutt, pine, elm, and mail, and GUI programs such as balsa, xfmail to name a few, and something more “sophisticated” like a WWW browser. These programs simply pass off the email transactions to the local “mailhost”, either by calling one of the server daemons available, or delivering it over TCP.

28.2.2 Mailhost Server Daemon

FreeBSD ships with sendmail by default, but also support numerous other mail server daemons, just some of which include:

exim;
postfix;
qmail.

The server daemon usually has two functions–it is responsible for receiving incoming mail as well as delivering outgoing mail. It is not responsible for the collection of mail using protocols such as POP or IMAP to read your email, nor does it allow connecting to local mbox or Maildir mailboxes. You may require an additional daemon for that.

Warning: Older versions of sendmail have some serious security issues which may result in an attacker gaining local and/or remote access to your machine. Make sure that you are running a current version to avoid these problems. Optionally, install an alternative MTA from the FreeBSD Ports Collection.

28.2.3 Email and DNS

The Domain Name System (DNS) and its daemon named play a large role in the delivery of email. In order to deliver mail from your site to another, the server daemon will look up the remote site in the DNS to determine the host that will receive mail for the destination. This process also occurs when mail is sent from a remote host to your mail server.

DNS is responsible for mapping hostnames to IP addresses, as well as for storing information specific to mail delivery, known as MX records. The MX (Mail eXchanger) record specifies which host, or hosts, will receive mail for a particular domain. If you do not have an MX record for your hostname or domain, the mail will be delivered directly to your host provided you have an A record pointing your hostname to your IP address.

You may view the MX records for any domain by using the host(1) command, as seen in the example below:

% host -t mx FreeBSD.org
FreeBSD.org mail is handled (pri=10) by mx1.FreeBSD.org

28.2.4 Receiving Mail

Receiving mail for your domain is done by the mail host. It will collect all mail sent to your domain and store it either in mbox (the default method for storing mail) or Maildir format, depending on your configuration. Once mail has been stored, it may either be read locally using applications such as mail(1) or mutt, or remotely accessed and collected using protocols such as POP or IMAP. This means that should you only wish to read mail locally, you are not required to install a POP or IMAP server.

28.2.4.1 Accessing remote mailboxes using POP and IMAP

In order to access mailboxes remotely, you are required to have access to a POP or IMAP server. These protocols allow users to connect to their mailboxes from remote locations with ease. Though both POP and IMAP allow users to remotely access mailboxes, IMAP offers many advantages, some of which are:

IMAP can store messages on a remote server as well as fetch them.
IMAP supports concurrent updates.
IMAP can be extremely useful over low-speed links as it allows users to fetch the structure of messages without downloading them; it can also perform tasks such as searching on the server in order to minimize data transfer between clients and servers.

In order to install a POP or IMAP server, the following steps should be performed:

Choose an IMAP or POP server that best suits your needs. The following POP and IMAP servers are well known and serve as some good examples:
- qpopper;
- teapop;
- imap-uw;
- courier-imap;
Install the POP or IMAP daemon of your choosing from the ports collection.
Where required, modify /etc/inetd.conf to load the POP or IMAP server.

Warning: It should be noted that both POP and IMAP transmit information, including username and password credentials in clear-text. This means that if you wish to secure the transmission of information across these protocols, you should consider tunneling sessions over ssh(1). Tunneling sessions is described in Section 14.11.8.

28.2.4.2 Accessing local mailboxes

Mailboxes may be accessed locally by directly utilizing MUAs on the server on which the mailbox resides. This can be done using applications such as mutt or mail(1).

28.2.5 The Mail Host

The mail host is the name given to a server that is responsible for delivering and receiving mail for your host, and possibly your network.

28.3 sendmail Configuration

Contributed by Christopher Shumway.

sendmail(8) is the default Mail Transfer Agent (MTA) in FreeBSD. sendmail’s job is to accept mail from Mail User Agents (MUA) and deliver it to the appropriate mailer as defined by its configuration file. sendmail can also accept network connections and deliver mail to local mailboxes or deliver it to another program.

sendmail uses the following configuration files:

Filename

Function

/etc/mail/access

sendmail access database file

/etc/mail/aliases

Mailbox aliases

/etc/mail/local-host-names

Lists of hosts sendmail accepts mail for

/etc/mail/mailer.conf

Mailer program configuration

/etc/mail/mailertable

Mailer delivery table

/etc/mail/sendmail.cf

sendmail master configuration file

/etc/mail/virtusertable

Virtual users and domain tables

28.3.1 `/etc/mail/access`

The access database defines what host(s) or IP addresses have access to the local mail server and what kind of access they have. Hosts can be listed as OK, REJECT, RELAY or simply passed to sendmail’s error handling routine with a given mailer error. Hosts that are listed as OK, which is the default, are allowed to send mail to this host as long as the mail’s final destination is the local machine. Hosts that are listed as REJECT are rejected for all mail connections. Hosts that have the RELAY option for their hostname are allowed to send mail for any destination through this mail server.

Example 28-1. Configuring the sendmail Access Database

cyberspammer.com                550 We do not accept mail from spammers
FREE.STEALTH.MAILER@            550 We do not accept mail from spammers
another.source.of.spam          REJECT
okay.cyberspammer.com           OK
128.32                          RELAY

In this example we have five entries. Mail senders that match the left hand side of the table are affected by the action on the right side of the table. The first two examples give an error code to sendmail’s error handling routine. The message is printed to the remote host when a mail matches the left hand side of the table. The next entry rejects mail from a specific host on the Internet, another.source.of.spam. The next entry accepts mail connections from a host okay.cyberspammer.com, which is more exact than the cyberspammer.com line above. More specific matches override less exact matches. The last entry allows relaying of electronic mail from hosts with an IP address that begins with 128.32. These hosts would be able to send mail through this mail server that are destined for other mail servers.

When this file is updated, you need to run make in /etc/mail/ to update the database.

28.3.2 `/etc/mail/aliases`

The aliases database contains a list of virtual mailboxes that are expanded to other user(s), files, programs or other aliases. Here are a few examples that can be used in /etc/mail/aliases:

Example 28-2. Mail Aliases

root: localuser
ftp-bugs: joe,eric,paul
bit.bucket:  /dev/null
procmail: "|/usr/local/bin/procmail"

The file format is simple; the mailbox name on the left side of the colon is expanded to the target(s) on the right. The first example simply expands the mailbox root to the mailbox localuser, which is then looked up again in the aliases database. If no match is found, then the message is delivered to the local user localuser. The next example shows a mail list. Mail to the mailbox ftp-bugs is expanded to the three local mailboxes joe, eric, and paul. Note that a remote mailbox could be specified as <user@example.com>. The next example shows writing mail to a file, in this case /dev/null. The last example shows sending mail to a program, in this case the mail message is written to the standard input of /usr/local/bin/procmail through a UNIX pipe.

When this file is updated, you need to run make in /etc/mail/ to update the database.

28.3.3 `/etc/mail/local-host-names`

This is a list of hostnames sendmail(8) is to accept as the local host name. Place any domains or hosts that sendmail is to be receiving mail for. For example, if this mail server was to accept mail for the domain example.com and the host mail.example.com, its local-host-names might look something like this:

example.com
mail.example.com

When this file is updated, sendmail(8) needs to be restarted to read the changes.

28.3.4 `/etc/mail/sendmail.cf`

sendmail’s master configuration file, sendmail.cf controls the overall behavior of sendmail, including everything from rewriting e-mail addresses to printing rejection messages to remote mail servers. Naturally, with such a diverse role, this configuration file is quite complex and its details are a bit out of the scope of this section. Fortunately, this file rarely needs to be changed for standard mail servers.

The master sendmail configuration file can be built from m4(1) macros that define the features and behavior of sendmail. Please see /usr/src/contrib/sendmail/cf/README for some of the details.

When changes to this file are made, sendmail needs to be restarted for the changes to take effect.

28.3.5 `/etc/mail/virtusertable`

The virtusertable maps mail addresses for virtual domains and mailboxes to real mailboxes. These mailboxes can be local, remote, aliases defined in /etc/mail/aliases or files.

Example 28-3. Example Virtual Domain Mail Map

root@example.com                root
postmaster@example.com          postmaster@noc.example.net
@example.com                    joe

In the above example, we have a mapping for a domain example.com. This file is processed in a first match order down the file. The first item maps <root@example.com> to the local mailbox root. The next entry maps <postmaster@example.com> to the mailbox postmaster on the host noc.example.net. Finally, if nothing from example.com has matched so far, it will match the last mapping, which matches every other mail message addressed to someone at example.com. This will be mapped to the local mailbox joe.

28.4 Changing Your Mail Transfer Agent

Written by Andrew Boothman. Information taken from e-mails written by Gregory Neil Shapiro.

As already mentioned, FreeBSD comes with sendmail already installed as your MTA (Mail Transfer Agent). Therefore by default it is in charge of your outgoing and incoming mail.

However, for a variety of reasons, some system administrators want to change their system’s MTA. These reasons range from simply wanting to try out another MTA to needing a specific feature or package which relies on another mailer. Fortunately, whatever the reason, FreeBSD makes it easy to make the change.

28.4.1 Install a New MTA

You have a wide choice of MTAs available. A good starting point is the FreeBSD Ports Collection where you will be able to find many. Of course you are free to use any MTA you want from any location, as long as you can make it run under FreeBSD.

Start by installing your new MTA. Once it is installed it gives you a chance to decide if it really fulfills your needs, and also gives you the opportunity to configure your new software before getting it to take over from sendmail. When doing this, you should be sure that installing the new software will not attempt to overwrite system binaries such as /usr/bin/sendmail. Otherwise, your new mail software has essentially been put into service before you have configured it.

Please refer to your chosen MTA’s documentation for information on how to configure the software you have chosen.

28.4.2 Disable sendmail

Warning: If you disable sendmail’s outgoing mail service, it is important that you replace it with an alternative mail delivery system. If you choose not to, system functions such as periodic(8) will be unable to deliver their results by e-mail as they would normally expect to. Many parts of your system may expect to have a functional sendmail-compatible system. If applications continue to use sendmail’s binaries to try to send e-mail after you have disabled them, mail could go into an inactive sendmail queue, and never be delivered.

In order to completely disable sendmail, including the outgoing mail service, you must use

sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

in /etc/rc.conf.

If you only want to disable sendmail’s incoming mail service, you should set

sendmail_enable="NO"

in /etc/rc.conf. More information on sendmail’s startup options is available from the rc.sendmail(8) manual page.

28.4.3 Running Your New MTA on Boot

The new MTA can be started during boot by adding a configuration line to /etc/rc.conf like the following example for postfix:

# echo 'postfix_enable=“YES”' >> /etc/rc.conf

The MTA will now be automatically started during boot.

28.4.4 Replacing sendmail as the System’s Default Mailer

The program sendmail is so ubiquitous as standard software on UNIX systems that some software just assumes it is already installed and configured. For this reason, many alternative MTA’s provide their own compatible implementations of the sendmail command-line interface; this facilitates using them as “drop-in” replacements for sendmail.

Therefore, if you are using an alternative mailer, you will need to make sure that software trying to execute standard sendmail binaries such as /usr/bin/sendmail actually executes your chosen mailer instead. Fortunately, FreeBSD provides a system called mailwrapper(8) that does this job for you.

When sendmail is operating as installed, you will find something like the following in /etc/mail/mailer.conf:

sendmail    /usr/libexec/sendmail/sendmail
send-mail   /usr/libexec/sendmail/sendmail
mailq       /usr/libexec/sendmail/sendmail
newaliases  /usr/libexec/sendmail/sendmail
hoststat    /usr/libexec/sendmail/sendmail
purgestat   /usr/libexec/sendmail/sendmail

This means that when any of these common commands (such as sendmail itself) are run, the system actually invokes a copy of mailwrapper named sendmail, which checks mailer.conf and executes /usr/libexec/sendmail/sendmail instead. This system makes it easy to change what binaries are actually executed when these default sendmail functions are invoked.

Therefore if you wanted /usr/local/supermailer/bin/sendmail-compat to be run instead of sendmail, you could change /etc/mail/mailer.conf to read:

sendmail    /usr/local/supermailer/bin/sendmail-compat
send-mail   /usr/local/supermailer/bin/sendmail-compat
mailq       /usr/local/supermailer/bin/mailq-compat
newaliases  /usr/local/supermailer/bin/newaliases-compat
hoststat    /usr/local/supermailer/bin/hoststat-compat
purgestat   /usr/local/supermailer/bin/purgestat-compat

28.4.5 Finishing

Once you have everything configured the way you want it, you should either kill the sendmail processes that you no longer need and start the processes belonging to your new software, or simply reboot. Rebooting will also give you the opportunity to ensure that you have correctly configured your system to start your new MTA automatically on boot.

28.5 Troubleshooting

28.5.1. Why do I have to use the FQDN for hosts on my site?
28.5.2. sendmail says “mail loops back to myself”
28.5.3. How can I run a mail server on a dial-up PPP host?
28.5.4. Why do I keep getting “Relaying Denied” errors when sending mail from other hosts?

28.5.1. Why do I have to use the FQDN for hosts on my site?

You will probably find that the host is actually in a different domain; for example, if you are in foo.bar.edu and you wish to reach a host called mumble in the bar.edu domain, you will have to refer to it by the fully-qualified domain name, mumble.bar.edu, instead of just mumble.

Traditionally, this was allowed by BSD BIND resolvers. However the current version of BIND that ships with FreeBSD no longer provides default abbreviations for non-fully qualified domain names other than the domain you are in. So an unqualified host mumble must either be found as mumble.foo.bar.edu, or it will be searched for in the root domain.

This is different from the previous behavior, where the search continued across mumble.bar.edu, and mumble.edu. Have a look at RFC 1535 for why this was considered bad practice, or even a security hole.

As a good workaround, you can place the line:

search foo.bar.edu bar.edu

instead of the previous:

domain foo.bar.edu

into your /etc/resolv.conf. However, make sure that the search order does not go beyond the “boundary between local and public administration”, as RFC 1535 calls it.

28.5.2. sendmail says “mail loops back to myself”

This is answered in the sendmail FAQ as follows:

I'm getting these error messages:

553 MX list for domain.net points back to relay.domain.net
554 <user@domain.net>... Local configuration error

How can I solve this problem?

You have asked mail to the domain (e.g., domain.net) to be
forwarded to a specific host (in this case, relay.domain.net)
by using an MX record, but the relay machine does not recognize
itself as domain.net. Add domain.net to /etc/mail/local-host-names
[known as /etc/sendmail.cw prior to version 8.10]
(if you are using FEATURE(use_cw_file)) or add “Cw domain.net”
to /etc/mail/sendmail.cf.

The sendmail FAQ can be found at http://www.sendmail.org/faq/ and is recommended reading if you want to do any “tweaking” of your mail setup.

28.5.3. How can I run a mail server on a dial-up PPP host?

You want to connect a FreeBSD box on a LAN to the Internet. The FreeBSD box will be a mail gateway for the LAN. The PPP connection is non-dedicated.

There are at least two ways to do this. One way is to use UUCP.

Another way is to get a full-time Internet server to provide secondary MX services for your domain. For example, if your company’s domain is example.com and your Internet service provider has set example.net up to provide secondary MX services to your domain:

example.com.          MX        10      example.com.
                      MX        20      example.net.

Only one host should be specified as the final recipient (add Cw example.com in /etc/mail/sendmail.cf on example.com).

When the sending sendmail is trying to deliver the mail it will try to connect to you (example.com) over the modem link. It will most likely time out because you are not online. The program sendmail will automatically deliver it to the secondary MX site, i.e. your Internet provider (example.net). The secondary MX site will then periodically try to connect to your host and deliver the mail to the primary MX host (example.com).

You might want to use something like this as a login script:

#!/bin/sh
# Put me in /usr/local/bin/pppmyisp
( sleep 60 ; /usr/sbin/sendmail -q ) &
/usr/sbin/ppp -direct pppmyisp

If you are going to create a separate login script for a user you could use sendmail -qRexample.com instead in the script above. This will force all mail in your queue for example.com to be processed immediately.

A further refinement of the situation is as follows:

Message stolen from the FreeBSD Internet service provider’s mailing list.

> we provide the secondary MX for a customer. The customer connects to
> our services several times a day automatically to get the mails to
> his primary MX (We do not call his site when a mail for his domains
> arrived). Our sendmail sends the mailqueue every 30 minutes. At the
> moment he has to stay 30 minutes online to be sure that all mail is
> gone to the primary MX.
>
> Is there a command that would initiate sendmail to send all the mails
> now? The user has not root-privileges on our machine of course.

In the “privacy flags” section of sendmail.cf, there is a
definition Opgoaway,restrictqrun

Remove restrictqrun to allow non-root users to start the queue processing.
You might also like to rearrange the MXs. We are the 1st MX for our
customers like this, and we have defined:

# If we are the best MX for a host, try directly instead of generating
# local config error.
OwTrue

That way a remote site will deliver straight to you, without trying
the customer connection.  You then send to your customer.  Only works for
“hosts”, so you need to get your customer to name their mail
machine “customer.com” as well as
“hostname.customer.com” in the DNS.  Just put an A record in
the DNS for “customer.com”.

28.5.4. Why do I keep getting “Relaying Denied” errors when sending mail from other hosts?

In default FreeBSD installations, sendmail is configured to only send mail from the host it is running on. For example, if a POP server is available, then users will be able to check mail from school, work, or other remote locations but they still will not be able to send outgoing emails from outside locations. Typically, a few moments after the attempt, an email will be sent from MAILER-DAEMON with a “5.7 Relaying Denied” error message.

There are several ways to get around this. The most straightforward solution is to put your ISP’s address in a relay-domains file at /etc/mail/relay-domains. A quick way to do this would be:

# echo "your.isp.example.com" > /etc/mail/relay-domains

After creating or editing this file you must restart sendmail. This works great if you are a server administrator and do not wish to send mail locally, or would like to use a point and click client/system on another machine or even another ISP. It is also very useful if you only have one or two email accounts set up. If there is a large number of addresses to add, you can simply open this file in your favorite text editor and then add the domains, one per line:

your.isp.example.com
other.isp.example.net
users-isp.example.org
www.example.org

Now any mail sent through your system, by any host in this list (provided the user has an account on your system), will succeed. This is a very nice way to allow users to send mail from your system remotely without allowing people to send SPAM through your system.

28.6 Advanced Topics

The following section covers more involved topics such as mail configuration and setting up mail for your entire domain.

28.6.1 Basic Configuration

Out of the box, you should be able to send email to external hosts as long as you have set up /etc/resolv.conf or are running your own name server. If you would like to have mail for your host delivered to the MTA (e.g., sendmail) on your own FreeBSD host, there are two methods:

Run your own name server and have your own domain. For example, FreeBSD.org
Get mail delivered directly to your host. This is done by delivering mail directly to the current DNS name for your machine. For example, example.FreeBSD.org.

Regardless of which of the above you choose, in order to have mail delivered directly to your host, it must have a permanent static IP address (not a dynamic address, as with most PPP dial-up configurations). If you are behind a firewall, it must pass SMTP traffic on to you. If you want to receive mail directly at your host, you need to be sure of either of two things:

Make sure that the (lowest-numbered) MX record in your DNS points to your host’s IP address.
Make sure there is no MX entry in your DNS for your host.

Either of the above will allow you to receive mail directly at your host.

Try this:

# hostname
example.FreeBSD.org
# host example.FreeBSD.org
example.FreeBSD.org has address 204.216.27.XX

If that is what you see, mail directly to <yourlogin@example.FreeBSD.org> should work without problems (assuming sendmail is running correctly on example.FreeBSD.org).

If instead you see something like this:

# host example.FreeBSD.org
example.FreeBSD.org has address 204.216.27.XX
example.FreeBSD.org mail is handled (pri=10) by hub.FreeBSD.org

All mail sent to your host (example.FreeBSD.org) will end up being collected on hub under the same username instead of being sent directly to your host.

The above information is handled by your DNS server. The DNS record that carries mail routing information is the Mail eXchange entry. If no MX record exists, mail will be delivered directly to the host by way of its IP address.

The MX entry for freefall.FreeBSD.org at one time looked like this:

freefall       MX  30  mail.crl.net
freefall        MX  40  agora.rdrop.com
freefall        MX  10  freefall.FreeBSD.org
freefall        MX  20  who.cdrom.com

As you can see, freefall had many MX entries. The lowest MX number is the host that receives mail directly if available; if it is not accessible for some reason, the others (sometimes called “backup MXes”) accept messages temporarily, and pass it along when a lower-numbered host becomes available, eventually to the lowest-numbered host.

Alternate MX sites should have separate Internet connections from your own in order to be most useful. Your ISP or another friendly site should have no problem providing this service for you.

28.6.2 Mail for Your Domain

In order to set up a “mailhost” (a.k.a. mail server) you need to have any mail sent to various workstations directed to it. Basically, you want to “claim” any mail for any hostname in your domain (in this case *.FreeBSD.org) and divert it to your mail server so your users can receive their mail on the master mail server.

To make life easiest, a user account with the same username should exist on both machines. Use adduser(8) to do this.

The mailhost you will be using must be the designated mail exchanger for each workstation on the network. This is done in your DNS configuration like so:

example.FreeBSD.org    A   204.216.27.XX       ; Workstation
            MX  10 hub.FreeBSD.org  ; Mailhost

This will redirect mail for the workstation to the mailhost no matter where the A record points. The mail is sent to the MX host.

You cannot do this yourself unless you are running a DNS server. If you are not, or cannot run your own DNS server, talk to your ISP or whoever provides your DNS.

If you are doing virtual email hosting, the following information will come in handy. For this example, we will assume you have a customer with his own domain, in this case customer1.org, and you want all the mail for customer1.org sent to your mailhost, mail.myhost.com. The entry in your DNS should look like this:

customer1.org      MX  10  mail.myhost.com

You do not need an A record for customer1.org if you only want to handle email for that domain.

Note: Be aware that pinging customer1.org will not work unless an A record exists for it.

The last thing that you must do is tell sendmail on your mailhost what domains and/or hostnames it should be accepting mail for. There are a few different ways this can be done. Either of the following will work:

Add the hosts to your /etc/mail/local-host-names file if you are using the FEATURE(use_cw_file). If you are using a version of sendmail earlier than 8.10, the file is /etc/sendmail.cw.
Add a Cwyour.host.com line to your /etc/sendmail.cf or /etc/mail/sendmail.cf if you are using sendmail 8.10 or higher.

28.7 SMTP with UUCP

The sendmail configuration that ships with FreeBSD is designed for sites that connect directly to the Internet. Sites that wish to exchange their mail via UUCP must install another sendmail configuration file.

Tweaking /etc/mail/sendmail.cf manually is an advanced topic. sendmail version 8 generates config files via m4(1) preprocessing, where the actual configuration occurs on a higher abstraction level. The m4(1) configuration files can be found under /usr/share/sendmail/cf. The file README in the cf directory can serve as a basic introduction to m4(1) configuration.

The best way to support UUCP delivery is to use the mailertable feature. This creates a database that sendmail can use to make routing decisions.

First, you have to create your .mc file. The directory /usr/share/sendmail/cf/cf contains a few examples. Assuming you have named your file foo.mc, all you need to do in order to convert it into a valid sendmail.cf is:

# cd /etc/mail
# make foo.cf
# cp foo.cf /etc/mail/sendmail.cf

A typical .mc file might look like:

VERSIONID(`Your version number') OSTYPE(bsd4.4)

FEATURE(accept_unresolvable_domains)
FEATURE(nocanonify)
FEATURE(mailertable, `hash -o /etc/mail/mailertable')

define(`UUCP_RELAY', your.uucp.relay)
define(`UUCP_MAX_SIZE', 200000)
define(`confDONT_PROBE_INTERFACES')

MAILER(local)
MAILER(smtp)
MAILER(uucp)

Cw    your.alias.host.name
Cw    youruucpnodename.UUCP

The lines containing accept_unresolvable_domains, nocanonify, and confDONT_PROBE_INTERFACES features will prevent any usage of the DNS during mail delivery. The UUCP_RELAY clause is needed to support UUCP delivery. Simply put an Internet hostname there that is able to handle .UUCP pseudo-domain addresses; most likely, you will enter the mail relay of your ISP there.

Once you have this, you need an /etc/mail/mailertable file. If you have only one link to the outside that is used for all your mails, the following file will suffice:

#
# makemap hash /etc/mail/mailertable.db < /etc/mail/mailertable
.                             uucp-dom:your.uucp.relay

A more complex example might look like this:

#
# makemap hash /etc/mail/mailertable.db < /etc/mail/mailertable
#
horus.interface-business.de   uucp-dom:horus
.interface-business.de        uucp-dom:if-bus
interface-business.de         uucp-dom:if-bus
.heep.sax.de                  smtp8:%1
horus.UUCP                    uucp-dom:horus
if-bus.UUCP                   uucp-dom:if-bus
.                             uucp-dom:

The first three lines handle special cases where domain-addressed mail should not be sent out to the default route, but instead to some UUCP neighbor in order to “shortcut” the delivery path. The next line handles mail to the local Ethernet domain that can be delivered using SMTP. Finally, the UUCP neighbors are mentioned in the .UUCP pseudo-domain notation, to allow for a uucp-neighbor !recipient override of the default rules. The last line is always a single dot, matching everything else, with UUCP delivery to a UUCP neighbor that serves as your universal mail gateway to the world. All of the node names behind the uucp-dom: keyword must be valid UUCP neighbors, as you can verify using the command uuname.

As a reminder that this file needs to be converted into a DBM database file before use. The command line to accomplish this is best placed as a comment at the top of the mailertable file. You always have to execute this command each time you change your mailertable file.

Final hint: if you are uncertain whether some particular mail routing would work, remember the -bt option to sendmail. It starts sendmail in address test mode; simply enter 3,0, followed by the address you wish to test for the mail routing. The last line tells you the used internal mail agent, the destination host this agent will be called with, and the (possibly translated) address. Leave this mode by typing Ctrl+D.

% sendmail -bt
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter <ruleset> <address>
> 3,0 foo@example.com
canonify           input: foo @ example . com
...
parse            returns: $# uucp-dom $@ your.uucp.relay $: foo < @ example . com . >
> ^D

28.8 Setting Up to Send Only

Contributed by Bill Moran.

There are many instances where you may only want to send mail through a relay. Some examples are:

Your computer is a desktop machine, but you want to use programs such as send-pr(1). To do so, you should use your ISP’s mail relay.
The computer is a server that does not handle mail locally, but needs to pass off all mail to a relay for processing.

Just about any MTA is capable of filling this particular niche. Unfortunately, it can be very difficult to properly configure a full-featured MTA just to handle offloading mail. Programs such as sendmail and postfix are largely overkill for this use.

Additionally, if you are using a typical Internet access service, your agreement may forbid you from running a “mail server”.

The easiest way to fulfill those needs is to install the mail/ssmtp port. Execute the following commands as root:

# cd /usr/ports/mail/ssmtp
# make install replace clean

Once installed, mail/ssmtp can be configured with a four-line file located at /usr/local/etc/ssmtp/ssmtp.conf:

root=yourrealemail@example.com
mailhub=mail.example.com
rewriteDomain=example.com
hostname=_HOSTNAME_

Make sure you use your real email address for root. Enter your ISP’s outgoing mail relay in place of mail.example.com (some ISPs call this the “outgoing mail server” or “SMTP server”).

Make sure you disable sendmail, including the outgoing mail service. See Section 28.4.2 for details.

mail/ssmtp has some other options available. See the example configuration file in /usr/local/etc/ssmtp or the manual page of ssmtp for some examples and more information.

Setting up ssmtp in this manner will allow any software on your computer that needs to send mail to function properly, while not violating your ISP’s usage policy or allowing your computer to be hijacked for spamming.

28.9 Using Mail with a Dialup Connection

If you have a static IP address, you should not need to adjust anything from the defaults. Set your host name to your assigned Internet name and sendmail will do the rest.

If you have a dynamically assigned IP number and use a dialup PPP connection to the Internet, you will probably have a mailbox on your ISPs mail server. Let’s assume your ISP’s domain is example.net, and that your user name is user, you have called your machine bsd.home, and your ISP has told you that you may use relay.example.net as a mail relay.

In order to retrieve mail from your mailbox, you must install a retrieval agent. The fetchmail utility is a good choice as it supports many different protocols. This program is available as a package or from the Ports Collection (mail/fetchmail). Usually, your ISP will provide POP. If you are using user PPP, you can automatically fetch your mail when an Internet connection is established with the following entry in /etc/ppp/ppp.linkup:

MYADDR:
!bg su user -c fetchmail

If you are using sendmail (as shown below) to deliver mail to non-local accounts, you probably want to have sendmail process your mailqueue as soon as your Internet connection is established. To do this, put this command after the fetchmail command in /etc/ppp/ppp.linkup:

  !bg su user -c "sendmail -q"

Assume that you have an account for user on bsd.home. In the home directory of user on bsd.home, create a .fetchmailrc file:

poll example.net protocol pop3 fetchall pass MySecret

This file should not be readable by anyone except user as it contains the password MySecret.

In order to send mail with the correct from: header, you must tell sendmail to use <user@example.net> rather than <user@bsd.home>. You may also wish to tell sendmail to send all mail via relay.example.net, allowing quicker mail transmission.

The following .mc file should suffice:

VERSIONID(`bsd.home.mc version 1.0')
OSTYPE(bsd4.4)dnl
FEATURE(nouucp)dnl
MAILER(local)dnl
MAILER(smtp)dnl
Cwlocalhost
Cwbsd.home
MASQUERADE_AS(`example.net')dnl
FEATURE(allmasquerade)dnl
FEATURE(masquerade_envelope)dnl
FEATURE(nocanonify)dnl
FEATURE(nodns)dnl
define(`SMART_HOST', `relay.example.net')
Dmbsd.home
define(`confDOMAIN_NAME',`bsd.home')dnl
define(`confDELIVERY_MODE',`deferred')dnl

Refer to the previous section for details of how to turn this .mc file into a sendmail.cf file. Also, do not forget to restart sendmail after updating sendmail.cf.

28.10 SMTP Authentication

Written by James Gorham.

Having SMTP Authentication in place on your mail server has a number of benefits. SMTP Authentication can add another layer of security to sendmail, and has the benefit of giving mobile users who switch hosts the ability to use the same mail server without the need to reconfigure their mail client settings each time.

Install security/cyrus-sasl2 from the ports. You can find this port in security/cyrus-sasl2. The security/cyrus-sasl2 port supports a number of compile-time options. For the SMTP Authentication method we will be using here, make sure that the LOGIN option is not disabled.
After installing security/cyrus-sasl2, edit /usr/local/lib/sasl2/Sendmail.conf (or create it if it does not exist) and add the following line:
```
pwcheck_method: saslauthd
```
Next, install security/cyrus-sasl2-saslauthd, edit /etc/rc.conf to add the following line:
```
saslauthd_enable="YES"
```
and finally start the saslauthd daemon:
```
# /usr/local/etc/rc.d/saslauthd start
```
This daemon serves as a broker for sendmail to authenticate against your FreeBSD passwd database. This saves the trouble of creating a new set of usernames and passwords for each user that needs to use SMTP authentication, and keeps the login and mail password the same.
Now edit /etc/make.conf and add the following lines:
```
SENDMAIL_CFLAGS=-I/usr/local/include/sasl -DSASL
SENDMAIL_LDFLAGS=-L/usr/local/lib
SENDMAIL_LDADD=-lsasl2
```
These lines will give sendmail the proper configuration options for linking to cyrus-sasl2 at compile time. Make sure that cyrus-sasl2 has been installed before recompiling sendmail.

Recompile sendmail by executing the following commands:

# cd /usr/src/lib/libsmutil
# make cleandir && make obj && make
# cd /usr/src/lib/libsm
# make cleandir && make obj && make
# cd /usr/src/usr.sbin/sendmail
# make cleandir && make obj && make && make install

The compile of sendmail should not have any problems if /usr/src has not been changed extensively and the shared libraries it needs are available.

After sendmail has been compiled and reinstalled, edit your /etc/mail/freebsd.mc file (or whichever file you use as your .mc file. Many administrators choose to use the output from hostname(1) as the .mc file for uniqueness). Add these lines to it:
```
dnl set SASL options
TRUST_AUTH_MECH(`GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl
define(`confAUTH_MECHANISMS', `GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl
```
These options configure the different methods available to sendmail for authenticating users. If you would like to use a method other than pwcheck, please see the included documentation.
Finally, run make(1) while in /etc/mail. That will run your new .mc file and create a .cf file named freebsd.cf (or whatever name you have used for your .mc file). Then use the command make install restart, which will copy the file to sendmail.cf, and will properly restart sendmail. For more information about this process, you should refer to /etc/mail/Makefile.

If all has gone correctly, you should be able to enter your login information into the mail client and send a test message. For further investigation, set the LogLevel of sendmail to 13 and watch /var/log/maillog for any errors.

For more information, please see the sendmail page regarding SMTP authentication.

28.11 Mail User Agents

Contributed by Marc Silver.

A Mail User Agent (MUA) is an application that is used to send and receive email. Furthermore, as email “evolves” and becomes more complex, MUA’s are becoming increasingly powerful in the way they interact with email; this gives users increased functionality and flexibility. FreeBSD contains support for numerous mail user agents, all of which can be easily installed using the FreeBSD Ports Collection. Users may choose between graphical email clients such as evolution or balsa, console based clients such as mutt, pine or mail, or the web interfaces used by some large organizations.

28.11.1 mail

mail(1) is the default Mail User Agent (MUA) in FreeBSD. It is a console based MUA that offers all the basic functionality required to send and receive text-based email, though it is limited in interaction abilities with attachments and can only support local mailboxes.

Although mail does not natively support interaction with POP or IMAP servers, these mailboxes may be downloaded to a local mbox file using an application such as fetchmail, which will be discussed later in this chapter (Section 28.12).

In order to send and receive email, simply invoke the mail command as per the following example:

% mail

The contents of the user mailbox in /var/mail are automatically read by the mail utility. Should the mailbox be empty, the utility exits with a message indicating that no mails could be found. Once the mailbox has been read, the application interface is started, and a list of messages will be displayed. Messages are automatically numbered, as can be seen in the following example:

Mail version 8.1 6/6/93.  Type ? for help.
"/var/mail/marcs": 3 messages 3 new
>N  1 root@localhost        Mon Mar  8 14:05  14/510   "test"
 N  2 root@localhost        Mon Mar  8 14:05  14/509   "user account"
 N  3 root@localhost        Mon Mar  8 14:05  14/509   "sample"

Messages can now be read by using the t mail command, suffixed by the message number that should be displayed. In this example, we will read the first email:

& t 1
Message 1:
From root@localhost  Mon Mar  8 14:05:52 2004
X-Original-To: marcs@localhost
Delivered-To: marcs@localhost
To: marcs@localhost
Subject: test
Date: Mon,  8 Mar 2004 14:05:52 +0200 (SAST)
From: root@localhost (Charlie Root)

This is a test message, please reply if you receive it.

As can be seen in the example above, the t key will cause the message to be displayed with full headers. To display the list of messages again, the h key should be used.

If the email requires a response, you may use mail to reply, by using either the R or r mail keys. The R key instructs mail to reply only to the sender of the email, while r replies not only to the sender, but also to other recipients of the message. You may also suffix these commands with the mail number which you would like make a reply to. Once this has been done, the response should be entered, and the end of the message should be marked by a single . on a new line. An example can be seen below:

& R 1
To: root@localhost
Subject: Re: test

Thank you, I did get your email.
.
EOT

In order to send new email, the m key should be used, followed by the recipient email address. Multiple recipients may also be specified by separating each address with the , delimiter. The subject of the message may then be entered, followed by the message contents. The end of the message should be specified by putting a single . on a new line.

& mail root@localhost
Subject: I mastered mail

Now I can send and receive email using mail ...  .
EOT

While inside the mail utility, the ? command may be used to display help at any time, the mail(1) manual page should also be consulted for more help with mail.

Note: As previously mentioned, the mail(1) command was not originally designed to handle attachments, and thus deals with them very poorly. Newer MUAs such as mutt handle attachments in a much more intelligent way. But should you still wish to use the mail command, the converters/mpack port may be of considerable use.

28.11.2 mutt

mutt is a small yet very powerful Mail User Agent, with excellent features, just some of which include:

The ability to thread messages;
PGP support for digital signing and encryption of email;
MIME Support;
Maildir Support;
Highly customizable.

All of these features help to make mutt one of the most advanced mail user agents available. See http://www.mutt.org for more information on mutt.

The stable version of mutt may be installed using the mail/mutt port, while the current development version may be installed via the mail/mutt-devel port. After the port has been installed, mutt can be started by issuing the following command:

% mutt

mutt will automatically read the contents of the user mailbox in /var/mail and display the contents if applicable. If no mails are found in the user mailbox, then mutt will wait for commands from the user. The example below shows mutt displaying a list of messages:

In order to read an email, simply select it using the cursor keys, and press the Enter key. An example of mutt displaying email can be seen below:

As with the mail(1) command, mutt allows users to reply only to the sender of the message as well as to all recipients. To reply only to the sender of the email, use the r keyboard shortcut. To send a group reply, which will be sent to the original sender as well as all the message recipients, use the g shortcut.

Note: mutt makes use of the vi(1) command as an editor for creating and replying to emails. This may be customized by the user by creating or editing their own .muttrc file in their home directory and setting the editor variable or by setting the EDITOR environment variable. See http://www.mutt.org/ for more information about configuring mutt.

In order to compose a new mail message, press m. After a valid subject has been given, mutt will start vi(1) and the mail can be written. Once the contents of the mail are complete, save and quit from vi and mutt will resume, displaying a summary screen of the mail that is to be delivered. In order to send the mail, press y. An example of the summary screen can be seen below:

mutt also contains extensive help, which can be accessed from most of the menus by pressing the ? key. The top line also displays the keyboard shortcuts where appropriate.

28.11.3 pine

pine is aimed at a beginner user, but also includes some advanced features.

Warning: The pine software has had several remote vulnerabilities discovered in the past, which allowed remote attackers to execute arbitrary code as users on the local system, by the action of sending a specially-prepared email. All such known problems have been fixed, but the pine code is written in a very insecure style and the FreeBSD Security Officer believes there are likely to be other undiscovered vulnerabilities. You install pine at your own risk.

The current version of pine may be installed using the mail/pine4 port. Once the port has installed, pine can be started by issuing the following command:

% pine

The first time that pine is run it displays a greeting page with a brief introduction, as well as a request from the pine development team to send an anonymous email message allowing them to judge how many users are using their client. To send this anonymous message, press Enter, or alternatively press E to exit the greeting without sending an anonymous message. An example of the greeting page can be seen below:

Users are then presented with the main menu, which can be easily navigated using the cursor keys. This main menu provides shortcuts for the composing new mails, browsing of mail directories, and even the administration of address book entries. Below the main menu, relevant keyboard shortcuts to perform functions specific to the task at hand are shown.

The default directory opened by pine is the inbox. To view the message index, press I, or select the MESSAGE INDEX option as seen below:

The message index shows messages in the current directory, and can be navigated by using the cursor keys. Highlighted messages can be read by pressing the Enter key.

In the screenshot below, a sample message is displayed by pine. Keyboard shortcuts are displayed as a reference at the bottom of the screen. An example of one of these shortcuts is the r key, which tells the MUA to reply to the current message being displayed.

Replying to an email in pine is done using the pico editor, which is installed by default with pine. The pico utility makes it easy to navigate around the message and is slightly more forgiving on novice users than vi(1) or mail(1). Once the reply is complete, the message can be sent by pressing Ctrl+X. The pine application will ask for confirmation.

The pine application can be customized using the SETUP option from the main menu. Consult http://www.washington.edu/pine/ for more information.

28.12 Using fetchmail

Contributed by Marc Silver.

fetchmail is a full-featured IMAP and POP client which allows users to automatically download mail from remote IMAP and POP servers and save it into local mailboxes; there it can be accessed more easily. fetchmail can be installed using the mail/fetchmail port, and offers various features, some of which include:

Support of POP3, APOP, KPOP, IMAP, ETRN and ODMR protocols.
Ability to forward mail using SMTP, which allows filtering, forwarding, and aliasing to function normally.
May be run in daemon mode to check periodically for new messages.
Can retrieve multiple mailboxes and forward them based on configuration, to different local users.

While it is outside the scope of this document to explain all of fetchmail’s features, some basic features will be explained. The fetchmail utility requires a configuration file known as .fetchmailrc, in order to run correctly. This file includes server information as well as login credentials. Due to the sensitive nature of the contents of this file, it is advisable to make it readable only by the owner, with the following command:

% chmod 600 .fetchmailrc

The following .fetchmailrc serves as an example for downloading a single user mailbox using POP. It tells fetchmail to connect to example.com using a username of joesoap and a password of XXX. This example assumes that the user joesoap is also a user on the local system.

poll example.com protocol pop3 username "joesoap" password "XXX"

The next example connects to multiple POP and IMAP servers and redirects to different local usernames where applicable:

poll example.com proto pop3:
user "joesoap", with password "XXX", is "jsoap" here;
user "andrea", with password "XXXX";
poll example2.net proto imap:
user "john", with password "XXXXX", is "myth" here;

The fetchmail utility can be run in daemon mode by running it with the -d flag, followed by the interval (in seconds) that fetchmail should poll servers listed in the .fetchmailrc file. The following example would cause fetchmail to poll every 600 seconds:

% fetchmail -d 600

More information on fetchmail can be found at http://fetchmail.berlios.de/.

28.13 Using procmail

Contributed by Marc Silver.

The procmail utility is an incredibly powerful application used to filter incoming mail. It allows users to define “rules” which can be matched to incoming mails to perform specific functions or to reroute mail to alternative mailboxes and/or email addresses. procmail can be installed using the mail/procmail port. Once installed, it can be directly integrated into most MTAs; consult your MTA documentation for more information. Alternatively, procmail can be integrated by adding the following line to a .forward in the home directory of the user utilizing procmail features:

"|exec /usr/local/bin/procmail || exit 75"

The following section will display some basic procmail rules, as well as brief descriptions on what they do. These rules, and others must be inserted into a .procmailrc file, which must reside in the user’s home directory.

The majority of these rules can also be found in the procmailex(5) manual page.

Forward all mail from <user@example.com> to an external address of <goodmail@example2.com>:

:0
* ^From.*user@example.com
! goodmail@example2.com

Forward all mails shorter than 1000 bytes to an external address of <goodmail@example2.com>:

:0
* < 1000
! goodmail@example2.com

Send all mail sent to <alternate@example.com> into a mailbox called alternate:

:0
* ^TOalternate@example.com
alternate

Send all mail with a subject of “Spam” to /dev/null:

:0
^Subject:.*Spam
/dev/null

A useful recipe that parses incoming FreeBSD.org mailing lists and places each list in its own mailbox:

:0
* ^Sender:.owner-freebsd-\/[^@]+@FreeBSD.ORG
{
    LISTNAME=${MATCH}
    :0
    * LISTNAME??^\/[^@]+
    FreeBSD-${MATCH}
}

Tags: account, agora, backup, bsd, cron, database, domain, domain name, email, filters, freebsd, FreeBSD Handbook, ftp, imap, inetd, password, pop, sendmail, smtp, software, ssh

FreeBSD Handbook account, agora, backup, bsd, cron, database, domain, domain name, email, filters, freebsd, ftp, imap, inetd, password, pop, sendmail, smtp, software, ssh

FreeBSD Handbook - Chapter 24 Updating and Upgrading FreeBSD

January 6th, 2009

No comments

Restructured, reorganized, and parts updated by Jim Mock. Original work by Jordan Hubbard, Poul-Henning Kamp, John Polstra, and Nik Clayton.

24.1 Synopsis

FreeBSD is under constant development between releases. Some people prefer to use the officially released versions, while others prefer to keep in sync with the latest developments. However, even official releases are often updated with security and other critical fixes. Regardless of the version used, FreeBSD provides all necessary tools to keep your system updated, and also allows for easy upgrades between versions. This chapter will help you decide if you want to track the development system, or stick with one of the released versions. The basic tools for keeping your system up to date are also presented.

After reading this chapter, you will know:

What utilities may be used to update the system and the Ports Collection.
How to keep your system up to date with freebsd-update, CVSup, CVS, or CTM.
How to compare the state of an installed system against a known pristine copy.
The difference between the two development branches: FreeBSD-STABLE and FreeBSD-CURRENT.
How to rebuild and reinstall the entire base system with make buildworld (etc).

Before reading this chapter, you should:

Properly set up your network connection (Chapter 31).
Know how to install additional third-party software (Chapter 4).

Note: Throughout this chapter, the cvsup command is used to obtain and update FreeBSD sources. To use it, you will need to install a port or package like net/cvsup-without-gui. If you are using FreeBSD 6.2-RELEASE or later, you may wish to substitute this with csup(1), which is now part of the base system.

24.2 FreeBSD Update

Written by Tom Rhodes. Based on notes provided by Colin Percival.

Applying security patches is an important part of maintaining computer software, especially the operating system. For the longest time on FreeBSD this process was not an easy one. Patches had to be applied to the source code, the code rebuilt into binaries, and then the binaries had to be re-installed.

This is no longer the case as FreeBSD now includes a utility simply called freebsd-update. This utility provides two separate functions. First, it allows for binary security and errata updates to be applied to the FreeBSD base system without the build and install requirements. Second, the utility supports minor and major release upgrades.

Note: Binary updates are available for all architectures and releases currently supported by the security team; however, some features, such as the FreeBSD operating system upgrades, require the latest release of freebsd-update(8) and FreeBSD 6.3 or greater. Before updating to a new release, the current release announcements should be reviewed as they may contain important information pertinent to the desired release. These announcements may be viewed at the following link: http://www.FreeBSD.org/releases/.

If a crontab utilizing the features of freebsd-update exists, it must be disabled before the following operation is started.

24.2.1 The Configuration File

Some users may wish to tweak the configuration file, allowing better control of the process. The options are very well documented, but the following few may require a bit more explanation:

# Components of the base system which should be kept updated.
Components src world kernel

This parameter controls what parts of FreeBSD will be kept up to date. The default is to update the source code, the entire base system, and the kernel. Components are the same as those available during the install, for instance, adding "world/games" here would allow game patches to be applied. Using "src/bin" would allow the source code in src/bin to be updated.

The best option is to leave this at the default as changing it to include specific items will require the user to list every item they prefer to be updated. This could have disastrous consequences as source code and binaries may become out of sync.

# Paths which start with anything matching an entry in an IgnorePaths
# statement will be ignored.
IgnorePaths

Add paths, such as /bin or /sbin to leave these specific directories untouched during the update process. This option may be used to prevent freebsd-update from overwriting local modifications.

# Paths which start with anything matching an entry in an UpdateIfUnmodified
# statement will only be updated if the contents of the file have not been
# modified by the user (unless changes are merged; see below).
UpdateIfUnmodified /etc/ /var/ /root/ /.cshrc /.profile

Update configuration files in the specified directories only if they have not been modified. Any changes made by the user will invalidate the automatic updating of these files. There is another option, KeepModifiedMetadata, which will instruct freebsd-update to save the changes during the merge.

# When upgrading to a new FreeBSD release, files which match MergeChanges
# will have any local changes merged into the version from the new release.
MergeChanges /etc/ /var/named/etc/

List of directories with configuration files that freebsd-update should attempt merges in. The file merge process is a series of diff(1) patches similar to mergemaster(8) with fewer options, the merges are either accepted, open an editor, or freebsd-update will abort. When in doubt, backup /etc and just accept the merges. See Section 24.6.11.1 for more information about the mergemaster command.

# Directory in which to store downloaded updates and temporary
# files used by FreeBSD Update.
# WorkDir /var/db/freebsd-update

This directory is where all patches and temporary files will be placed. In cases where the user is doing a version upgrade, this location should have a least a gigabyte of disk space available.

# When upgrading between releases, should the list of Components be
# read strictly (StrictComponents yes) or merely as a list of components
# which *might* be installed of which FreeBSD Update should figure out
# which actually are installed and upgrade those (StrictComponents no)?
# StrictComponents no

When set to yes, freebsd-update will assume that the Components list is complete and will not attempt to make changes outside of the list. Effectively, freebsd-update will attempt to update every file which belongs to the Components list.

24.2.2 Security Patches

Security patches are stored on a remote machine and may be downloaded and installed using the following command:

# freebsd-update fetch
# freebsd-update install

If any kernel patches have been applied the system will need a reboot. If all went well the system should be patched and freebsd-update may be ran as a nightly cron(8) job. An entry in /etc/crontab would be sufficient to accomplish this task:

@daily                                  root    freebsd-update cron

This entry states that once every day, the freebsd-update will be ran. In this way, using the cron argument, freebsd-update will only check if updates exist. If patches exist, they will automatically be downloaded to the local disk but not applied. The root user will be sent an email so they may install them manually.

If anything went wrong, freebsd-update has the ability to roll back the last set of changes with the following command:

# freebsd-update rollback

Once complete, the system should be restarted if the kernel or any kernel modules were modified. This will allow FreeBSD to load the new binaries into memory.

The freebsd-update utility can automatically update the GENERIC kernel only. If a custom kernel is in use, it will have to be rebuilt and reinstalled after freebsd-update finishes installing the rest of the updates. However, freebsd-update will detect and update the GENERIC kernel in /boot/GENERIC (if it exists), even if it is not the current (running) kernel of the system.

Note: It is a good idea to always keep a copy of the GENERIC kernel in /boot/GENERIC. It will be helpful in diagnosing a variety of problems, and in performing version upgrades using freebsd-update as described in Section 24.2.3.

Unless the default configuration in /etc/freebsd-update.conf has been changed, freebsd-update will install the updated kernel sources along with the rest of the updates. Rebuilding and reinstalling your new custom kernel can then be performed in the usual way.

Note: The updates distributed via freebsd-update, do not always involve the kernel. It will not be necessary to rebuild your custom kernel if the kernel sources have not been modified by the execution of freebsd-update install. However, freebsd-update will always update the /usr/src/sys/conf/newvers.sh file. The current patch level (as indicated by the -p number reported by uname -r) is obtained from this file. Rebuilding your custom kernel, even if nothing else changed, will allow uname(1) to accurately report the current patch level of the system. This is particularly helpful when maintaining multiple systems, as it allows for a quick assessment of the updates installed in each one.

24.2.3 Major and Minor Upgrades

This process will remove old object files and libraries which will break most third party applications. It is recommended that all installed ports either be removed and re-installed or upgraded later using the ports-mgmt/portupgrade utility. Most users will want to run a test build using the following command:

# portupgrade -af

This will ensure everything will be re-installed correctly. Note that setting the BATCH environment variable to yes will answer yes to any prompts during this process, removing the need for manual intervention during the build process.

If a custom kernel is in use, the upgrade process is slightly more involved. A copy of the GENERIC kernel is needed, and it should be placed in /boot/GENERIC. If the GENERIC kernel is not already present in the system, it may be obtained using one of the following methods:

If a custom kernel has only been built once, the kernel in /boot/kernel.old is actually the GENERIC one. Simply rename this directory to /boot/GENERIC.
Assuming physical access to the machine is possible, a copy of the GENERIC kernel can be installed from the CD-ROM media. Insert your installation disc and use the following commands:
```
# mount /cdrom
# cd /cdrom/X.Y-RELEASE/kernels
# ./install.sh GENERIC
```
Replace X.Y-RELEASE with the actual version of the release you are using. The GENERIC kernel will be installed in /boot/GENERIC by default.
Failing all the above, the GENERIC kernel may be rebuilt and installed from the sources:
```
# cd /usr/src
# env DESTDIR=/boot/GENERIC make kernel
# mv /boot/GENERIC/boot/kernel/* /boot/GENERIC
# rm -rf /boot/GENERIC/boot
```
For this kernel to be picked up as GENERIC by freebsd-update, the GENERIC configuration file must not have been modified in any way. It is also suggested that it is built without any other special options (preferably with an empty /etc/make.conf).

Rebooting to the GENERIC kernel is not required at this stage.

Major and minor version updates may be performed by providing freebsd-update with a release version target, for example, the following command will update to FreeBSD 6.4:

# freebsd-update -r 6.4-RELEASE upgrade

After the command has been received, freebsd-update will evaluate the configuration file and current system in an attempt to gather the information necessary to update the system. A screen listing will display what components have been detected and what components have not been detected. For example:

Looking up update.FreeBSD.org mirrors... 1 mirrors found.
Fetching metadata signature for 6.3-RELEASE from update1.FreeBSD.org... done.
Fetching metadata index... done.
Inspecting system... done.

The following components of FreeBSD seem to be installed:
kernel/smp src/base src/bin src/contrib src/crypto src/etc src/games
src/gnu src/include src/krb5 src/lib src/libexec src/release src/rescue
src/sbin src/secure src/share src/sys src/tools src/ubin src/usbin
world/base world/info world/lib32 world/manpages

The following components of FreeBSD do not seem to be installed:
kernel/generic world/catpages world/dict world/doc world/games
world/proflibs

Does this look reasonable (y/n)? y

At this point, freebsd-update will attempt to download all files required for the upgrade. In some cases, the user may be prompted with questions regarding what to install or how to proceed.

When using a custom kernel, the above step will produce a warning similar to the following:

WARNING: This system is running a "MYKERNEL" kernel, which is not a
kernel configuration distributed as part of FreeBSD 6.3-RELEASE.
This kernel will not be updated: you MUST update the kernel manually
before running "/usr/sbin/freebsd-update install"

This warning may be safely ignored at this point. The updated GENERIC kernel will be used as an intermediate step in the upgrade process.

After all patches have been downloaded to the local system, they will then be applied. This process may take a while depending on the speed and workload of the machine. Configuration files will then be merged — this part of the process requires some user intervention as a file may be merged or an editor may appear on screen for a manual merge. The results of every successful merge will be shown to the user as the process continues. A failed or ignored merge will cause the process to abort. Users may wish to make a backup of /etc and manually merge important files, such as master.passwd or group at a later time.

Note: The system is not being altered yet, all patching and merging is happening in another directory. When all patches have been applied successfully, all configuration files have been merged and it seems the process will go smoothly, the changes will need to be committed by the user.

Once this process is complete, the upgrade may be committed to disk using the following command.

# freebsd-update install

The kernel and kernel modules will be patched first. At this point the machine must be rebooted. If the system was running with a custom kernel, use the nextboot(8) command to set the kernel for the next boot to /boot/GENERIC (which was updated):

# nextboot -k GENERIC

Warning: Before rebooting with the GENERIC kernel, make sure it contains all drivers required for your system to boot properly (and connect to the network, if the machine that is being updated is accessed remotely). In particular, if the previously running custom kernel contained built-in functionality usually provided by kernel modules, make sure to temporarily load these modules into the GENERIC kernel using the /boot/loader.conf facility. You may also wish to disable non-essential services, disk and network mounts, etc. until the upgrade process is complete.

The machine should now be restarted with the updated kernel:

# shutdown -r now

Once the system has come back online, freebsd-update will need to be started again. The state of the process has been saved and thus, freebsd-update will not start from the beginning, but will remove all old shared libraries and object files. To continue to this stage, issue the following command:

# freebsd-update install

Note: Depending on whether any libraries version numbers got bumped, there may only be two install phases instead of three.

All third party software will now need to be rebuilt and re-installed. This is required as installed software may depend on libraries which have been removed during the upgrade process. The ports-mgmt/portupgrade command may be used to automate this process. The following commands may be used to begin this process:

# portupgrade -f ruby
# rm /var/db/pkg/pkgdb.db
# portupgrade -f ruby18-bdb
# rm /var/db/pkg/pkgdb.db /usr/ports/INDEX-*.db
# portupgrade -af

Once this has completed, finish the upgrade process with a final call to freebsd-update. Issue the following command to tie up all loose ends in the upgrade process:

# freebsd-update install

If the GENERIC kernel was temporarily used, this is the time to build and install a new custom kernel in the usual way.

Reboot the machine into the new FreeBSD version. The process is complete.

24.2.4 System State Comparison

The freebsd-update utility may be used to test the state of the installed FreeBSD version against a known good copy. This option evaluates the current version of system utilities, libraries, and configuration files. To begin the comparison, issue the following command:

# freebsd-update IDS >> outfile.ids

Warning: While the command name is IDS it should in no way be a replacement for an intrusion detection system such as security/snort. As freebsd-update stores data on disk, the possibility of tampering is evident. While this possibility may be reduced by using the kern.securelevel setting and storing the freebsd-update data on a read only file system when not in use, a better solution would be to compare the system against a secure disk, such as a DVD or securely stored external USB disk device.

The system will now be inspected, and a list of files along with their sha256(1) hash values, both the known value in the release and the current installed value, will be printed. This is why the output has been sent to the outfile.ids file. It scrolls by too quickly for eye comparisons, and soon it fills up the console buffer.

These lines are also extremely long, but the output format may be parsed quite easily. For instance, to obtain a list of all files different from those in the release, issue the following command:

# cat outfile.ids | awk '{ print $1 }' | more
/etc/master.passwd
/etc/motd
/etc/passwd
/etc/pf.conf

This output has been truncated, many more files exist. Some of these files have natural modifications, the /etc/passwd has been modified because users have been added to the system. In some cases, there may be other files, such as kernel modules, which differ as freebsd-update may have updated them. To exclude specific files or directories, add them to the IDSIgnorePaths option in /etc/freebsd-update.conf.

This system may be used as part of an elaborate upgrade method, aside from the previously discussed version.

24.3 Portsnap: A Ports Collection Update Tool

Written by Tom Rhodes. Based on notes provided by Colin Percival.

The base system of FreeBSD includes a utility for updating the Ports Collection too: the portsnap(8) utility. Upon execution, it will connect to a remote site, verify the secure key, and download a new copy of the Ports Collection. The key is used to verify the integrity of all downloaded files, ensuring they have not been modified in-flight. To download the latest Ports Collection files, issue the following command:

# portsnap fetch
Looking up portsnap.FreeBSD.org mirrors... 3 mirrors found.
Fetching snapshot tag from portsnap1.FreeBSD.org... done.
Fetching snapshot metadata... done.
Updating from Wed Aug  6 18:00:22 EDT 2008 to Sat Aug 30 20:24:11 EDT 2008.
Fetching 3 metadata patches.. done.
Applying metadata patches... done.
Fetching 3 metadata files... done.
Fetching 90 patches.....10....20....30....40....50....60....70....80....90. done.
Applying patches... done.
Fetching 133 new ports or files... done.

What this example shows is that portsnap(8) has found and verified several patches to the current ports data. This also indicates that the utility was run previously, if it was a first time run, the collection would have simply been downloaded.

When portsnap(8) successfully completes a fetch operation, the Ports Collection and subsequent patches exist on the local system that have passed verification. The updated files may be installed by typing:

# portsnap extract
/usr/ports/.cvsignore
/usr/ports/CHANGES
/usr/ports/COPYRIGHT
/usr/ports/GIDs
/usr/ports/KNOBS
/usr/ports/LEGAL
/usr/ports/MOVED
/usr/ports/Makefile
/usr/ports/Mk/bsd.apache.mk
/usr/ports/Mk/bsd.autotools.mk
/usr/ports/Mk/bsd.cmake.mk
...

The process is now complete, and applications may be installed or upgraded using the updated Ports Collection.

To run both processes consecutively, issue the following command:

# portsnap fetch update

24.4 Tracking a Development Branch

There are two development branches to FreeBSD: FreeBSD-CURRENT and FreeBSD-STABLE. This section will explain a bit about each and describe how to keep your system up-to-date with each respective tree. FreeBSD-CURRENT will be discussed first, then FreeBSD-STABLE.

24.4.1 Staying Current with FreeBSD

As you read this, keep in mind that FreeBSD-CURRENT is the “bleeding edge” of FreeBSD development. FreeBSD-CURRENT users are expected to have a high degree of technical skill, and should be capable of solving difficult system problems on their own. If you are new to FreeBSD, think twice before installing it.

24.4.1.1 What Is FreeBSD-CURRENT?

FreeBSD-CURRENT is the latest working sources for FreeBSD. This includes work in progress, experimental changes, and transitional mechanisms that might or might not be present in the next official release of the software. While many FreeBSD developers compile the FreeBSD-CURRENT source code daily, there are periods of time when the sources are not buildable. These problems are resolved as expeditiously as possible, but whether or not FreeBSD-CURRENT brings disaster or greatly desired functionality can be a matter of which exact moment you grabbed the source code in!

24.4.1.2 Who Needs FreeBSD-CURRENT?

FreeBSD-CURRENT is made available for 3 primary interest groups:

Members of the FreeBSD community who are actively working on some part of the source tree and for whom keeping “current” is an absolute requirement.
Members of the FreeBSD community who are active testers, willing to spend time solving problems in order to ensure that FreeBSD-CURRENT remains as sane as possible. These are also people who wish to make topical suggestions on changes and the general direction of FreeBSD, and submit patches to implement them.
Those who merely wish to keep an eye on things, or to use the current sources for reference purposes (e.g. for reading, not running). These people also make the occasional comment or contribute code.

24.4.1.3 What Is FreeBSD-CURRENT Not?

A fast-track to getting pre-release bits because you heard there is some cool new feature in there and you want to be the first on your block to have it. Being the first on the block to get the new feature means that you are the first on the block to get the new bugs.
A quick way of getting bug fixes. Any given version of FreeBSD-CURRENT is just as likely to introduce new bugs as to fix existing ones.
In any way “officially supported”. We do our best to help people genuinely in one of the 3 “legitimate” FreeBSD-CURRENT groups, but we simply do not have the time to provide tech support. This is not because we are mean and nasty people who do not like helping people out (we would not even be doing FreeBSD if we were). We simply cannot answer hundreds messages a day and work on FreeBSD! Given the choice between improving FreeBSD and answering lots of questions on experimental code, the developers opt for the former.

24.4.1.4 Using FreeBSD-CURRENT

Join the freebsd-current and the svn-src-head lists. This is not just a good idea, it is essential. If you are not on the freebsd-current list, you will not see the comments that people are making about the current state of the system and thus will probably end up stumbling over a lot of problems that others have already found and solved. Even more importantly, you will miss out on important bulletins which may be critical to your system’s continued health.

The svn-src-head list will allow you to see the commit log entry for each change as it is made, along with any pertinent information on possible side-effects.

To join these lists, or one of the others available go to http://lists.FreeBSD.org/mailman/listinfo and click on the list that you wish to subscribe to. Instructions on the rest of the procedure are available there. If you are interested in tracking changes for the whole source tree, we would recommend subscribing to the svn-src-all list.
Grab the sources from a FreeBSD mirror site. You can do this in one of two ways:
1. Use the cvsup program with the supfile named standard-supfile available from /usr/share/examples/cvsup. This is the most recommended method, since it allows you to grab the entire collection once and then only what has changed from then on. Many people run cvsup from cron and keep their sources up-to-date automatically. You have to customize the sample supfile above, and configure cvsup for your environment.
  Note: The sample standard-supfile is intended for tracking a specific security branch of FreeBSD, and not FreeBSD-CURRENT. You will need to edit this file and replace the following line:
```
*default release=cvs tag=RELENG_X_Y
```
  With this one:
```
*default release=cvs tag=.
```
  For a detailed explanation of usable tags, please refer to the Handbook’s CVS Tags section.
2. Use the CTM facility. If you have very bad connectivity (high price connections or only email access) CTM is an option. However, it is a lot of hassle and can give you broken files. This leads to it being rarely used, which again increases the chance of it not working for fairly long periods of time. We recommend using CVSup for anybody with a 9600 bps modem or faster connection.
If you are grabbing the sources to run, and not just look at, then grab all of FreeBSD-CURRENT, not just selected portions. The reason for this is that various parts of the source depend on updates elsewhere, and trying to compile just a subset is almost guaranteed to get you into trouble.

Before compiling FreeBSD-CURRENT, read the Makefile in /usr/src carefully. You should at least install a new kernel and rebuild the world the first time through as part of the upgrading process. Reading the FreeBSD-CURRENT mailing list and /usr/src/UPDATING will keep you up-to-date on other bootstrapping procedures that sometimes become necessary as we move toward the next release.
Be active! If you are running FreeBSD-CURRENT, we want to know what you have to say about it, especially if you have suggestions for enhancements or bug fixes. Suggestions with accompanying code are received most enthusiastically!

24.4.2 Staying Stable with FreeBSD

24.4.2.1 What Is FreeBSD-STABLE?

FreeBSD-STABLE is our development branch from which major releases are made. Changes go into this branch at a different pace, and with the general assumption that they have first gone into FreeBSD-CURRENT for testing. This is still a development branch, however, and this means that at any given time, the sources for FreeBSD-STABLE may or may not be suitable for any particular purpose. It is simply another engineering development track, not a resource for end-users.

24.4.2.2 Who Needs FreeBSD-STABLE?

If you are interested in tracking or contributing to the FreeBSD development process, especially as it relates to the next “point” release of FreeBSD, then you should consider following FreeBSD-STABLE.

While it is true that security fixes also go into the FreeBSD-STABLE branch, you do not need to track FreeBSD-STABLE to do this. Every security advisory for FreeBSD explains how to fix the problem for the releases it affects [13], and tracking an entire development branch just for security reasons is likely to bring in a lot of unwanted changes as well.

Although we endeavor to ensure that the FreeBSD-STABLE branch compiles and runs at all times, this cannot be guaranteed. In addition, while code is developed in FreeBSD-CURRENT before including it in FreeBSD-STABLE, more people run FreeBSD-STABLE than FreeBSD-CURRENT, so it is inevitable that bugs and corner cases will sometimes be found in FreeBSD-STABLE that were not apparent in FreeBSD-CURRENT.

For these reasons, we do not recommend that you blindly track FreeBSD-STABLE, and it is particularly important that you do not update any production servers to FreeBSD-STABLE without first thoroughly testing the code in your development environment.

If you do not have the resources to do this then we recommend that you run the most recent release of FreeBSD, and use the binary update mechanism to move from release to release.

24.4.2.3 Using FreeBSD-STABLE

Join the freebsd-stable list. This will keep you informed of build-dependencies that may appear in FreeBSD-STABLE or any other issues requiring special attention. Developers will also make announcements in this mailing list when they are contemplating some controversial fix or update, giving the users a chance to respond if they have any issues to raise concerning the proposed change.

Join the relevant SVN list for the branch you are tracking. For example, if you are tracking the 7-STABLE branch, join the svn-src-stable-7 list. This will allow you to view the commit log entry for each change as it is made, along with any pertinent information on possible side-effects.

To join these lists, or one of the others available go to http://lists.FreeBSD.org/mailman/listinfo and click on the list that you wish to subscribe to. Instructions on the rest of the procedure are available there. If you are interested in tracking changes for the whole source tree, we would recommend subscribing to the svn-src-all list.
If you are going to install a new system and want it to run monthly snapshot built from FreeBSD-STABLE, please check the Snapshots web page for more information. Alternatively, it is possible to install the most recent FreeBSD-STABLE release from the mirror sites and follow the instructions below to upgrade your system to the most up to date FreeBSD-STABLE source code.

If you are already running a previous release of FreeBSD and wish to upgrade via sources then you can easily do so from FreeBSD mirror site. This can be done in one of two ways:
1. Use the cvsup program with the supfile named stable-supfile from the directory /usr/share/examples/cvsup. This is the most recommended method, since it allows you to grab the entire collection once and then only what has changed from then on. Many people run cvsup from cron to keep their sources up-to-date automatically. You have to customize the sample supfile above, and configure cvsup for your environment.
2. Use the CTM facility. If you do not have a fast and inexpensive connection to the Internet, this is the method you should consider using.
Essentially, if you need rapid on-demand access to the source and communications bandwidth is not a consideration, use cvsup or ftp. Otherwise, use CTM.
Before compiling FreeBSD-STABLE, read the Makefile in /usr/src carefully. You should at least install a new kernel and rebuild the world the first time through as part of the upgrading process. Reading the FreeBSD-STABLE mailing list and /usr/src/UPDATING will keep you up-to-date on other bootstrapping procedures that sometimes become necessary as we move toward the next release.

24.5 Synchronizing Your Source

There are various ways of using an Internet (or email) connection to stay up-to-date with any given area of the FreeBSD project sources, or all areas, depending on what interests you. The primary services we offer are Anonymous CVS, CVSup, and CTM.

Warning: While it is possible to update only parts of your source tree, the only supported update procedure is to update the entire tree and recompile both userland (i.e., all the programs that run in user space, such as those in /bin and /sbin) and kernel sources. Updating only part of your source tree, only the kernel, or only userland will often result in problems. These problems may range from compile errors to kernel panics or data corruption.

Anonymous CVS and CVSup use the pull model of updating sources. In the case of CVSup the user (or a cron script) invokes the cvsup program, and it interacts with a cvsupd server somewhere to bring your files up-to-date. The updates you receive are up-to-the-minute and you get them when, and only when, you want them. You can easily restrict your updates to the specific files or directories that are of interest to you. Updates are generated on the fly by the server, according to what you have and what you want to have. Anonymous CVS is quite a bit more simplistic than CVSup in that it is just an extension to CVS which allows it to pull changes directly from a remote CVS repository. CVSup can do this far more efficiently, but Anonymous CVS is easier to use.

CTM, on the other hand, does not interactively compare the sources you have with those on the master archive or otherwise pull them across. Instead, a script which identifies changes in files since its previous run is executed several times a day on the master CTM machine, any detected changes being compressed, stamped with a sequence-number and encoded for transmission over email (in printable ASCII only). Once received, these “CTM deltas” can then be handed to the ctm_rmail(1) utility which will automatically decode, verify and apply the changes to the user’s copy of the sources. This process is far more efficient than CVSup, and places less strain on our server resources since it is a push rather than a pull model.

There are other trade-offs, of course. If you inadvertently wipe out portions of your archive, CVSup will detect and rebuild the damaged portions for you. CTM will not do this, and if you wipe some portion of your source tree out (and do not have it backed up) then you will have to start from scratch (from the most recent CVS “base delta”) and rebuild it all with CTM or, with Anonymous CVS, simply delete the bad bits and resync.

24.6 Rebuilding “world”

Once you have synchronized your local source tree against a particular version of FreeBSD (FreeBSD-STABLE, FreeBSD-CURRENT, and so on) you can then use the source tree to rebuild the system.

Make a Backup: It cannot be stressed enough how important it is to make a backup of your system before you do this. While rebuilding the world is (as long as you follow these instructions) an easy task to do, there will inevitably be times when you make mistakes, or when mistakes made by others in the source tree render your system unbootable.

Make sure you have taken a backup. And have a fixit floppy or bootable CD at hand. You will probably never have to use it, but it is better to be safe than sorry!

Subscribe to the Right Mailing List: The FreeBSD-STABLE and FreeBSD-CURRENT branches are, by their nature, in development. People that contribute to FreeBSD are human, and mistakes occasionally happen.

Sometimes these mistakes can be quite harmless, just causing your system to print a new diagnostic warning. Or the change may be catastrophic, and render your system unbootable or destroy your file systems (or worse).

If problems like these occur, a “heads up” is posted to the appropriate mailing list, explaining the nature of the problem and which systems it affects. And an “all clear” announcement is posted when the problem has been solved.

If you try to track FreeBSD-STABLE or FreeBSD-CURRENT and do not read the FreeBSD-STABLE mailing list or the FreeBSD-CURRENT mailing list respectively, then you are asking for trouble.

Do not use make world: A lot of older documentation recommends using make world for this. Doing that skips some important steps and should only be used if you are sure of what you are doing. For almost all circumstances make world is the wrong thing to do, and the procedure described here should be used instead.

24.6.1 The Canonical Way to Update Your System

To update your system, you should check /usr/src/UPDATING for any pre-buildworld steps necessary for your version of the sources and then use the following procedure:

# cd /usr/src
# make buildworld
# make buildkernel
# make installkernel
# shutdown -r now

Note: There are a few rare cases when an extra run of mergemaster -p is needed before the buildworld step. These are described in UPDATING. In general, though, you can safely omit this step if you are not updating across one or more major FreeBSD versions.

After installkernel finishes successfully, you should boot in single user mode (i.e. using boot -s from the loader prompt). Then run:

# mount -a -t ufs
# mergemaster -p
# cd /usr/src
# make installworld
# mergemaster
# reboot

Read Further Explanations: The sequence described above is only a short resume to help you getting started. You should however read the following sections to clearly understand each step, especially if you want to use a custom kernel configuration.

24.6.2 Read `/usr/src/UPDATING`

Before you do anything else, read /usr/src/UPDATING (or the equivalent file wherever you have a copy of the source code). This file should contain important information about problems you might encounter, or specify the order in which you might have to run certain commands. If UPDATING contradicts something you read here, UPDATING takes precedence.

Important: Reading UPDATING is not an acceptable substitute for subscribing to the correct mailing list, as described previously. The two requirements are complementary, not exclusive.

24.6.3 Check `/etc/make.conf`

Examine the files /usr/share/examples/etc/make.conf and /etc/make.conf. The first contains some default defines - most of which are commented out. To make use of them when you rebuild your system from source, add them to /etc/make.conf. Keep in mind that anything you add to /etc/make.conf is also used every time you run make, so it is a good idea to set them to something sensible for your system.

A typical user will probably want to copy the CFLAGS and NO_PROFILE lines found in /usr/share/examples/etc/make.conf to /etc/make.conf and uncomment them.

Examine the other definitions (COPTFLAGS, NOPORTDOCS and so on) and decide if they are relevant to you.

24.6.4 Update the Files in `/etc`

The /etc directory contains a large part of your system’s configuration information, as well as scripts that are run at system startup. Some of these scripts change from version to version of FreeBSD.

Some of the configuration files are also used in the day to day running of the system. In particular, /etc/group.

There have been occasions when the installation part of make installworld has expected certain usernames or groups to exist. When performing an upgrade it is likely that these users or groups did not exist. This caused problems when upgrading. In some cases make buildworld will check to see if these users or groups exist.

An example of this is when the smmsp user was added. Users had the installation process fail for them when mtree(8) was trying to create /var/spool/clientmqueue.

The solution is to run mergemaster(8) in pre-buildworld mode by providing the -p option. This will compare only those files that are essential for the success of buildworld or installworld. If your old version of mergemaster does not support -p, use the new version in the source tree when running for the first time:

# cd /usr/src/usr.sbin/mergemaster
# ./mergemaster.sh -p

Tip: If you are feeling particularly paranoid, you can check your system to see which files are owned by the group you are renaming or deleting:
# find / -group GID -print
will show all files owned by group GID (which can be either a group name or a numeric group ID).

24.6.5 Drop to Single User Mode

You may want to compile the system in single user mode. Apart from the obvious benefit of making things go slightly faster, reinstalling the system will touch a lot of important system files, all the standard system binaries, libraries, include files and so on. Changing these on a running system (particularly if you have active users on the system at the time) is asking for trouble.

Another method is to compile the system in multi-user mode, and then drop into single user mode for the installation. If you would like to do it this way, simply hold off on the following steps until the build has completed. You can postpone dropping to single user mode until you have to installkernel or installworld.

As the superuser, you can execute:

# shutdown now

from a running system, which will drop it to single user mode.

Alternatively, reboot the system, and at the boot prompt, select the “single user” option. The system will then boot single user. At the shell prompt you should then run:

# fsck -p
# mount -u /
# mount -a -t ufs
# swapon -a

This checks the file systems, remounts / read/write, mounts all the other UFS file systems referenced in /etc/fstab and then turns swapping on.

Note: If your CMOS clock is set to local time and not to GMT (this is true if the output of the date(1) command does not show the correct time and zone), you may also need to run the following command:
# adjkerntz -i
This will make sure that your local time-zone settings get set up correctly — without this, you may later run into some problems.

24.6.6 Remove `/usr/obj`

As parts of the system are rebuilt they are placed in directories which (by default) go under /usr/obj. The directories shadow those under /usr/src.

You can speed up the make buildworld process, and possibly save yourself some dependency headaches by removing this directory as well.

Some files below /usr/obj may have the immutable flag set (see chflags(1) for more information) which must be removed first.

# cd /usr/obj
# chflags -R noschg *
# rm -rf *

24.6.7 Recompile the Base System

24.6.7.1 Saving the Output

It is a good idea to save the output you get from running make(1) to another file. If something goes wrong you will have a copy of the error message. While this might not help you in diagnosing what has gone wrong, it can help others if you post your problem to one of the FreeBSD mailing lists.

The easiest way to do this is to use the script(1) command, with a parameter that specifies the name of the file to save all output to. You would do this immediately before rebuilding the world, and then type exit when the process has finished.

# script /var/tmp/mw.out
Script started, output file is /var/tmp/mw.out
# make TARGET
... compile, compile, compile ...
# exit
Script done, ...

If you do this, do not save the output in /tmp. This directory may be cleared next time you reboot. A better place to store it is in /var/tmp (as in the previous example) or in root’s home directory.

24.6.7.2 Compile the Base System

You must be in the /usr/src directory:

# cd /usr/src

(unless, of course, your source code is elsewhere, in which case change to that directory instead).

To rebuild the world you use the make(1) command. This command reads instructions from the Makefile, which describes how the programs that comprise FreeBSD should be rebuilt, the order in which they should be built, and so on.

The general format of the command line you will type is as follows:

# make -x -DVARIABLE target

In this example, -x is an option that you would pass to make(1). See the make(1) manual page for an example of the options you can pass.

-DVARIABLE passes a variable to the Makefile. The behavior of the Makefile is controlled by these variables. These are the same variables as are set in /etc/make.conf, and this provides another way of setting them.

# make -DNO_PROFILE target

is another way of specifying that profiled libraries should not be built, and corresponds with the

NO_PROFILE=    true     #    Avoid compiling profiled libraries

line in /etc/make.conf.

target tells make(1) what you want to do. Each Makefile defines a number of different “targets”, and your choice of target determines what happens.

Some targets are listed in the Makefile, but are not meant for you to run. Instead, they are used by the build process to break out the steps necessary to rebuild the system into a number of sub-steps.

Most of the time you will not need to pass any parameters to make(1), and so your command like will look like this:

# make target

Where target will be one of many build options. The first target should always be buildworld.

As the names imply, buildworld builds a complete new tree under /usr/obj, and installworld, another target, installs this tree on the current machine.

Having separate options is very useful for two reasons. First, it allows you to do the build safe in the knowledge that no components of your running system will be affected. The build is “self hosted”. Because of this, you can safely run buildworld on a machine running in multi-user mode with no fear of ill-effects. It is still recommended that you run the installworld part in single user mode, though.

Secondly, it allows you to use NFS mounts to upgrade multiple machines on your network. If you have three machines, A, B and C that you want to upgrade, run make buildworld and make installworld on A. B and C should then NFS mount /usr/src and /usr/obj from A, and you can then run make installworld to install the results of the build on B and C.

Although the world target still exists, you are strongly encouraged not to use it.

Run

# make buildworld

It is possible to specify a -j option to make which will cause it to spawn several simultaneous processes. This is most useful on multi-CPU machines. However, since much of the compiling process is IO bound rather than CPU bound it is also useful on single CPU machines.

On a typical single-CPU machine you would run:

# make -j4 buildworld

make(1) will then have up to 4 processes running at any one time. Empirical evidence posted to the mailing lists shows this generally gives the best performance benefit.

If you have a multi-CPU machine and you are using an SMP configured kernel try values between 6 and 10 and see how they speed things up.

24.6.7.3 Timings

Many factors influence the build time, but fairly recent machines may only take a one or two hours to build the FreeBSD-STABLE tree, with no tricks or shortcuts used during the process. A FreeBSD-CURRENT tree will take somewhat longer.

24.6.8 Compile and Install a New Kernel

To take full advantage of your new system you should recompile the kernel. This is practically a necessity, as certain memory structures may have changed, and programs like ps(1) and top(1) will fail to work until the kernel and source code versions are the same.

The simplest, safest way to do this is to build and install a kernel based on GENERIC. While GENERIC may not have all the necessary devices for your system, it should contain everything necessary to boot your system back to single user mode. This is a good test that the new system works properly. After booting from GENERIC and verifying that your system works you can then build a new kernel based on your normal kernel configuration file.

On FreeBSD it is important to build world before building a new kernel.

Note: If you want to build a custom kernel, and already have a configuration file, just use KERNCONF=MYKERNEL like this:
# cd /usr/src
# make buildkernel KERNCONF=MYKERNEL
# make installkernel KERNCONF=MYKERNEL

Note that if you have raised kern.securelevel above 1 and you have set either the noschg or similar flags to your kernel binary, you might find it necessary to drop into single user mode to use installkernel. Otherwise you should be able to run both these commands from multi user mode without problems. See init(8) for details about kern.securelevel and chflags(1) for details about the various file flags.

24.6.9 Reboot into Single User Mode

You should reboot into single user mode to test the new kernel works. Do this by following the instructions in Section 24.6.5.

24.6.10 Install the New System Binaries

If you were building a version of FreeBSD recent enough to have used make buildworld then you should now use installworld to install the new system binaries.

Run

# cd /usr/src
# make installworld

Note: If you specified variables on the make buildworld command line, you must specify the same variables in the make installworld command line. This does not necessarily hold true for other options; for example, -j must never be used with installworld.

For example, if you ran:
# make -DNO_PROFILE buildworld
you must install the results with:
# make -DNO_PROFILE installworld
otherwise it would try to install profiled libraries that had not been built during the make buildworld phase.

24.6.11 Update Files Not Updated by `make installworld`

Remaking the world will not update certain directories (in particular, /etc, /var and /usr) with new or changed configuration files.

The simplest way to update these files is to use mergemaster(8), though it is possible to do it manually if you would prefer to do that. Regardless of which way you choose, be sure to make a backup of /etc in case anything goes wrong.

24.6.11.1 `mergemaster`

Contributed by Tom Rhodes.

The mergemaster(8) utility is a Bourne script that will aid you in determining the differences between your configuration files in /etc, and the configuration files in the source tree /usr/src/etc. This is the recommended solution for keeping the system configuration files up to date with those located in the source tree.

To begin simply type mergemaster at your prompt, and watch it start going. mergemaster will then build a temporary root environment, from / down, and populate it with various system configuration files. Those files are then compared to the ones currently installed in your system. At this point, files that differ will be shown in diff(1) format, with the + sign representing added or modified lines, and - representing lines that will be either removed completely, or replaced with a new line. See the diff(1) manual page for more information about the diff(1) syntax and how file differences are shown.

mergemaster(8) will then show you each file that displays variances, and at this point you will have the option of either deleting the new file (referred to as the temporary file), installing the temporary file in its unmodified state, merging the temporary file with the currently installed file, or viewing the diff(1) results again.

Choosing to delete the temporary file will tell mergemaster(8) that we wish to keep our current file unchanged, and to delete the new version. This option is not recommended, unless you see no reason to change the current file. You can get help at any time by typing ? at the mergemaster(8) prompt. If the user chooses to skip a file, it will be presented again after all other files have been dealt with.

Choosing to install the unmodified temporary file will replace the current file with the new one. For most unmodified files, this is the best option.

Choosing to merge the file will present you with a text editor, and the contents of both files. You can now merge them by reviewing both files side by side on the screen, and choosing parts from both to create a finished product. When the files are compared side by side, the l key will select the left contents and the r key will select contents from your right. The final output will be a file consisting of both parts, which can then be installed. This option is customarily used for files where settings have been modified by the user.

Choosing to view the diff(1) results again will show you the file differences just like mergemaster(8) did before prompting you for an option.

After mergemaster(8) is done with the system files you will be prompted for other options. mergemaster(8) may ask if you want to rebuild the password file and will finish up with an option to remove left-over temporary files.

24.6.11.2 Manual Update

If you wish to do the update manually, however, you cannot just copy over the files from /usr/src/etc to /etc and have it work. Some of these files must be “installed” first. This is because the /usr/src/etc directory is not a copy of what your /etc directory should look like. In addition, there are files that should be in /etc that are not in /usr/src/etc.

If you are using mergemaster(8) (as recommended), you can skip forward to the next section.

The simplest way to do this by hand is to install the files into a new directory, and then work through them looking for differences.

Backup Your Existing /etc: Although, in theory, nothing is going to touch this directory automatically, it is always better to be sure. So copy your existing /etc directory somewhere safe. Something like:
# cp -Rp /etc /etc.old
-R does a recursive copy, -p preserves times, ownerships on files and suchlike.

You need to build a dummy set of directories to install the new /etc and other files into. /var/tmp/root is a reasonable choice, and there are a number of subdirectories required under this as well.

# mkdir /var/tmp/root
# cd /usr/src/etc
# make DESTDIR=/var/tmp/root distrib-dirs distribution

This will build the necessary directory structure and install the files. A lot of the subdirectories that have been created under /var/tmp/root are empty and should be deleted. The simplest way to do this is to:

# cd /var/tmp/root
# find -d . -type d | xargs rmdir 2>/dev/null

This will remove all empty directories. (Standard error is redirected to /dev/null to prevent the warnings about the directories that are not empty.)

/var/tmp/root now contains all the files that should be placed in appropriate locations below /. You now have to go through each of these files, determining how they differ with your existing files.

Note that some of the files that will have been installed in /var/tmp/root have a leading “.”. At the time of writing the only files like this are shell startup files in /var/tmp/root/ and /var/tmp/root/root/, although there may be others (depending on when you are reading this). Make sure you use ls -a to catch them.

The simplest way to do this is to use diff(1) to compare the two files:

# diff /etc/shells /var/tmp/root/etc/shells

This will show you the differences between your /etc/shells file and the new /var/tmp/root/etc/shells file. Use these to decide whether to merge in changes that you have made or whether to copy over your old file.

Name the New Root Directory (/var/tmp/root) with a Time Stamp, so You Can Easily Compare Differences Between Versions: Frequently rebuilding the world means that you have to update /etc frequently as well, which can be a bit of a chore.

You can speed this process up by keeping a copy of the last set of changed files that you merged into /etc. The following procedure gives one idea of how to do this.
Make the world as normal. When you want to update /etc and the other directories, give the target directory a name based on the current date. If you were doing this on the 14th of February 1998 you could do the following:
# mkdir /var/tmp/root-19980214
# cd /usr/src/etc
# make DESTDIR=/var/tmp/root-19980214 \
    distrib-dirs distribution
Merge in the changes from this directory as outlined above.

Do not remove the /var/tmp/root-19980214 directory when you have finished.

When you have downloaded the latest version of the source and remade it, follow step 1. This will give you a new directory, which might be called /var/tmp/root-19980221 (if you wait a week between doing updates).
You can now see the differences that have been made in the intervening week using diff(1) to create a recursive diff between the two directories:
# cd /var/tmp
# diff -r root-19980214 root-19980221
Typically, this will be a much smaller set of differences than those between /var/tmp/root-19980221/etc and /etc. Because the set of differences is smaller, it is easier to migrate those changes across into your /etc directory.
You can now remove the older of the two /var/tmp/root-* directories:
# rm -rf /var/tmp/root-19980214
Repeat this process every time you need to merge in changes to /etc.
You can use date(1) to automate the generation of the directory names:
# mkdir /var/tmp/root-`date "+%Y%m%d"`

24.6.12 Rebooting

You are now done. After you have verified that everything appears to be in the right place you can reboot the system. A simple shutdown(8) should do it:

# shutdown -r now

24.6.13 Finished

You should now have successfully upgraded your FreeBSD system. Congratulations.

If things went slightly wrong, it is easy to rebuild a particular piece of the system. For example, if you accidentally deleted /etc/magic as part of the upgrade or merge of /etc, the file(1) command will stop working. In this case, the fix would be to run:

# cd /usr/src/usr.bin/file
# make all install

24.6.14 Questions

24.6.14.1. Do I need to re-make the world for every change?
24.6.14.2. My compile failed with lots of signal 11 (or other signal number) errors. What has happened?
24.6.14.3. Can I remove /usr/obj when I have finished?
24.6.14.4. Can interrupted builds be resumed?
24.6.14.5. How can I speed up making the world?
24.6.14.6. What do I do if something goes wrong?

24.6.14.1. Do I need to re-make the world for every change?

There is no easy answer to this one, as it depends on the nature of the change. For example, if you just ran CVSup, and it has shown the following files as being updated:

src/games/cribbage/instr.c
src/games/sail/pl_main.c
src/release/sysinstall/config.c
src/release/sysinstall/media.c
src/share/mk/bsd.port.mk

it probably is not worth rebuilding the entire world. You could just go to the appropriate sub-directories and make all install, and that’s about it. But if something major changed, for example src/lib/libc/stdlib then you should either re-make the world, or at least those parts of it that are statically linked (as well as anything else you might have added that is statically linked).

At the end of the day, it is your call. You might be happy re-making the world every fortnight say, and let changes accumulate over that fortnight. Or you might want to re-make just those things that have changed, and be confident you can spot all the dependencies.

And, of course, this all depends on how often you want to upgrade, and whether you are tracking FreeBSD-STABLE or FreeBSD-CURRENT.

24.6.14.2. My compile failed with lots of signal 11 (or other signal number) errors. What has happened?

This is normally indicative of hardware problems. (Re)making the world is an effective way to stress test your hardware, and will frequently throw up memory problems. These normally manifest themselves as the compiler mysteriously dying on receipt of strange signals.

A sure indicator of this is if you can restart the make and it dies at a different point in the process.

In this instance there is little you can do except start swapping around the components in your machine to determine which one is failing.

24.6.14.3. Can I remove /usr/obj when I have finished?

The short answer is yes.

/usr/obj contains all the object files that were produced during the compilation phase. Normally, one of the first steps in the make buildworld process is to remove this directory and start afresh. In this case, keeping /usr/obj around after you have finished makes little sense, and will free up a large chunk of disk space (currently about 340 MB).

However, if you know what you are doing you can have make buildworld skip this step. This will make subsequent builds run much faster, since most of sources will not need to be recompiled. The flip side of this is that subtle dependency problems can creep in, causing your build to fail in odd ways. This frequently generates noise on the FreeBSD mailing lists, when one person complains that their build has failed, not realizing that it is because they have tried to cut corners.

24.6.14.4. Can interrupted builds be resumed?

This depends on how far through the process you got before you found a problem.

In general (and this is not a hard and fast rule) the make buildworld process builds new copies of essential tools (such as gcc(1), and make(1)) and the system libraries. These tools and libraries are then installed. The new tools and libraries are then used to rebuild themselves, and are installed again. The entire system (now including regular user programs, such as ls(1) or grep(1)) is then rebuilt with the new system files.

If you are at the last stage, and you know it (because you have looked through the output that you were storing) then you can (fairly safely) do:

... fix the problem ...
# cd /usr/src
# make -DNO_CLEAN all

This will not undo the work of the previous make buildworld.

If you see the message:

--------------------------------------------------------------
Building everything..
--------------------------------------------------------------

in the make buildworld output then it is probably fairly safe to do so.

If you do not see that message, or you are not sure, then it is always better to be safe than sorry, and restart the build from scratch.

24.6.14.5. How can I speed up making the world?

Run in single user mode.
Put the /usr/src and /usr/obj directories on separate file systems held on separate disks. If possible, put these disks on separate disk controllers.
Better still, put these file systems across multiple disks using the ccd(4) (concatenated disk driver) device.
Turn off profiling (set “NO_PROFILE=true” in /etc/make.conf). You almost certainly do not need it.
Also in /etc/make.conf, set CFLAGS to something like -O -pipe. The optimization -O2 is much slower, and the optimization difference between -O and -O2 is normally negligible. -pipe lets the compiler use pipes rather than temporary files for communication, which saves disk access (at the expense of memory).
Pass the -jn option to make(1) to run multiple processes in parallel. This usually helps regardless of whether you have a single or a multi processor machine.
The file system holding /usr/src can be mounted (or remounted) with the noatime option. This prevents the file system from recording the file access time. You probably do not need this information anyway.
```
# mount -u -o noatime /usr/src
```
Warning: The example assumes /usr/src is on its own file system. If it is not (if it is a part of /usr for example) then you will need to use that file system mount point, and not /usr/src.
The file system holding /usr/obj can be mounted (or remounted) with the async option. This causes disk writes to happen asynchronously. In other words, the write completes immediately, and the data is written to the disk a few seconds later. This allows writes to be clustered together, and can be a dramatic performance boost.

Warning: Keep in mind that this option makes your file system more fragile. With this option there is an increased chance that, should power fail, the file system will be in an unrecoverable state when the machine restarts.

If /usr/obj is the only thing on this file system then it is not a problem. If you have other, valuable data on the same file system then ensure your backups are fresh before you enable this option.
```
# mount -u -o async /usr/obj
```
Warning: As above, if /usr/obj is not on its own file system, replace it in the example with the name of the appropriate mount point.

24.6.14.6. What do I do if something goes wrong?

Make absolutely sure your environment has no extraneous cruft from earlier builds. This is simple enough.

# chflags -R noschg /usr/obj/usr
# rm -rf /usr/obj/usr
# cd /usr/src
# make cleandir
# make cleandir

Yes, make cleandir really should be run twice.

Then restart the whole process, starting with make buildworld.

If you still have problems, send the error and the output of uname -a to FreeBSD general questions mailing list. Be prepared to answer other questions about your setup!

24.7 Tracking for Multiple Machines

Contributed by Mike Meyer.

If you have multiple machines that you want to track the same source tree, then having all of them download sources and rebuild everything seems like a waste of resources: disk space, network bandwidth, and CPU cycles. It is, and the solution is to have one machine do most of the work, while the rest of the machines mount that work via NFS. This section outlines a method of doing so.

24.7.1 Preliminaries

First, identify a set of machines that is going to run the same set of binaries, which we will call a build set. Each machine can have a custom kernel, but they will be running the same userland binaries. From that set, choose a machine to be the build machine. It is going to be the machine that the world and kernel are built on. Ideally, it should be a fast machine that has sufficient spare CPU to run make buildworld and make buildkernel. You will also want to choose a machine to be the test machine, which will test software updates before they are put into production. This must be a machine that you can afford to have down for an extended period of time. It can be the build machine, but need not be.

All the machines in this build set need to mount /usr/obj and /usr/src from the same machine, and at the same point. Ideally, those are on two different drives on the build machine, but they can be NFS mounted on that machine as well. If you have multiple build sets, /usr/src should be on one build machine, and NFS mounted on the rest.

Finally make sure that /etc/make.conf and /etc/src.conf on all the machines in the build set agrees with the build machine. That means that the build machine must build all the parts of the base system that any machine in the build set is going to install. Also, each build machine should have its kernel name set with KERNCONF in /etc/make.conf, and the build machine should list them all in KERNCONF, listing its own kernel first. The build machine must have the kernel configuration files for each machine in /usr/src/sys/arch/conf if it is going to build their kernels.

24.7.2 The Base System

Now that all that is done, you are ready to build everything. Build the kernel and world as described in Section 24.6.7.2 on the build machine, but do not install anything. After the build has finished, go to the test machine, and install the kernel you just built. If this machine mounts /usr/src and /usr/obj via NFS, when you reboot to single user you will need to enable the network and mount them. The easiest way to do this is to boot to multi-user, then run shutdown now to go to single user mode. Once there, you can install the new kernel and world and run mergemaster just as you normally would. When done, reboot to return to normal multi-user operations for this machine.

After you are certain that everything on the test machine is working properly, use the same procedure to install the new software on each of the other machines in the build set.

24.7.3 Ports

The same ideas can be used for the ports tree. The first critical step is mounting /usr/ports from the same machine to all the machines in the build set. You can then set up /etc/make.conf properly to share distfiles. You should set DISTDIR to a common shared directory that is writable by whichever user root is mapped to by your NFS mounts. Each machine should set WRKDIRPREFIX to a local build directory. Finally, if you are going to be building and distributing packages, you should set PACKAGES to a directory similar to DISTDIR.

Tags: Apache, archive, backup, bsd, catch, cron, email, freebsd, FreeBSD Handbook, ftp, password, pop, software, ssl

FreeBSD Handbook Apache, archive, backup, bsd, catch, cron, email, freebsd, ftp, password, pop, software, ssl

FreeBSD Handbook - Chapter 19 GEOM Modular Disk Transformation Framework

January 6th, 2009

No comments

Written by Tom Rhodes.

19.1 Synopsis

This chapter covers the use of disks under the GEOM framework in FreeBSD. This includes the major RAID control utilities which use the framework for configuration. This chapter will not go into in depth discussion on how GEOM handles or controls I/O, the underlying subsystem, or code. This information is provided through the geom(4) manual page and its various SEE ALSO references. This chapter is also not a definitive guide to RAID configurations. Only GEOM-supported RAID classifications will be discussed.

After reading this chapter, you will know:

What type of RAID support is available through GEOM.
How to use the base utilities to configure, maintain, and manipulate the various RAID levels.
How to mirror, stripe, encrypt, and remotely connect disk devices through GEOM.
How to troubleshoot disks attached to the GEOM framework.

Before reading this chapter, you should:

Understand how FreeBSD treats disk devices (Chapter 18).
Know how to configure and install a new FreeBSD kernel (Chapter 8).

19.2 GEOM Introduction

GEOM permits access and control to classes — Master Boot Records, BSD labels, etc — through the use of providers, or the special files in /dev. Supporting various software RAID configurations, GEOM will transparently provide access to the operating system and operating system utilities.

19.3 RAID0 - Striping

Written by Tom Rhodes and Murray Stokely.

Striping is a method used to combine several disk drives into a single volume. In many cases, this is done through the use of hardware controllers. The GEOM disk subsystem provides software support for RAID0, also known as disk striping.

In a RAID0 system, data are split up in blocks that get written across all the drives in the array. Instead of having to wait on the system to write 256k to one disk, a RAID0 system can simultaneously write 64k to each of four different disks, offering superior I/O performance. This performance can be enhanced further by using multiple disk controllers.

Each disk in a RAID0 stripe must be of the same size, since I/O requests are interleaved to read or write to multiple disks in parallel.

Creating a stripe of unformatted ATA disks

Load the geom_stripe.ko module:
```
# kldload geom_stripe
```
Ensure that a suitable mount point exists. If this volume will become a root partition, then temporarily use another mount point such as /mnt:
```
# mkdir /mnt
```
Determine the device names for the disks which will be striped, and create the new stripe device. For example, to stripe two unused and unpartitioned ATA disks, for example /dev/ad2 and /dev/ad3:
```
# gstripe label -v st0 /dev/ad2 /dev/ad3
Metadata value stored on /dev/ad2.
Metadata value stored on /dev/ad3.
Done.
```
Write a standard label, also known as a partition table, on the new volume and install the default bootstrap code:
```
# bsdlabel -wB /dev/stripe/st0
```
This process should have created two other devices in the /dev/stripe directory in addition to the st0 device. Those include st0a and st0c. At this point a file system may be created on the st0a device with the newfs utility:
```
# newfs -U /dev/stripe/st0a
```
Many numbers will glide across the screen, and after a few seconds, the process will be complete. The volume has been created and is ready to be mounted.

To manually mount the created disk stripe:

# mount /dev/stripe/st0a /mnt

To mount this striped file system automatically during the boot process, place the volume information in /etc/fstab file. For this purpose, a permanent mount point, named stripe, is created:

# mkdir /stripe
# echo "/dev/stripe/st0a /stripe ufs rw 2 2" \
    >> /etc/fstab

The geom_stripe.ko module must also be automatically loaded during system initialization, by adding a line to /boot/loader.conf:

# echo 'geom_stripe_load="YES"' >> /boot/loader.conf

19.4 RAID1 - Mirroring

Mirroring is a technology used by many corporations and home users to back up data without interruption. When a mirror exists, it simply means that diskB replicates diskA. Or, perhaps diskC+D replicates diskA+B. Regardless of the disk configuration, the important aspect is that information on one disk or partition is being replicated. Later, that information could be more easily restored, backed up without causing service or access interruption, and even be physically stored in a data safe.

To begin, ensure the system has two disk drives of equal size, these exercises assume they are direct access (da(4)) SCSI disks.

19.4.1 Mirroring Primary Disks

Assuming FreeBSD has been installed on the first, da0 disk device, gmirror(8) should be told to store its primary data there.

Before building the mirror, enable additional debugging information and opening access to the device by setting the kern.geom.debugflags sysctl(8) option to the following value:

# sysctl kern.geom.debugflags=17

Now create the mirror. Begin the process by storing meta-data information on the primary disk device, effectively creating the /dev/mirror/gm device using the following command:

Warning: Creating a mirror out of the boot drive may result in data loss if any data has been stored on the last sector of the disk. This risk is reduced if creating the mirror is done promptly after a fresh install of FreeBSD.

# gmirror label -vb round-robin gm0 /dev/da0

The system should respond with:

Metadata value stored on /dev/da0.
Done.

Initialize GEOM, this will load the /boot/kernel/geom_mirror.ko kernel module:

# gmirror load

Note: When this command completes successfully, it creates the gm0 device node under the /dev/mirror directory.

Enable loading of the geom_mirror.ko kernel module during system initialization:

# echo 'geom_mirror_load="YES"' >> /boot/loader.conf

Edit the /etc/fstab file, replacing references to the old da0 with the new device nodes of the gm0 mirror device.

Note: If vi(1) is your preferred editor, the following is an easy way to accomplish this task:
# vi /etc/fstab
In vi(1) back up the current contents of fstab by typing :w /etc/fstab.bak. Then replace all old da0 references with gm0 by typing :%s/da/mirror\/gm/g.

The resulting fstab file should look similar to the following. It does not matter if the disk drives are SCSI or ATA, the RAID device will be gm regardless.

# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/mirror/gm0s1b                       none            swap    sw              0       0
/dev/mirror/gm0s1a                      /               ufs     rw              1       1
/dev/mirror/gm0s1d              /usr        ufs rw      0   0
/dev/mirror/gm0s1f              /home       ufs rw      2   2
#/dev/mirror/gm0s2d                     /store          ufs     rw              2       2
/dev/mirror/gm0s1e                      /var            ufs     rw              2       2
/dev/acd0               /cdrom          cd9660  ro,noauto       0       0

Reboot the system:

# shutdown -r now

During system initialization, the gm0 should be used in place of the da0 device. Once fully initialized, this may be checked by visually inspecting the output from the mount command:

# mount
Filesystem         1K-blocks    Used    Avail Capacity  Mounted on
/dev/mirror/gm0s1a   1012974  224604   707334    24%    /
devfs                      1       1        0   100%    /dev
/dev/mirror/gm0s1f  45970182   28596 42263972     0%    /home
/dev/mirror/gm0s1d   6090094 1348356  4254532    24%    /usr
/dev/mirror/gm0s1e   3045006 2241420   559986    80%    /var
devfs                      1       1        0   100%    /var/named/dev

The output looks good, as expected. Finally, to begin synchronization, insert the da1 disk into the mirror using the following command:

# gmirror insert gm0 /dev/da1

As the mirror is built the status may be checked using the following command:

# gmirror status

Once the mirror has been built and all current data has been synchronized, the output from the above command should look like:

      Name    Status  Components
mirror/gm0  COMPLETE  da0
                      da1

If there are any issues, or the mirror is still completing the build process, the example will show DEGRADED in place of COMPLETE.

19.4.2 Troubleshooting

19.4.2.1 System refuses to boot

If the system boots up to a prompt similar to:

ffs_mountroot: can't find rootvp
Root mount failed: 6
mountroot>

Reboot the machine using the power or reset button. At the boot menu, select option six (6). This will drop the system to a loader(8) prompt. Load the kernel module manually:

OK? load geom_mirror
OK? boot

If this works then for whatever reason the module was not being loaded properly. Check whether the relevant entry in /boot/loader.conf is correct. If the problem persists, place:

options    GEOM_MIRROR

in the kernel configuration file, rebuild and reinstall. That should remedy this issue.

19.4.3 Recovering From Disk Failure

The wonderful part about disk mirroring is that when a disk fails, it may be replaced, presumably, without losing any data.

Considering the previous RAID1 configuration, assume that da1 has failed and now needs to be replaced. To replace it, determine which disk has failed and power down the system. At this point, the disk may be swapped with a new one and the system brought back up. After the system has restarted, the following commands may be used to replace the disk:

# gmirror forget gm0

# gmirror insert gm0 /dev/da1

Use the gmirror status command to monitor the progress of the rebuild. It is that simple.

19.5 GEOM Gate Network Devices

GEOM supports the remote use of devices, such as disks, CD-ROMs, files, etc. through the use of the gate utilities. This is similar to NFS.

To begin, an exports file must be created. This file specifies who is permitted to access the exported resources and what level of access they are offered. For example, to export the fourth slice on the first SCSI disk, the following /etc/gg.exports is more than adequate:

192.168.1.0/24 RW /dev/da0s4d

It will allow all hosts inside the private network access the file system on the da0s4d partition.

To export this device, ensure it is not currently mounted, and start the ggated(8) server daemon:

# ggated

Now to mount the device on the client machine, issue the following commands:

# ggatec create -o rw 192.168.1.1 /dev/da0s4d
ggate0
# mount /dev/ggate0 /mnt

From here on, the device may be accessed through the /mnt mount point.

Note: It should be pointed out that this will fail if the device is currently mounted on either the server machine or any other machine on the network.

When the device is no longer needed, it may be safely unmounted with the umount(8) command, similar to any other disk device.

19.6 Labeling Disk Devices

During system initialization, the FreeBSD kernel will create device nodes as devices are found. This method of probing for devices raises some issues, for instance what if a new disk device is added via USB? It is very likely that a flash device may be handed the device name of da0 and the original da0 shifted to da1. This will cause issues mounting file systems if they are listed in /etc/fstab, effectively, this may also prevent the system from booting.

One solution to this issue is to chain the SCSI devices in order so a new device added to the SCSI card will be issued unused device numbers. But what about USB devices which may replace the primary SCSI disk? This happens because USB devices are usually probed before the SCSI card. One solution is to only insert these devices after the system has been booted. Another method could be to use only a single ATA drive and never list the SCSI devices in /etc/fstab.

A better solution is available. By using the glabel utility, an administrator or user may label their disk devices and use these labels in /etc/fstab. Because glabel stores the label in the last sector of a given provider, the label will remain persistent across reboots. By using this label as a device, the file system may always be mounted regardless of what device node it is accessed through.

Note: This goes without saying that a label be permanent. The glabel utility may be used to create both a transient and permanent label. Only the permanent label will remain consistent across reboots. See the glabel(8) manual page for more information on the differences between labels.

19.6.1 Label Types and Examples

There are two types of labels, a generic label and a file system label. Labels can be permanent or temporary. Permanent labels can be created with the tunefs(8) or newfs(8) commands. They will then be created in a sub-directory of /dev, which will be named according to their file system type. For example, UFS2 file system labels will be created in the /dev/ufs directory. Permanent labels can also be created with the glabel label command. These are not file system specific, and will be created in the /dev/label directory.

A temporary label will go away with the next reboot. These labels will be created in the /dev/label directory and are perfect for experimentation. A temporary label can be created using the glabel create command. For more information, please read the manual page of glabel(8).

To create a permanent label for a UFS2 file system without destroying any data, issue the following command:

# tunefs -L home /dev/da3

Warning: If the file system is full, this may cause data corruption; however, if the file system is full then the main goal should be removing stale files and not adding labels.

A label should now exist in /dev/ufs which may be added to /etc/fstab:

/dev/ufs/home      /home            ufs     rw              2      2

Note: The file system must not be mounted while attempting to run tunefs.

Now the file system may be mounted like normal:

# mount /home

From this point on, so long as the geom_label.ko kernel module is loaded at boot with /boot/loader.conf or the GEOM_LABEL kernel option is present, the device node may change without any ill effect on the system.

File systems may also be created with a default label by using the -L flag with newfs. See the newfs(8) manual page for more information.

The following command can be used to destroy the label:

# glabel destroy home

The following example shows how to label the partitions of a boot disk.

Example 19-1. Labeling Partitions on the Boot Disk

By permanently labeling the partitions on the boot disk, the system should be able to continue to boot normally, even if the disk is moved to another controller or transferred to a different system. For this example, it is assumed that a single ATA disk is used, which is currently recognized by the system as ad0. It is also assumed that the standard FreeBSD partition scheme is used, with /, /var, /usr and /tmp file systems, as well as a swap partition.

Reboot the system, and at the loader(8) prompt, press 4 to boot into single user mode. Then enter the following commands:

# glabel label rootfs /dev/ad0s1a
GEOM_LABEL: Label for provider /dev/ad0s1a is label/rootfs
# glabel label var /dev/ad0s1d
GEOM_LABEL: Label for provider /dev/ad0s1d is label/var
# glabel label usr /dev/ad0s1f
GEOM_LABEL: Label for provider /dev/ad0s1f is label/usr
# glabel label tmp /dev/ad0s1e
GEOM_LABEL: Label for provider /dev/ad0s1e is label/tmp
# glabel label swap /dev/ad0s1b
GEOM_LABEL: Label for provider /dev/ad0s1b is label/swap
# exit

The system will continue with multi-user boot. After the boot completes, edit /etc/fstab and replace the conventional device names, with their respective labels. The final /etc/fstab file will look like the following:

# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/label/swap         none            swap    sw              0       0
/dev/label/rootfs       /               ufs     rw              1       1
/dev/label/tmp          /tmp            ufs     rw              2       2
/dev/label/usr          /usr            ufs     rw              2       2
/dev/label/var          /var            ufs     rw              2       2

The system can now be rebooted. If everything went well, it will come up normally and mount will show:

# mount
/dev/label/rootfs on / (ufs, local)
devfs on /dev (devfs, local)
/dev/label/tmp on /tmp (ufs, local, soft-updates)
/dev/label/usr on /usr (ufs, local, soft-updates)
/dev/label/var on /var (ufs, local, soft-updates)

19.7 UFS Journaling Through GEOM

With the release of FreeBSD 7.0, the long awaited feature of UFS journals has been implemented. The implementation itself is provided through the GEOM subsystem and is easily configured via the gjournal(8) utility.

What is journaling? Journaling capability stores a log of file system transactions, i.e.: changes that make up a complete disk write operation, before meta-data and file writes are committed to the disk proper. This transaction log can later be replayed to redo file system transactions, preventing file system inconsistencies.

This method is yet another mechanism to protect against data loss and inconsistencies of the file system. Unlike Soft Updates which tracks and enforces meta-data updates and Snapshots which is an image of the file system, an actual log is stored in disk space specifically reserved for this task, and in some cases may be stored on another disk entirely.

Unlike other file system journaling implementations, the gjournal method is block based and not implemented as part of the file system - only as a GEOM extension.

To enable support for gjournal, the FreeBSD kernel must have the following option - which is the default on 7.X systems:

options    UFS_GJOURNAL

If journaled volumes need to be mounted during startup, the geom_journal.ko kernel module will also have to be loaded, by adding the following line in /boot/loader.conf:

geom_journal_load="YES"

Alternatively, this function can also be built into a custom kernel, by adding the following line in the kernel configuration file:

options    GEOM_JOURNAL

Creating a journal on a free file system may now be done using the following steps, considering that the da4 is a new SCSI disk:

# gjournal label /dev/da4
# gjournal load

At this point, there should be a /dev/da4 device node and a /dev/da4.journal device node. A file system may now be created on this device:

# newfs -O 2 -J /dev/da4.journal

The previously issued command will create a UFS2 file system with journaling being made active.

Effectively mount the device at the desired point with:

# mount /dev/da4.journal /mnt

Note: In the case of several slices, a journal will be created for each individual slice. For instance, if ad4s1 and ad4s2 are both slices, then gjournal will create ad4s1.journal and ad4s2.journal. In the case of the command being run twice, the result will be “journals”.

Under some circumstances, keeping the journal on another disk may be desired. For these cases, the journal provider or storage device should be listed after the device to enable journaling on. Journaling may also be enabled on current file systems by using tunefs; however, always make a backup before attempting to alter a file system. In most cases, the gjournal will fail if it is unable to create the actual journal but this does not protect against data loss incurred as a result of misusing tunefs.

It is also possible to journal the boot disk of a FreeBSD system. Please refer to the article Implementing UFS Journaling on a Desktop PC for detailed instructions on this task.

Tags: backup, bsd, cron, freebsd, FreeBSD Handbook, software

FreeBSD Handbook backup, bsd, cron, freebsd, software

FreeBSD Handbook - Chapter 18 Storage

January 6th, 2009

No comments

18.1 Synopsis

This chapter covers the use of disks in FreeBSD. This includes memory-backed disks, network-attached disks, standard SCSI/IDE storage devices, and devices using the USB interface.

After reading this chapter, you will know:

The terminology FreeBSD uses to describe the organization of data on a physical disk (partitions and slices).
How to add additional hard disks to your system.
How to configure FreeBSD to use USB storage devices.
How to set up virtual file systems, such as memory disks.
How to use quotas to limit disk space usage.
How to encrypt disks to secure them against attackers.
How to create and burn CDs and DVDs on FreeBSD.
The various storage media options for backups.
How to use backup programs available under FreeBSD.
How to backup to floppy disks.
What file system snapshots are and how to use them efficiently.

Before reading this chapter, you should:

Know how to configure and install a new FreeBSD kernel (Chapter 8).

18.2 Device Names

The following is a list of physical storage devices supported in FreeBSD, and the device names associated with them.

Table 18-1. Physical Disk Naming Conventions

Drive type
Drive device name

IDE hard drives
ad

IDE CDROM drives
acd

SCSI hard drives and USB Mass storage devices
da

SCSI CDROM drives
cd

Assorted non-standard CDROM drives
mcd for Mitsumi CD-ROM and scd for Sony CD-ROM devices

Floppy drives
fd

SCSI tape drives
sa

IDE tape drives
ast

Flash drives
fla for DiskOnChip® Flash device

RAID drives
aacd for Adaptec AdvancedRAID, mlxd and mlyd for Mylex, amrd for AMI MegaRAID, idad for Compaq Smart RAID, twed for 3ware® RAID.

18.3 Adding Disks

Originally contributed by David O’Brien.

Lets say we want to add a new SCSI disk to a machine that currently only has a single drive. First turn off the computer and install the drive in the computer following the instructions of the computer, controller, and drive manufacturer. Due to the wide variations of procedures to do this, the details are beyond the scope of this document.

Login as user root. After you have installed the drive, inspect /var/run/dmesg.boot to ensure the new disk was found. Continuing with our example, the newly added drive will be da1 and we want to mount it on /1 (if you are adding an IDE drive, the device name will be ad1).

FreeBSD runs on IBM-PC compatible computers, therefore it must take into account the PC BIOS partitions. These are different from the traditional BSD partitions. A PC disk has up to four BIOS partition entries. If the disk is going to be truly dedicated to FreeBSD, you can use the dedicated mode. Otherwise, FreeBSD will have to live within one of the PC BIOS partitions. FreeBSD calls the PC BIOS partitions slices so as not to confuse them with traditional BSD partitions. You may also use slices on a disk that is dedicated to FreeBSD, but used in a computer that also has another operating system installed. This is a good way to avoid confusing the fdisk utility of other, non-FreeBSD operating systems.

In the slice case the drive will be added as /dev/da1s1e. This is read as: SCSI disk, unit number 1 (second SCSI disk), slice 1 (PC BIOS partition 1), and e BSD partition. In the dedicated case, the drive will be added simply as /dev/da1e.

Due to the use of 32-bit integers to store the number of sectors, bsdlabel(8) is limited to 2^32-1 sectors per disk or 2TB in most cases. The fdisk(8) format allows a starting sector of no more than 2^32-1 and a length of no more than 2^32-1, limiting partitions to 2TB and disks to 4TB in most cases. The sunlabel(8) format is limited to 2^32-1 sectors per partition and 8 partitions for a total of 16TB. For larger disks, gpt(8) partitions may be used.

18.3.1 Using sysinstall(8)

Navigating Sysinstall

You may use sysinstall to partition and label a new disk using its easy to use menus. Either login as user root or use the su command. Run sysinstall and enter the Configure menu. Within the FreeBSD Configuration Menu, scroll down and select the Fdisk option.
fdisk Partition Editor

Once inside fdisk, pressing A will use the entire disk for FreeBSD. When asked if you want to “remain cooperative with any future possible operating systems”, answer YES. Write the changes to the disk using W. Now exit the FDISK editor by pressing Q. Next you will be asked about the “Master Boot Record”. Since you are adding a disk to an already running system, choose None.
Disk Label Editor

Next, you need to exit sysinstall and start it again. Follow the directions above, although this time choose the Label option. This will enter the Disk Label Editor. This is where you will create the traditional BSD partitions. A disk can have up to eight partitions, labeled a-h. A few of the partition labels have special uses. The a partition is used for the root partition (/). Thus only your system disk (e.g, the disk you boot from) should have an a partition. The b partition is used for swap partitions, and you may have many disks with swap partitions. The c partition addresses the entire disk in dedicated mode, or the entire FreeBSD slice in slice mode. The other partitions are for general use.

sysinstall’s Label editor favors the e partition for non-root, non-swap partitions. Within the Label editor, create a single file system by pressing C. When prompted if this will be a FS (file system) or swap, choose FS and type in a mount point (e.g, /mnt). When adding a disk in post-install mode, sysinstall will not create entries in /etc/fstab for you, so the mount point you specify is not important.

You are now ready to write the new label to the disk and create a file system on it. Do this by pressing W. Ignore any errors from sysinstall that it could not mount the new partition. Exit the Label Editor and sysinstall completely.
Finish

The last step is to edit /etc/fstab to add an entry for your new disk.

18.3.2 Using Command Line Utilities

18.3.2.1 Using Slices

This setup will allow your disk to work correctly with other operating systems that might be installed on your computer and will not confuse other operating systems’ fdisk utilities. It is recommended to use this method for new disk installs. Only use dedicated mode if you have a good reason to do so!

# dd if=/dev/zero of=/dev/da1 bs=1k count=1
# fdisk -BI da1 #Initialize your new disk
# bsdlabel -B -w da1s1 auto #Label it.
# bsdlabel -e da1s1 # Edit the bsdlabel just created and add any partitions.
# mkdir -p /1
# newfs /dev/da1s1e # Repeat this for every partition you created.
# mount /dev/da1s1e /1 # Mount the partition(s)
# vi /etc/fstab # Add the appropriate entry/entries to your /etc/fstab.

If you have an IDE disk, substitute ad for da.

18.3.2.2 Dedicated

If you will not be sharing the new drive with another operating system, you may use the dedicated mode. Remember this mode can confuse Microsoft operating systems; however, no damage will be done by them. IBM’s OS/2 however, will “appropriate” any partition it finds which it does not understand.

# dd if=/dev/zero of=/dev/da1 bs=1k count=1
# bsdlabel -Bw da1 auto
# bsdlabel -e da1               # create the `e' partition
# newfs /dev/da1e
# mkdir -p /1
# vi /etc/fstab               # add an entry for /dev/da1e
# mount /1

An alternate method is:

# dd if=/dev/zero of=/dev/da1 count=2
# bsdlabel /dev/da1 | bsdlabel -BR da1 /dev/stdin
# newfs /dev/da1e
# mkdir -p /1
# vi /etc/fstab                   # add an entry for /dev/da1e
# mount /1

18.4 RAID

18.4.1 Software RAID

18.4.1.1 Concatenated Disk Driver (CCD) Configuration

Original work by Christopher Shumway. Revised by Jim Brown.

When choosing a mass storage solution the most important factors to consider are speed, reliability, and cost. It is rare to have all three in balance; normally a fast, reliable mass storage device is expensive, and to cut back on cost either speed or reliability must be sacrificed.

In designing the system described below, cost was chosen as the most important factor, followed by speed, then reliability. Data transfer speed for this system is ultimately constrained by the network. And while reliability is very important, the CCD drive described below serves online data that is already fully backed up on CD-R’s and can easily be replaced.

Defining your own requirements is the first step in choosing a mass storage solution. If your requirements prefer speed or reliability over cost, your solution will differ from the system described in this section.

18.4.1.1.1 Installing the Hardware

In addition to the IDE system disk, three Western Digital 30GB, 5400 RPM IDE disks form the core of the CCD disk described below providing approximately 90GB of online storage. Ideally, each IDE disk would have its own IDE controller and cable, but to minimize cost, additional IDE controllers were not used. Instead the disks were configured with jumpers so that each IDE controller has one master, and one slave.

Upon reboot, the system BIOS was configured to automatically detect the disks attached. More importantly, FreeBSD detected them on reboot:

ad0: 19574MB <WDC WD205BA> [39770/16/63] at ata0-master UDMA33
ad1: 29333MB <WDC WD307AA> [59598/16/63] at ata0-slave UDMA33
ad2: 29333MB <WDC WD307AA> [59598/16/63] at ata1-master UDMA33
ad3: 29333MB <WDC WD307AA> [59598/16/63] at ata1-slave UDMA33

Note: If FreeBSD does not detect all the disks, ensure that you have jumpered them correctly. Most IDE drives also have a “Cable Select” jumper. This is not the jumper for the master/slave relationship. Consult the drive documentation for help in identifying the correct jumper.

Next, consider how to attach them as part of the file system. You should research both vinum(8) (Chapter 21) and ccd(4). In this particular configuration, ccd(4) was chosen.

18.4.1.1.2 Setting Up the CCD

The ccd(4) driver allows you to take several identical disks and concatenate them into one logical file system. In order to use ccd(4), you need a kernel with ccd(4) support built in. Add this line to your kernel configuration file, rebuild, and reinstall the kernel:

device   ccd

The ccd(4) support can also be loaded as a kernel loadable module.

To set up ccd(4), you must first use bsdlabel(8) to label the disks:

bsdlabel -w ad1 auto
bsdlabel -w ad2 auto
bsdlabel -w ad3 auto

This creates a bsdlabel for ad1c, ad2c and ad3c that spans the entire disk.

The next step is to change the disk label type. You can use bsdlabel(8) to edit the disks:

bsdlabel -e ad1
bsdlabel -e ad2
bsdlabel -e ad3

This opens up the current disk label on each disk with the editor specified by the EDITOR environment variable, typically vi(1).

An unmodified disk label will look something like this:

8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 60074784        0    unused        0     0     0   # (Cyl.    0 - 59597)

Add a new e partition for ccd(4) to use. This can usually be copied from the c partition, but the fstype must be 4.2BSD. The disk label should now look something like this:

8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 60074784        0    unused        0     0     0   # (Cyl.    0 - 59597)
  e: 60074784        0    4.2BSD        0     0     0   # (Cyl.    0 - 59597)

18.4.1.1.3 Building the File System

Now that you have all the disks labeled, you must build the ccd(4). To do that, use ccdconfig(8), with options similar to the following:

ccdconfig ccd0 32 0 /dev/ad1e /dev/ad2e /dev/ad3e

The use and meaning of each option is shown below:

: The first argument is the device to configure, in this case, /dev/ccd0c. The /dev/ portion is optional.
: The interleave for the file system. The interleave defines the size of a stripe in disk blocks, each normally 512 bytes. So, an interleave of 32 would be 16,384 bytes.
: Flags for ccdconfig(8). If you want to enable drive mirroring, you can specify a flag here. This configuration does not provide mirroring for ccd(4), so it is set at 0 (zero).
: The final arguments to ccdconfig(8) are the devices to place into the array. Use the complete pathname for each device.

After running ccdconfig(8) the ccd(4) is configured. A file system can be installed. Refer to newfs(8) for options, or simply run:

newfs /dev/ccd0c

18.4.1.1.4 Making it All Automatic

Generally, you will want to mount the ccd(4) upon each reboot. To do this, you must configure it first. Write out your current configuration to /etc/ccd.conf using the following command:

ccdconfig -g > /etc/ccd.conf

During reboot, the script /etc/rc runs ccdconfig -C if /etc/ccd.conf exists. This automatically configures the ccd(4) so it can be mounted.

Note: If you are booting into single user mode, before you can mount(8) the ccd(4), you need to issue the following command to configure the array:
ccdconfig -C

To automatically mount the ccd(4), place an entry for the ccd(4) in /etc/fstab so it will be mounted at boot time:

/dev/ccd0c              /media       ufs     rw      2       2

18.4.1.2 The Vinum Volume Manager

The Vinum Volume Manager is a block device driver which implements virtual disk drives. It isolates disk hardware from the block device interface and maps data in ways which result in an increase in flexibility, performance and reliability compared to the traditional slice view of disk storage. vinum(8) implements the RAID-0, RAID-1 and RAID-5 models, both individually and in combination.

See Chapter 21 for more information about vinum(8).

18.4.2 Hardware RAID

FreeBSD also supports a variety of hardware RAID controllers. These devices control a RAID subsystem without the need for FreeBSD specific software to manage the array.

Using an on-card BIOS, the card controls most of the disk operations itself. The following is a brief setup description using a Promise IDE RAID controller. When this card is installed and the system is started up, it displays a prompt requesting information. Follow the instructions to enter the card’s setup screen. From here, you have the ability to combine all the attached drives. After doing so, the disk(s) will look like a single drive to FreeBSD. Other RAID levels can be set up accordingly.

18.4.3 Rebuilding ATA RAID1 Arrays

FreeBSD allows you to hot-replace a failed disk in an array. This requires that you catch it before you reboot.

You will probably see something like the following in /var/log/messages or in the dmesg(8) output:

ad6 on monster1 suffered a hard error.
ad6: READ command timeout tag=0 serv=0 - resetting
ad6: trying fallback to PIO mode
ata3: resetting devices .. done
ad6: hard error reading fsbn 1116119 of 0-7 (ad6 bn 1116119; cn 1107 tn 4 sn 11)\\
status=59 error=40
ar0: WARNING - mirror lost

Using atacontrol(8), check for further information:

# atacontrol list
ATA channel 0:
    Master:      no device present
    Slave:   acd0 <HL-DT-ST CD-ROM GCR-8520B/1.00> ATA/ATAPI rev 0

ATA channel 1:
    Master:      no device present
    Slave:       no device present

ATA channel 2:
    Master:  ad4 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5
    Slave:       no device present

ATA channel 3:
    Master:  ad6 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5
    Slave:       no device present

# atacontrol status ar0
ar0: ATA RAID1 subdisks: ad4 ad6 status: DEGRADED

You will first need to detach the ata channel with the failed disk so you can safely remove it:
```
# atacontrol detach ata3
```
Replace the disk.

Reattach the ata channel:

# atacontrol attach ata3
Master:  ad6 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5
Slave:   no device present

Add the new disk to the array as a spare:
```
# atacontrol addspare ar0 ad6
```
Rebuild the array:
```
# atacontrol rebuild ar0
```

It is possible to check on the progress by issuing the following command:

# dmesg | tail -10
[output removed]
ad6: removed from configuration
ad6: deleted from ar0 disk1
ad6: inserted into ar0 disk1 as spare

# atacontrol status ar0
ar0: ATA RAID1 subdisks: ad4 ad6 status: REBUILDING 0% completed

Wait until this operation completes.

18.5 USB Storage Devices

Contributed by Marc Fonvieille.

A lot of external storage solutions, nowadays, use the Universal Serial Bus (USB): hard drives, USB thumbdrives, CD-R burners, etc. FreeBSD provides support for these devices.

18.5.1 Configuration

The USB mass storage devices driver, umass(4), provides the support for USB storage devices. If you use the GENERIC kernel, you do not have to change anything in your configuration. If you use a custom kernel, be sure that the following lines are present in your kernel configuration file:

device scbus
device da
device pass
device uhci
device ohci
device usb
device umass

The umass(4) driver uses the SCSI subsystem to access to the USB storage devices, your USB device will be seen as a SCSI device by the system. Depending on the USB chipset on your motherboard, you only need either device uhci or device ohci, however having both in the kernel configuration file is harmless. Do not forget to compile and install the new kernel if you added any lines.

Note: If your USB device is a CD-R or DVD burner, the SCSI CD-ROM driver, cd(4), must be added to the kernel via the line:
device cd
Since the burner is seen as a SCSI drive, the driver atapicam(4) should not be used in the kernel configuration.

Support for USB 2.0 controllers is provided on FreeBSD; however, you must add:

device ehci

to your configuration file for USB 2.0 support. Note uhci(4) and ohci(4) drivers are still needed if you want USB 1.X support.

18.5.2 Testing the Configuration

The configuration is ready to be tested: plug in your USB device, and in the system message buffer (dmesg(8)), the drive should appear as something like:

umass0: USB Solid state disk, rev 1.10/1.00, addr 2
GEOM: create disk da0 dp=0xc2d74850
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <Generic Traveling Disk 1.11> Removable Direct Access SCSI-2 device
da0: 1.000MB/s transfers
da0: 126MB (258048 512 byte sectors: 64H 32S/T 126C)

Of course, the brand, the device node (da0) and other details can differ according to your configuration.

Since the USB device is seen as a SCSI one, the camcontrol command can be used to list the USB storage devices attached to the system:

# camcontrol devlist
<Generic Traveling Disk 1.11>      at scbus0 target 0 lun 0 (da0,pass0)

If the drive comes with a file system, you should be able to mount it. The Section 18.3 will help you to format and create partitions on the USB drive if needed.

To make this device mountable as a normal user, certain steps have to be taken. First, the devices that are created when a USB storage device is connected need to be accessible by the user. A solution is to make all users of these devices a member of the operator group. This is done with pw(8). Second, when the devices are created, the operator group should be able to read and write them. This is accomplished by adding these lines to /etc/devfs.rules:

[localrules=5]
add path 'da*' mode 0660 group operator

Note: If there already are SCSI disks in the system, it must be done a bit different. E.g., if the system already contains disks da0 through da2 attached to the system, change the second line as follows:
add path 'da[3-9]*' mode 0660 group operator
This will exclude the already existing disks from belonging to the operator group.

You also have to enable your devfs.rules(5) ruleset in your /etc/rc.conf file:

devfs_system_ruleset="localrules"

Next, the kernel has to be configured to allow regular users to mount file systems. The easiest way is to add the following line to /etc/sysctl.conf:

vfs.usermount=1

Note that this only takes effect after the next reboot. Alternatively, one can also use sysctl(8) to set this variable.

The final step is to create a directory where the file system is to be mounted. This directory needs to be owned by the user that is to mount the file system. One way to do that is for root to create a subdirectory owned by that user as /mnt/username (replace username by the login name of the actual user and usergroup by the user’s primary group):

# mkdir /mnt/username
# chown username:usergroup /mnt/username

Suppose a USB thumbdrive is plugged in, and a device /dev/da0s1 appears. Since these devices usually come preformatted with a FAT file system, one can mount them like this:

% mount -t msdosfs -o -m=644,-M=755 /dev/da0s1 /mnt/username

If you unplug the device (the disk must be unmounted before), you should see, in the system message buffer, something like the following:

umass0: at uhub0 port 1 (addr 2) disconnected
(da0:umass-sim0:0:0:0): lost device
(da0:umass-sim0:0:0:0): removing device entry
GEOM: destroy disk da0 dp=0xc2d74850
umass0: detached

18.5.3 Further Reading

Beside the Adding Disks and Mounting and Unmounting File Systems sections, reading various manual pages may be also useful: umass(4), camcontrol(8), and usbdevs(8).

18.6 Creating and Using Optical Media (CDs)

Contributed by Mike Meyer.

18.6.1 Introduction

CDs have a number of features that differentiate them from conventional disks. Initially, they were not writable by the user. They are designed so that they can be read continuously without delays to move the head between tracks. They are also much easier to transport between systems than similarly sized media were at the time.

CDs do have tracks, but this refers to a section of data to be read continuously and not a physical property of the disk. To produce a CD on FreeBSD, you prepare the data files that are going to make up the tracks on the CD, then write the tracks to the CD.

The ISO 9660 file system was designed to deal with these differences. It unfortunately codifies file system limits that were common then. Fortunately, it provides an extension mechanism that allows properly written CDs to exceed those limits while still working with systems that do not support those extensions.

The sysutils/cdrtools port includes mkisofs(8), a program that you can use to produce a data file containing an ISO 9660 file system. It has options that support various extensions, and is described below.

Which tool to use to burn the CD depends on whether your CD burner is ATAPI or something else. ATAPI CD burners use the burncd program that is part of the base system. SCSI and USB CD burners should use cdrecord from the sysutils/cdrtools port. It is also possible to use cdrecord and other tools for SCSI drives on ATAPI hardware with the ATAPI/CAM module.

If you want CD burning software with a graphical user interface, you may wish to take a look at either X-CD-Roast or K3b. These tools are available as packages or from the sysutils/xcdroast and sysutils/k3b ports. X-CD-Roast and K3b require the ATAPI/CAM module with ATAPI hardware.

18.6.2 mkisofs

The mkisofs(8) program, which is part of the sysutils/cdrtools port, produces an ISO 9660 file system that is an image of a directory tree in the UNIX file system name space. The simplest usage is:

# mkisofs -o imagefile.iso /path/to/tree

This command will create an imagefile.iso containing an ISO 9660 file system that is a copy of the tree at /path/to/tree. In the process, it will map the file names to names that fit the limitations of the standard ISO 9660 file system, and will exclude files that have names uncharacteristic of ISO file systems.

A number of options are available to overcome those restrictions. In particular, -R enables the Rock Ridge extensions common to UNIX systems, -J enables Joliet extensions used by Microsoft systems, and -hfs can be used to create HFS file systems used by Mac OS.

For CDs that are going to be used only on FreeBSD systems, -U can be used to disable all filename restrictions. When used with -R, it produces a file system image that is identical to the FreeBSD tree you started from, though it may violate the ISO 9660 standard in a number of ways.

The last option of general use is -b. This is used to specify the location of the boot image for use in producing an “El Torito” bootable CD. This option takes an argument which is the path to a boot image from the top of the tree being written to the CD. By default, mkisofs(8) creates an ISO image in the so-called “floppy disk emulation” mode, and thus expects the boot image to be exactly 1200, 1440 or 2880 KB in size. Some boot loaders, like the one used by the FreeBSD distribution disks, do not use emulation mode; in this case, the -no-emul-boot option should be used. So, if /tmp/myboot holds a bootable FreeBSD system with the boot image in /tmp/myboot/boot/cdboot, you could produce the image of an ISO 9660 file system in /tmp/bootable.iso like so:

# mkisofs -R -no-emul-boot -b boot/cdboot -o /tmp/bootable.iso /tmp/myboot

Having done that, if you have md configured in your kernel, you can mount the file system with:

# mdconfig -a -t vnode -f /tmp/bootable.iso -u 0
# mount -t cd9660 /dev/md0 /mnt

At which point you can verify that /mnt and /tmp/myboot are identical.

There are many other options you can use with mkisofs(8) to fine-tune its behavior. In particular: modifications to an ISO 9660 layout and the creation of Joliet and HFS discs. See the mkisofs(8) manual page for details.

18.6.3 burncd

If you have an ATAPI CD burner, you can use the burncd command to burn an ISO image onto a CD. burncd is part of the base system, installed as /usr/sbin/burncd. Usage is very simple, as it has few options:

# burncd -f cddevice data imagefile.iso fixate

Will burn a copy of imagefile.iso on cddevice. The default device is /dev/acd0. See burncd(8) for options to set the write speed, eject the CD after burning, and write audio data.

18.6.4 cdrecord

If you do not have an ATAPI CD burner, you will have to use cdrecord to burn your CDs. cdrecord is not part of the base system; you must install it from either the port at sysutils/cdrtools or the appropriate package. Changes to the base system can cause binary versions of this program to fail, possibly resulting in a “coaster”. You should therefore either upgrade the port when you upgrade your system, or if you are tracking -STABLE, upgrade the port when a new version becomes available.

While cdrecord has many options, basic usage is even simpler than burncd. Burning an ISO 9660 image is done with:

# cdrecord dev=device imagefile.iso

The tricky part of using cdrecord is finding the dev to use. To find the proper setting, use the -scanbus flag of cdrecord, which might produce results like this:

# cdrecord -scanbus
Cdrecord-Clone 2.01 (i386-unknown-freebsd7.0) Copyright (C) 1995-2004 Jörg Schilling
Using libscg version 'schily-0.1'
scsibus0:
        0,0,0     0) 'SEAGATE ' 'ST39236LW       ' '0004' Disk
        0,1,0     1) 'SEAGATE ' 'ST39173W        ' '5958' Disk
        0,2,0     2) *
        0,3,0     3) 'iomega  ' 'jaz 1GB         ' 'J.86' Removable Disk
        0,4,0     4) 'NEC     ' 'CD-ROM DRIVE:466' '1.26' Removable CD-ROM
        0,5,0     5) *
        0,6,0     6) *
        0,7,0     7) *
scsibus1:
        1,0,0   100) *
        1,1,0   101) *
        1,2,0   102) *
        1,3,0   103) *
        1,4,0   104) *
        1,5,0   105) 'YAMAHA  ' 'CRW4260         ' '1.0q' Removable CD-ROM
        1,6,0   106) 'ARTEC   ' 'AM12S           ' '1.06' Scanner
        1,7,0   107) *

This lists the appropriate dev value for the devices on the list. Locate your CD burner, and use the three numbers separated by commas as the value for dev. In this case, the CRW device is 1,5,0, so the appropriate input would be dev=1,5,0. There are easier ways to specify this value; see cdrecord(1) for details. That is also the place to look for information on writing audio tracks, controlling the speed, and other things.

18.6.5 Duplicating Audio CDs

You can duplicate an audio CD by extracting the audio data from the CD to a series of files, and then writing these files to a blank CD. The process is slightly different for ATAPI and SCSI drives.

SCSI Drives

Use cdda2wav to extract the audio.
```
% cdda2wav -v255 -D2,0 -B -Owav
```
Use cdrecord to write the .wav files.
```
% cdrecord -v dev=2,0 -dao -useinfo  *.wav
```
Make sure that 2,0 is set appropriately, as described in Section 18.6.4.

ATAPI Drives

The ATAPI CD driver makes each track available as /dev/acddtnn, where d is the drive number, and nn is the track number written with two decimal digits, prefixed with zero as needed. So the first track on the first disk is /dev/acd0t01, the second is /dev/acd0t02, the third is /dev/acd0t03, and so on.

Make sure the appropriate files exist in /dev. If the entries are missing, force the system to retaste the media:
```
# dd if=/dev/acd0 of=/dev/null count=1
```
Extract each track using dd(1). You must also use a specific block size when extracting the files.
```
# dd if=/dev/acd0t01 of=track1.cdr bs=2352
# dd if=/dev/acd0t02 of=track2.cdr bs=2352
...
```
Burn the extracted files to disk using burncd. You must specify that these are audio files, and that burncd should fixate the disk when finished.
```
# burncd -f /dev/acd0 audio track1.cdr track2.cdr ... fixate
```

18.6.6 Duplicating Data CDs

You can copy a data CD to a image file that is functionally equivalent to the image file created with mkisofs(8), and you can use it to duplicate any data CD. The example given here assumes that your CDROM device is acd0. Substitute your correct CDROM device.

# dd if=/dev/acd0 of=file.iso bs=2048

Now that you have an image, you can burn it to CD as described above.

18.6.7 Using Data CDs

Now that you have created a standard data CDROM, you probably want to mount it and read the data on it. By default, mount(8) assumes that a file system is of type ufs. If you try something like:

# mount /dev/cd0 /mnt

you will get a complaint about “Incorrect super block”, and no mount. The CDROM is not a UFS file system, so attempts to mount it as such will fail. You just need to tell mount(8) that the file system is of type ISO9660, and everything will work. You do this by specifying the -t cd9660 option mount(8). For example, if you want to mount the CDROM device, /dev/cd0, under /mnt, you would execute:

# mount -t cd9660 /dev/cd0 /mnt

Note that your device name (/dev/cd0 in this example) could be different, depending on the interface your CDROM uses. Also, the -t cd9660 option just executes mount_cd9660(8). The above example could be shortened to:

# mount_cd9660 /dev/cd0 /mnt

You can generally use data CDROMs from any vendor in this way. Disks with certain ISO 9660 extensions might behave oddly, however. For example, Joliet disks store all filenames in two-byte Unicode characters. The FreeBSD kernel does not speak Unicode, but the FreeBSD CD9660 driver is able to convert Unicode characters on the fly. If some non-English characters show up as question marks you will need to specify the local charset you use with the -C option. For more information, consult the mount_cd9660(8) manual page.

Note: To be able to do this character conversion with the help of the -C option, the kernel will require the cd9660_iconv.ko module to be loaded. This can be done either by adding this line to loader.conf:
cd9660_iconv_load="YES"
and then rebooting the machine, or by directly loading the module with kldload(8).

Occasionally, you might get “Device not configured” when trying to mount a CDROM. This usually means that the CDROM drive thinks that there is no disk in the tray, or that the drive is not visible on the bus. It can take a couple of seconds for a CDROM drive to realize that it has been fed, so be patient.

Sometimes, a SCSI CDROM may be missed because it did not have enough time to answer the bus reset. If you have a SCSI CDROM please add the following option to your kernel configuration and rebuild your kernel.

options SCSI_DELAY=15000

This tells your SCSI bus to pause 15 seconds during boot, to give your CDROM drive every possible chance to answer the bus reset.

18.6.8 Burning Raw Data CDs

You can choose to burn a file directly to CD, without creating an ISO 9660 file system. Some people do this for backup purposes. This runs more quickly than burning a standard CD:

# burncd -f /dev/acd1 -s 12 data archive.tar.gz fixate

In order to retrieve the data burned to such a CD, you must read data from the raw device node:

# tar xzvf /dev/acd1

You cannot mount this disk as you would a normal CDROM. Such a CDROM cannot be read under any operating system except FreeBSD. If you want to be able to mount the CD, or share data with another operating system, you must use mkisofs(8) as described above.

18.6.9 Using the ATAPI/CAM Driver

Contributed by Marc Fonvieille.

This driver allows ATAPI devices (CD-ROM, CD-RW, DVD drives etc…) to be accessed through the SCSI subsystem, and so allows the use of applications like sysutils/cdrdao or cdrecord(1).

To use this driver, you will need to add the following line to the /boot/loader.conf file:

atapicam_load="YES"

then, reboot your machine.

Note: If you prefer to statically compile the atapicam(4) support in your kernel, you will have to add this line to your kernel configuration file:
device atapicam
You also need the following lines in your kernel configuration file:
device ata
device scbus
device cd
device pass
which should already be present. Then rebuild, install your new kernel, and reboot your machine.

During the boot process, your burner should show up, like so:

acd0: CD-RW <MATSHITA CD-RW/DVD-ROM UJDA740> at ata1-master PIO4
cd0 at ata1 bus 0 target 0 lun 0
cd0: <MATSHITA CDRW/DVD UJDA740 1.00> Removable CD-ROM SCSI-0 device
cd0: 16.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed

The drive could now be accessed via the /dev/cd0 device name, for example to mount a CD-ROM on /mnt, just type the following:

# mount -t cd9660 /dev/cd0 /mnt

As root, you can run the following command to get the SCSI address of the burner:

# camcontrol devlist
<MATSHITA CDRW/DVD UJDA740 1.00>   at scbus1 target 0 lun 0 (pass0,cd0)

So 1,0,0 will be the SCSI address to use with cdrecord(1) and other SCSI application.

For more information about ATAPI/CAM and SCSI system, refer to the atapicam(4) and cam(4) manual pages.

18.7 Creating and Using Optical Media (DVDs)

Contributed by Marc Fonvieille. With inputs from Andy Polyakov.

18.7.1 Introduction

Compared to the CD, the DVD is the next generation of optical media storage technology. The DVD can hold more data than any CD and is nowadays the standard for video publishing.

Five physical recordable formats can be defined for what we will call a recordable DVD:

DVD-R: This was the first DVD recordable format available. The DVD-R standard is defined by the DVD Forum. This format is write once.
DVD-RW: This is the rewritable version of the DVD-R standard. A DVD-RW can be rewritten about 1000 times.
DVD-RAM: This is also a rewritable format supported by the DVD Forum. A DVD-RAM can be seen as a removable hard drive. However, this media is not compatible with most DVD-ROM drives and DVD-Video players; only a few DVD writers support the DVD-RAM format. Read the Section 18.7.9 for more information on DVD-RAM use.
DVD+RW: This is a rewritable format defined by the DVD+RW Alliance. A DVD+RW can be rewritten about 1000 times.
DVD+R: This format is the write once variation of the DVD+RW format.

A single layer recordable DVD can hold up to 4,700,000,000 bytes which is actually 4.38 GB or 4485 MB (1 kilobyte is 1024 bytes).

Note: A distinction must be made between the physical media and the application. For example, a DVD-Video is a specific file layout that can be written on any recordable DVD physical media: DVD-R, DVD+R, DVD-RW etc. Before choosing the type of media, you must be sure that both the burner and the DVD-Video player (a standalone player or a DVD-ROM drive on a computer) are compatible with the media under consideration.

18.7.2 Configuration

The program growisofs(1) will be used to perform DVD recording. This command is part of the dvd+rw-tools utilities (sysutils/dvd+rw-tools). The dvd+rw-tools support all DVD media types.

These tools use the SCSI subsystem to access to the devices, therefore the ATAPI/CAM support must be added to your kernel. If your burner uses the USB interface this addition is useless, and you should read the Section 18.5 for more details on USB devices configuration.

You also have to enable DMA access for ATAPI devices, this can be done in adding the following line to the /boot/loader.conf file:

hw.ata.atapi_dma="1"

Before attempting to use the dvd+rw-tools you should consult the dvd+rw-tools’ hardware compatibility notes for any information related to your DVD burner.

Note: If you want a graphical user interface, you should have a look to K3b (sysutils/k3b) which provides a user friendly interface to growisofs(1) and many other burning tools.

18.7.3 Burning Data DVDs

The growisofs(1) command is a frontend to mkisofs, it will invoke mkisofs(8) to create the file system layout and will perform the write on the DVD. This means you do not need to create an image of the data before the burning process.

To burn onto a DVD+R or a DVD-R the data from the /path/to/data directory, use the following command:

# growisofs -dvd-compat -Z /dev/cd0 -J -R /path/to/data

The options -J -R are passed to mkisofs(8) for the file system creation (in this case: an ISO 9660 file system with Joliet and Rock Ridge extensions), consult the mkisofs(8) manual page for more details.

The option -Z is used for the initial session recording in any case: multiple sessions or not. The DVD device, /dev/cd0, must be changed according to your configuration. The -dvd-compat parameter will close the disk, the recording will be unappendable. In return this should provide better media compatibility with DVD-ROM drives.

It is also possible to burn a pre-mastered image, for example to burn the image imagefile.iso, we will run:

# growisofs -dvd-compat -Z /dev/cd0=imagefile.iso

The write speed should be detected and automatically set according to the media and the drive being used. If you want to force the write speed, use the -speed= parameter. For more information, read the growisofs(1) manual page.

18.7.4 Burning a DVD-Video

A DVD-Video is a specific file layout based on ISO 9660 and the micro-UDF (M-UDF) specifications. The DVD-Video also presents a specific data structure hierarchy, it is the reason why you need a particular program such as multimedia/dvdauthor to author the DVD.

If you already have an image of the DVD-Video file system, just burn it in the same way as for any image, see the previous section for an example. If you have made the DVD authoring and the result is in, for example, the directory /path/to/video, the following command should be used to burn the DVD-Video:

# growisofs -Z /dev/cd0 -dvd-video /path/to/video

The -dvd-video option will be passed down to mkisofs(8) and will instruct it to create a DVD-Video file system layout. Beside this, the -dvd-video option implies -dvd-compat growisofs(1) option.

18.7.5 Using a DVD+RW

Unlike CD-RW, a virgin DVD+RW needs to be formatted before first use. The growisofs(1) program will take care of it automatically whenever appropriate, which is the recommended way. However you can use the dvd+rw-format command to format the DVD+RW:

# dvd+rw-format /dev/cd0

You need to perform this operation just once, keep in mind that only virgin DVD+RW medias need to be formatted. Then you can burn the DVD+RW in the way seen in previous sections.

If you want to burn new data (burn a totally new file system not append some data) onto a DVD+RW, you do not need to blank it, you just have to write over the previous recording (in performing a new initial session), like this:

# growisofs -Z /dev/cd0 -J -R /path/to/newdata

DVD+RW format offers the possibility to easily append data to a previous recording. The operation consists in merging a new session to the existing one, it is not multisession writing, growisofs(1) will grow the ISO 9660 file system present on the media.

For example, if we want to append data to our previous DVD+RW, we have to use the following:

# growisofs -M /dev/cd0 -J -R /path/to/nextdata

The same mkisofs(8) options we used to burn the initial session should be used during next writes.

Note: You may want to use the -dvd-compat option if you want better media compatibility with DVD-ROM drives. In the DVD+RW case, this will not prevent you from adding data.

If for any reason you really want to blank the media, do the following:

# growisofs -Z /dev/cd0=/dev/zero

18.7.6 Using a DVD-RW

A DVD-RW accepts two disc formats: the incremental sequential one and the restricted overwrite. By default DVD-RW discs are in sequential format.

A virgin DVD-RW can be directly written without the need of a formatting operation, however a non-virgin DVD-RW in sequential format needs to be blanked before to be able to write a new initial session.

To blank a DVD-RW in sequential mode, run:

# dvd+rw-format -blank=full /dev/cd0

Note: A full blanking (-blank=full) will take about one hour on a 1x media. A fast blanking can be performed using the -blank option if the DVD-RW will be recorded in Disk-At-Once (DAO) mode. To burn the DVD-RW in DAO mode, use the command:
# growisofs -use-the-force-luke=dao -Z /dev/cd0=imagefile.iso
The -use-the-force-luke=dao option should not be required since growisofs(1) attempts to detect minimally (fast blanked) media and engage DAO write.

In fact one should use restricted overwrite mode with any DVD-RW, this format is more flexible than the default incremental sequential one.

To write data on a sequential DVD-RW, use the same instructions as for the other DVD formats:

# growisofs -Z /dev/cd0 -J -R /path/to/data

If you want to append some data to your previous recording, you will have to use the growisofs(1) -M option. However, if you perform data addition on a DVD-RW in incremental sequential mode, a new session will be created on the disc and the result will be a multi-session disc.

A DVD-RW in restricted overwrite format does not need to be blanked before a new initial session, you just have to overwrite the disc with the -Z option, this is similar to the DVD+RW case. It is also possible to grow an existing ISO 9660 file system written on the disc in a same way as for a DVD+RW with the -M option. The result will be a one-session DVD.

To put a DVD-RW in the restricted overwrite format, the following command must be used:

# dvd+rw-format /dev/cd0

To change back to the sequential format use:

# dvd+rw-format -blank=full /dev/cd0

18.7.7 Multisession

Very few DVD-ROM drives support multisession DVDs, they will most of time, hopefully, only read the first session. DVD+R, DVD-R and DVD-RW in sequential format can accept multiple sessions, the notion of multiple sessions does not exist for the DVD+RW and the DVD-RW restricted overwrite formats.

Using the following command after an initial (non-closed) session on a DVD+R, DVD-R, or DVD-RW in sequential format, will add a new session to the disc:

# growisofs -M /dev/cd0 -J -R /path/to/nextdata

Using this command line with a DVD+RW or a DVD-RW in restricted overwrite mode, will append data in merging the new session to the existing one. The result will be a single-session disc. This is the way used to add data after an initial write on these medias.

Note: Some space on the media is used between each session for end and start of sessions. Therefore, one should add sessions with large amount of data to optimize media space. The number of sessions is limited to 154 for a DVD+R, about 2000 for a DVD-R, and 127 for a DVD+R Double Layer.

18.7.8 For More Information

To obtain more information about a DVD, the dvd+rw-mediainfo /dev/cd0 command can be ran with the disc in the drive.

More information about the dvd+rw-tools can be found in the growisofs(1) manual page, on the dvd+rw-tools web site and in the cdwrite mailing list archives.

Note: The dvd+rw-mediainfo output of the resulting recording or the media with issues is mandatory for any problem report. Without this output, it will be quite impossible to help you.

18.7.9 Using a DVD-RAM

18.7.9.1 Configuration

DVD-RAM writers come with either SCSI or ATAPI interface. DMA access for ATAPI devices has to be enabled, this can be done by adding the following line to the /boot/loader.conf file:

hw.ata.atapi_dma="1"

18.7.9.2 Preparing the Medium

As previously mentioned in the chapter introduction, a DVD-RAM can be seen as a removable hard drive. As any other hard drive the DVD-RAM must be “prepared” before the first use. In the example, the whole disk space will be used with a standard UFS2 file system:

# dd if=/dev/zero of=/dev/acd0 bs=2k count=1
# bsdlabel -Bw acd0
# newfs /dev/acd0

The DVD device, acd0, must be changed according to the configuration.

18.7.9.3 Using the Medium

Once the previous operations have been performed on the DVD-RAM, it can be mounted as a normal hard drive:

# mount /dev/acd0 /mnt

After this the DVD-RAM will be both readable and writeable.

18.8 Creating and Using Floppy Disks

Original work by Julio Merino. Rewritten by Martin Karlsson.

Storing data on floppy disks is sometimes useful, for example when one does not have any other removable storage media or when one needs to transfer small amounts of data to another computer.

This section will explain how to use floppy disks in FreeBSD. It will primarily cover formatting and usage of 3.5inch DOS floppies, but the concepts are similar for other floppy disk formats.

18.8.1 Formatting Floppies

18.8.1.1 The Device

Floppy disks are accessed through entries in /dev, just like other devices. To access the raw floppy disk, simply use /dev/fdN.

18.8.1.2 Formatting

A floppy disk needs to be low-level formated before it can be used. This is usually done by the vendor, but formatting is a good way to check media integrity. Although it is possible to force larger (or smaller) disk sizes, 1440kB is what most floppy disks are designed for.

To low-level format the floppy disk you need to use fdformat(1). This utility expects the device name as an argument.

Make note of any error messages, as these can help determine if the disk is good or bad.

18.8.1.2.1 Formatting Floppy Disks

Use the /dev/fdN devices to format the floppy. Insert a new 3.5inch floppy disk in your drive and issue:

# /usr/sbin/fdformat -f 1440 /dev/fd0

18.8.2 The Disk Label

After low-level formatting the disk, you will need to place a disk label on it. This disk label will be destroyed later, but it is needed by the system to determine the size of the disk and its geometry later.

The new disk label will take over the whole disk, and will contain all the proper information about the geometry of the floppy. The geometry values for the disk label are listed in /etc/disktab.

You can run now bsdlabel(8) like so:

# /sbin/bsdlabel -B -w /dev/fd0 fd1440

18.8.3 The File System

Now the floppy is ready to be high-level formated. This will place a new file system on it, which will let FreeBSD read and write to the disk. After creating the new file system, the disk label is destroyed, so if you want to reformat the disk, you will have to recreate the disk label.

The floppy’s file system can be either UFS or FAT. FAT is generally a better choice for floppies.

To put a new file system on the floppy, issue:

# /sbin/newfs_msdos /dev/fd0

The disk is now ready for use.

18.8.4 Using the Floppy

To use the floppy, mount it with mount_msdosfs(8). One can also use emulators/mtools from the ports collection.

18.9 Creating and Using Data Tapes

The major tape media are the 4mm, 8mm, QIC, mini-cartridge and DLT.

18.9.1 4mm (DDS: Digital Data Storage)

4mm tapes are replacing QIC as the workstation backup media of choice. This trend accelerated greatly when Conner purchased Archive, a leading manufacturer of QIC drives, and then stopped production of QIC drives. 4mm drives are small and quiet but do not have the reputation for reliability that is enjoyed by 8mm drives. The cartridges are less expensive and smaller (3 x 2 x 0.5 inches, 76 x 51 x 12 mm) than 8mm cartridges. 4mm, like 8mm, has comparatively short head life for the same reason, both use helical scan.

Data throughput on these drives starts ~150 kB/s, peaking at ~500 kB/s. Data capacity starts at 1.3 GB and ends at 2.0 GB. Hardware compression, available with most of these drives, approximately doubles the capacity. Multi-drive tape library units can have 6 drives in a single cabinet with automatic tape changing. Library capacities reach 240 GB.

The DDS-3 standard now supports tape capacities up to 12 GB (or 24 GB compressed).

4mm drives, like 8mm drives, use helical-scan. All the benefits and drawbacks of helical-scan apply to both 4mm and 8mm drives.

Tapes should be retired from use after 2,000 passes or 100 full backups.

18.9.2 8mm (Exabyte)

8mm tapes are the most common SCSI tape drives; they are the best choice of exchanging tapes. Nearly every site has an Exabyte 2 GB 8mm tape drive. 8mm drives are reliable, convenient and quiet. Cartridges are inexpensive and small (4.8 x 3.3 x 0.6 inches; 122 x 84 x 15 mm). One downside of 8mm tape is relatively short head and tape life due to the high rate of relative motion of the tape across the heads.

Data throughput ranges from ~250 kB/s to ~500 kB/s. Data sizes start at 300 MB and go up to 7 GB. Hardware compression, available with most of these drives, approximately doubles the capacity. These drives are available as single units or multi-drive tape libraries with 6 drives and 120 tapes in a single cabinet. Tapes are changed automatically by the unit. Library capacities reach 840+ GB.

The Exabyte “Mammoth” model supports 12 GB on one tape (24 GB with compression) and costs approximately twice as much as conventional tape drives.

Data is recorded onto the tape using helical-scan, the heads are positioned at an angle to the media (approximately 6 degrees). The tape wraps around 270 degrees of the spool that holds the heads. The spool spins while the tape slides over the spool. The result is a high density of data and closely packed tracks that angle across the tape from one edge to the other.

18.9.3 QIC

QIC-150 tapes and drives are, perhaps, the most common tape drive and media around. QIC tape drives are the least expensive “serious” backup drives. The downside is the cost of media. QIC tapes are expensive compared to 8mm or 4mm tapes, up to 5 times the price per GB data storage. But, if your needs can be satisfied with a half-dozen tapes, QIC may be the correct choice. QIC is the most common tape drive. Every site has a QIC drive of some density or another. Therein lies the rub, QIC has a large number of densities on physically similar (sometimes identical) tapes. QIC drives are not quiet. These drives audibly seek before they begin to record data and are clearly audible whenever reading, writing or seeking. QIC tapes measure 6 x 4 x 0.7 inches (152 x 102 x 17 mm).

Data throughput ranges from ~150 kB/s to ~500 kB/s. Data capacity ranges from 40 MB to 15 GB. Hardware compression is available on many of the newer QIC drives. QIC drives are less frequently installed; they are being supplanted by DAT drives.

Data is recorded onto the tape in tracks. The tracks run along the long axis of the tape media from one end to the other. The number of tracks, and therefore the width of a track, varies with the tape’s capacity. Most if not all newer drives provide backward-compatibility at least for reading (but often also for writing). QIC has a good reputation regarding the safety of the data (the mechanics are simpler and more robust than for helical scan drives).

Tapes should be retired from use after 5,000 backups.

18.9.4 DLT

DLT has the fastest data transfer rate of all the drive types listed here. The 1/2" (12.5mm) tape is contained in a single spool cartridge (4 x 4 x 1 inches; 100 x 100 x 25 mm). The cartridge has a swinging gate along one entire side of the cartridge. The drive mechanism opens this gate to extract the tape leader. The tape leader has an oval hole in it which the drive uses to “hook” the tape. The take-up spool is located inside the tape drive. All the other tape cartridges listed here (9 track tapes are the only exception) have both the supply and take-up spools located inside the tape cartridge itself.

Data throughput is approximately 1.5 MB/s, three times the throughput of 4mm, 8mm, or QIC tape drives. Data capacities range from 10 GB to 20 GB for a single drive. Drives are available in both multi-tape changers and multi-tape, multi-drive tape libraries containing from 5 to 900 tapes over 1 to 20 drives, providing from 50 GB to 9 TB of storage.

With compression, DLT Type IV format supports up to 70 GB capacity.

Data is recorded onto the tape in tracks parallel to the direction of travel (just like QIC tapes). Two tracks are written at once. Read/write head lifetimes are relatively long; once the tape stops moving, there is no relative motion between the heads and the tape.

18.9.5 AIT

AIT is a new format from Sony, and can hold up to 50 GB (with compression) per tape. The tapes contain memory chips which retain an index of the tape’s contents. This index can be rapidly read by the tape drive to determine the position of files on the tape, instead of the several minutes that would be required for other tapes. Software such as SAMS:Alexandria can operate forty or more AIT tape libraries, communicating directly with the tape’s memory chip to display the contents on screen, determine what files were backed up to which tape, locate the correct tape, load it, and restore the data from the tape.

Libraries like this cost in the region of $20,000, pricing them a little out of the hobbyist market.

18.9.6 Using a New Tape for the First Time

The first time that you try to read or write a new, completely blank tape, the operation will fail. The console messages should be similar to:

sa0(ncr1:4:0): NOT READY asc:4,1
sa0(ncr1:4:0):  Logical unit is in process of becoming ready

The tape does not contain an Identifier Block (block number 0). All QIC tape drives since the adoption of QIC-525 standard write an Identifier Block to the tape. There are two solutions:

mt fsf 1 causes the tape drive to write an Identifier Block to the tape.
Use the front panel button to eject the tape.

Re-insert the tape and dump data to the tape.

dump will report “DUMP: End of tape detected” and the console will show: “HARDWARE FAILURE info:280 asc:80,96”.

rewind the tape using: mt rewind.

Subsequent tape operations are successful.

18.10 Backups to Floppies

18.10.1 Can I Use Floppies for Backing Up My Data?

Floppy disks are not really a suitable media for making backups as:

The media is unreliable, especially over long periods of time.
Backing up and restoring is very slow.
They have a very limited capacity (the days of backing up an entire hard disk onto a dozen or so floppies has long since passed).

However, if you have no other method of backing up your data then floppy disks are better than no backup at all.

If you do have to use floppy disks then ensure that you use good quality ones. Floppies that have been lying around the office for a couple of years are a bad choice. Ideally use new ones from a reputable manufacturer.

18.10.2 So How Do I Backup My Data to Floppies?

The best way to backup to floppy disk is to use tar(1) with the -M (multi volume) option, which allows backups to span multiple floppies.

To backup all the files in the current directory and sub-directory use this (as root):

# tar Mcvf /dev/fd0 *

When the first floppy is full tar(1) will prompt you to insert the next volume (because tar(1) is media independent it refers to volumes; in this context it means floppy disk).

Prepare volume #2 for /dev/fd0 and hit return:

This is repeated (with the volume number incrementing) until all the specified files have been archived.

18.10.3 Can I Compress My Backups?

Unfortunately, tar(1) will not allow the -z option to be used for multi-volume archives. You could, of course, gzip(1) all the files, tar(1) them to the floppies, then gunzip(1) the files again!

18.10.4 How Do I Restore My Backups?

To restore the entire archive use:

# tar Mxvf /dev/fd0

There are two ways that you can use to restore only specific files. First, you can start with the first floppy and use:

# tar Mxvf /dev/fd0 filename

The utility tar(1) will prompt you to insert subsequent floppies until it finds the required file.

Alternatively, if you know which floppy the file is on then you can simply insert that floppy and use the same command as above. Note that if the first file on the floppy is a continuation from the previous one then tar(1) will warn you that it cannot restore it, even if you have not asked it to!

18.11 Backup Strategies

Original work by Lowell Gilbert.

The first requirement in devising a backup plan is to make sure that all of the following problems are covered:

Disk failure
Accidental file deletion
Random file corruption
Complete machine destruction (e.g. fire), including destruction of any on-site backups.

It is perfectly possible that some systems will be best served by having each of these problems covered by a completely different technique. Except for strictly personal systems with very low-value data, it is unlikely that one technique would cover all of them.

Some of the techniques in the toolbox are:

Archives of the whole system, backed up onto permanent media offsite. This actually provides protection against all of the possible problems listed above, but is slow and inconvenient to restore from. You can keep copies of the backups onsite and/or online, but there will still be inconveniences in restoring files, especially for non-privileged users.
Filesystem snapshots. This is really only helpful in the accidental file deletion scenario, but it can be very helpful in that case, and is quick and easy to deal with.
Copies of whole filesystems and/or disks (e.g. periodic rsync(1) of the whole machine). This is generally most useful in networks with unique requirements. For general protection against disk failure, it is usually inferior to RAID. For restoring accidentally deleted files, it can be comparable to UFS snapshots, but that depends on your preferences.
RAID. Minimizes or avoids downtime when a disk fails. At the expense of having to deal with disk failures more often (because you have more disks), albeit at a much lower urgency.
Checking fingerprints of files. The mtree(8) utility is very useful for this. Although it is not a backup technique, it helps guarantee that you will notice when you need to resort to your backups. This is particularly important for offline backups, and should be checked periodically.

It is quite easy to come up with even more techniques, many of them variations on the ones listed above. Specialized requirements will usually lead to specialized techniques (for example, backing up a live database usually requires a method particular to the database software as an intermediate step). The important thing is to know what dangers you want to protect against, and how you will handle each.

admin.jugamanusia.com

Archive

FreeBSD Handbook - Chapter 29 Network Servers

29.1 Synopsis

29.2 The inetd “Super-Server”

29.2.1 Overview

29.2.2 Settings

29.2.3 Command-Line Options

29.2.4 inetd.conf

29.2.5 Security

29.2.6 Miscellaneous

29.3 Network File System (NFS)

29.3.1 How NFS Works

29.3.2 Configuring NFS

29.3.3 Locking

29.3.4 Practical Uses

29.3.5 Automatic Mounts with amd

29.3.6 Problems Integrating with Other Systems

29.4 Network Information System (NIS/YP)

29.4.1 What Is It?

29.4.2 Terms/Processes You Should Know

29.4.3 How Does It Work?

29.4.3.1 Machine Types

29.4.4 Using NIS/YP

29.4.4.1 Planning

29.4.4.1.1 Choosing a NIS Domain Name

29.4.4.1.2 Physical Server Requirements

29.4.4.2 NIS Servers

29.4.4.2.1 Setting Up a NIS Master Server

29.4.4.2.2 Initializing the NIS Maps

29.4.4.2.3 Setting up a NIS Slave Server

29.4.4.3 NIS Clients

29.4.4.3.1 Setting Up a NIS Client

29.4.5 NIS Security

29.4.6 Barring Some Users from Logging On

29.4.7 Using Netgroups

29.4.8 Important Things to Remember

29.4.9 NIS v1 Compatibility

29.4.10 NIS Servers That Are Also NIS Clients

29.4.11 Password Formats

29.5 Automatic Network Configuration (DHCP)

29.5.1 What Is DHCP?

29.5.2 What This Section Covers

29.5.3 How It Works

29.5.4 FreeBSD Integration

29.5.5 Files

29.5.6 Further Reading

29.5.7 Installing and Configuring a DHCP Server

29.5.7.1 What This Section Covers

29.5.7.2 DHCP Server Installation

29.5.7.3 Configuring the DHCP Server

29.5.7.4 Files

29.6 Domain Name System (DNS)

29.6.1 Overview

29.6.2 Terminology

29.6.3 Reasons to Run a Name Server

29.6.4 How It Works

29.6.5 Starting BIND

29.6.6 Configuration Files

29.6.6.1 Using make-localhost

29.6.6.2 /etc/namedb/named.conf

29.6.6.3 Zone Files

29.6.7 Caching Name Server

29.6.8 Security

29.6.9 Further Reading

29.7 Apache HTTP Server

29.7.1 Overview

29.7.2 Configuration

29.7.3 Running Apache

29.7.4 Virtual Hosting

29.7.5 Apache Modules

29.7.5.1 mod_ssl

29.7.5.2 Language Bindings

29.7.6 Dynamic Websites

29.7.6.1 Django

29.7.6.2 Ruby on Rails

29.7.6.3 mod_perl

29.7.6.4 mod_php

29.8 File Transfer Protocol (FTP)

29.8.1 Overview

29.2.4 `inetd.conf`

29.6.6.1 Using `make-localhost`

29.6.6.2 `/etc/namedb/named.conf`

29.11 Remote Host Logging with `syslogd`

28.3.1 `/etc/mail/access`

28.3.2 `/etc/mail/aliases`

28.3.3 `/etc/mail/local-host-names`

28.3.4 `/etc/mail/sendmail.cf`

28.3.5 `/etc/mail/virtusertable`