Hey guys, let's dive into setting up rsyslog with TLS! This guide provides a practical example to get your logging server secured and your data protected. We'll cover everything from generating certificates to configuring both the server and client. This is crucial for anyone who wants to ensure their log data is transmitted securely, preventing eavesdropping and tampering. Let's face it, in today's world of security threats, using plain text to transfer logs is a big no-no. This configuration uses authentication and encryption to secure your log streams.

Why Use TLS with rsyslog?

So, why bother with TLS (Transport Layer Security) when it comes to rsyslog? Well, the main reason is security. When you're sending log messages across a network, you want to make sure they're not intercepted and read by unauthorized parties. TLS does this by:

• Encryption: It encrypts the log data, making it unreadable to anyone who doesn't have the decryption key. Think of it like putting your logs in a locked box.
• Authentication: It verifies the identities of both the server and the client. This ensures that the log messages are being sent to the correct destination and that you're receiving logs from a trusted source. This is achieved using certificates.
• Data Integrity: TLS helps ensure the integrity of the data, meaning it hasn't been tampered with during transit. If someone tries to modify the logs, the TLS connection will detect it.

Without TLS, your log data is vulnerable. Anyone on the network could potentially read it, or even worse, modify it to cover their tracks. Using TLS adds a layer of security that is essential in a modern logging setup. Consider it an investment in your server's and data's security.

Prerequisites

Before we start, you'll need a few things set up:

1. A server and client with rsyslog installed: Make sure you have rsyslog installed on both the server (where you'll be receiving logs) and the client (where you'll be sending logs). You can typically install it using your system's package manager (e.g., apt install rsyslog on Debian/Ubuntu or yum install rsyslog on CentOS/RHEL).
2. OpenSSL: This is a command-line tool used to create and manage certificates and keys. Most Linux distributions have it pre-installed. If not, install it using your package manager.
3. Basic understanding of Linux: You should be comfortable navigating the command line and editing configuration files.
4. Root or sudo privileges: You'll need these to install packages, edit configuration files, and restart services.

Generating Certificates and Keys

Certificates are at the heart of TLS. They act like digital IDs, verifying the identity of the server and the client. Let's generate the certificates and keys we need. We'll use OpenSSL for this. The process involves creating a Certificate Authority (CA), generating server and client certificates, and exchanging them. The CA signs the certificates, vouching for their authenticity. This is like a notary public for your server and client.

Create a Certificate Authority (CA)

First, we'll create a CA. This is a self-signed certificate that we'll use to sign the server and client certificates. Think of the CA as the root of trust.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ca.key -out ca.crt

This command does the following:

• openssl req: Calls the OpenSSL certificate request tool.
• -x509: Creates a self-signed certificate.
• -nodes: Skips the passphrase prompt for the private key (for simplicity in this example; in a production environment, you should use a passphrase).
• -days 365: Sets the validity period to 365 days.
• -newkey rsa:2048: Generates a new RSA private key of 2048 bits.
• -keyout ca.key: Specifies the output file for the CA private key.
• -out ca.crt: Specifies the output file for the CA certificate.

You'll be prompted for information like country, state, and organization. Fill these in as appropriate. The most important is the common name, which can be anything, such as "My CA".

Generate the Server Certificate and Key

Next, let's create the server certificate and key. This is the certificate that the rsyslog server will use to identify itself to clients.

openssl req -nodes -newkey rsa:2048 -keyout server.key -out server.csr
openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt

Let's break this down:

• The first command (openssl req) generates a certificate signing request (CSR) and a private key (server.key).
• The second command (openssl x509) uses the CSR and the CA to create the signed server certificate (server.crt). The -CA ca.crt and -CAkey ca.key options specify the CA's certificate and private key, respectively. The -set_serial 01 option sets the serial number of the certificate.

You'll be prompted for information again. Make sure the common name (CN) matches the hostname or IP address of your rsyslog server. Otherwise, the client will get a warning that the certificate doesn't match the server.

Generate the Client Certificate and Key

Now, let's generate the client certificate and key. This is what the client will use to authenticate itself to the server.

openssl req -nodes -newkey rsa:2048 -keyout client.key -out client.csr
openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client.crt

The process is the same as for the server certificate, but this time, the client's common name should be something descriptive, such as "Client-1". The serial number is incremented to "02".

Distribute the Certificates

You'll need to distribute the necessary certificates to the appropriate machines:

• Server: You'll need ca.crt, server.crt, and server.key on the server. Also, you will need the ca.crt on the server so that it can verify the client certificate.
• Client: You'll need ca.crt, client.crt, and client.key on the client. Additionally, the client needs the CA's certificate (ca.crt) to trust the server.

Use scp, sftp, or a similar secure method to copy these files to the correct locations on your server and client machines. Be careful to protect the private keys (.key files), as these are critical for the security of the system.

Configuring rsyslog Server

Now, let's configure the rsyslog server to listen for TLS connections. We'll modify the rsyslog configuration file (/etc/rsyslog.conf or a file in /etc/rsyslog.d/). This will tell rsyslog to use the certificates and keys we generated earlier.

# /etc/rsyslog.conf

# Load the TLS module
module(load="/usr/lib/rsyslog/imtcp.so")

# Create a template for the logs
tmpl.log.format="%timestamp:::date-rfc3339% %HOSTNAME% %syslogtag%%msg%\n"

# Configure a TCP listener with TLS
input(type="imtcp" port="6514"  # Use a non-standard port for TLS
      tls="on"
      ca.crt="/etc/rsyslog/ca.crt"
      cert="/etc/rsyslog/server.crt"
      key="/etc/rsyslog/server.key"
      requireauth="on") # Require client authentication

# Define the output (e.g., to a local file)
*.* /var/log/secure_log.log;tmpl.log.format

# Restart rsyslog

Let's break down this configuration:

• module(load="/usr/lib/rsyslog/imtcp.so"): Loads the TCP input module, which is necessary for receiving logs over TCP. Replace /usr/lib/rsyslog/imtcp.so with the correct path to your module, which might be /lib/rsyslog/imtcp.so or similar.
• tmpl.log.format: Defines the format for the log messages. This is optional but can improve readability.
• input(type="imtcp" ...): Configures a TCP input. This section is key. We set the following options.
  • port="6514": Specifies the port to listen on. Choose a port that isn't already in use. You can use 514, but it is often used for other connections. The configuration example uses 6514 for TLS.
  • tls="on": Enables TLS encryption.
  • ca.crt="/etc/rsyslog/ca.crt": Specifies the path to the CA's certificate. The server uses this to verify the client's certificate.
  • cert="/etc/rsyslog/server.crt": Specifies the path to the server's certificate.
  • key="/etc/rsyslog/server.key": Specifies the path to the server's private key.
  • requireauth="on": Requires client authentication. Only clients with a certificate signed by the CA will be allowed to connect. This adds another layer of security.
• *.* /var/log/secure_log.log;tmpl.log.format: Defines the output. This logs all messages (*.*) to the file /var/log/secure_log.log using the format defined by tmpl.log.format. Make sure the file exists and has the correct permissions.

After making these changes, restart the rsyslog service on the server:

systemctl restart rsyslog

Check the rsyslog logs (usually in /var/log/syslog or /var/log/rsyslog.log) for any errors. If there are problems, the logs will provide valuable information. Remember to test and refine your configuration, such as setting up a CA to sign certificates, creating the server and client certificates, and configuring rsyslog to use TLS.

Configuring rsyslog Client

Now, let's configure the rsyslog client to send logs to the server using TLS. This involves modifying the client's rsyslog configuration file.

# /etc/rsyslog.conf

# Load the TLS module (if not already loaded)
module(load="/usr/lib/rsyslog/imtcp.so")

# Define the destination (server IP or hostname)
$DefaultNetstreamServerAddress <server_ip_or_hostname>
$DefaultNetstreamServerPort 6514  # Must match the server's port

# Configure TLS
$ActionSendStreamDriverMode 1   # Use TCP with TLS
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer  <server_hostname>

# Configure client certificate and key
$DefaultNetstreamDriverCAFile /etc/rsyslog/ca.crt
$DefaultNetstreamDriverCertFile /etc/rsyslog/client.crt
$DefaultNetstreamDriverKeyFile /etc/rsyslog/client.key

# Send all local messages to the server
*.* @@<server_ip_or_hostname>

# Restart rsyslog

Let's break this down as well:

• module(load="/usr/lib/rsyslog/imtcp.so"): As on the server, loads the TCP input module.
• $DefaultNetstreamServerAddress <server_ip_or_hostname>: Specifies the IP address or hostname of your rsyslog server. Replace <server_ip_or_hostname> with the actual address.
• $DefaultNetstreamServerPort 6514: Specifies the port to send logs to. This must match the port you configured on the server. Make sure it matches. In this example, it's 6514.
• $ActionSendStreamDriverMode 1: Enables TCP with TLS. The value 1 is for TLS encryption.
• $ActionSendStreamDriverAuthMode x509/name: This tells the client to authenticate the server using X.509 certificates and verify the common name (CN) of the server's certificate.
• $ActionSendStreamDriverPermittedPeer <server_hostname>: Specifies the expected hostname of the server. The client will verify that the server's certificate's CN matches this value. Replace <server_hostname> with the server's hostname. This adds another layer of security, verifying the server's identity.
• $DefaultNetstreamDriverCAFile /etc/rsyslog/ca.crt: Specifies the path to the CA's certificate. The client uses this to verify the server's certificate.
• $DefaultNetstreamDriverCertFile /etc/rsyslog/client.crt: Specifies the path to the client's certificate.
• $DefaultNetstreamDriverKeyFile /etc/rsyslog/client.key: Specifies the path to the client's private key.
• *.* @@<server_ip_or_hostname>: This line sends all log messages to the server over TLS. The @@ indicates TCP with TLS. Replace <server_ip_or_hostname> with the server's IP address or hostname.

After making these changes, restart the rsyslog service on the client:

systemctl restart rsyslog

Check the logs on both the client (usually /var/log/syslog or /var/log/rsyslog.log) and the server (in the file you specified, in this example, /var/log/secure_log.log) for any errors. If everything is configured correctly, you should see the log messages from the client appearing in the server's log file, transmitted securely over TLS. You can verify that the logs are being transmitted securely by examining the logs on the server side. They should contain information on the certificate authentication, confirming that the client is successfully authenticated, and that the data is encrypted. If any errors are present, they will usually give you clues on what the problem is.

Troubleshooting

Let's go over some common issues you might encounter and how to fix them:

1. Certificate Errors: The most common problems are related to certificates. Make sure the certificate paths are correct in both the server and client configurations. Also, ensure the common names (CN) in the server and client certificates match the hostnames or IP addresses. Finally, verify the ca.crt files are placed and correctly configured. An invalid certificate will make the system fail.
2. Firewall Issues: Make sure your firewall allows TCP traffic on the port you specified (in this example, 6514). Both the server and the client need to be able to communicate with each other on this port.
3. Permissions: Ensure that the rsyslog user has the necessary permissions to access the certificate and key files. Incorrect permissions can prevent rsyslog from reading the files. You may need to change the file owner and group. This could happen if the paths used are wrong.
4. Incorrect Configuration: Double-check your configuration files for typos or other errors. Even a small mistake can prevent TLS from working correctly. Sometimes, the syntax can be tricky, so make sure to double-check.
5. Logging Levels: Increase the rsyslog logging level to get more detailed error messages. This can help you pinpoint the cause of the problem. Look for errors related to TLS in your rsyslog logs.
6. Time Synchronization: Ensure your server and client clocks are synchronized. TLS relies on time accuracy, and significant clock skew can cause connection problems.

Best Practices for rsyslog TLS Configuration

Let's look at a few best practices for a rsyslog TLS setup:

• Use Strong Encryption: Always use strong encryption algorithms and key sizes when generating your certificates and keys. RSA keys of 2048 bits or higher are recommended.
• Protect Private Keys: Keep your private keys secure. Only grant access to authorized personnel. Secure the .key files as they are the most important part of the security setup.
• Regular Certificate Rotation: Rotate your certificates regularly. This reduces the risk if a key is compromised. It also helps to ensure the certificate is valid. Consider automating this process.
• Monitor Logs: Regularly monitor your rsyslog logs for errors or unusual activity. This can help you identify and address security issues promptly. Automated log analysis tools can be useful.
• Least Privilege: Grant the rsyslog user only the necessary permissions to access certificate and key files. Following the principle of least privilege reduces the impact of a potential security breach.
• Test Thoroughly: Test your TLS configuration thoroughly in a non-production environment before deploying it to production. Ensure all configurations are correctly in place. Conduct testing frequently to ensure the setup remains secure.

Conclusion

There you have it! A complete guide to setting up rsyslog with TLS. By following this example, you can significantly enhance the security of your logging infrastructure. Remember to adapt the configuration to your specific needs, such as modifying the paths to your certificates and adjusting the port numbers. This is a powerful step in securing your logging data and protecting your server and client communication. Implementing TLS will ensure that your log data is not only gathered efficiently but also securely.

By following these steps, you'll ensure that your log data is transmitted securely, preventing eavesdropping and tampering. So go ahead, implement TLS, and sleep soundly knowing your logs are protected! This is a core security practice that can easily enhance the security of your logging infrastructure. Keep your logging practices up to date, and stay secure! Remember to tailor the configuration to fit your particular needs. And remember to test everything thoroughly.