0

nice to meet you.

Suddenly nothing is being output in /var/log/messages, cron, secure, etc... I have not restarted rsyslog or modified /etc/rsyslog.conf, so I do not know why. Even after rebooting, the output is still not output. There are also servers that are outputting normally.

If you look at /proc/*/fd, the one that outputs correctly is

l-wx------ 1 root root 64 Apr 16  2021 8 -> /var/log/secure
l-wx------ 1 root root 64 Apr 16  2021 7 -> /var/log/maillog
l-wx------ 1 root root 64 Apr 16  2021 6 -> /var/log/cron
lrwx------ 1 root root 64 Apr 16  2021 5 -> socket:[8296]
lrwx------ 1 root root 64 Apr 16  2021 4 -> socket:[8295]
lr-x------ 1 root root 64 Apr 16  2021 3 -> /proc/kmsg
l-wx------ 1 root root 64 Apr 16  2021 2 -> /var/log/messages
lrwx------ 1 root root 64 Apr 16  2021 1 -> [eventpoll]
lrwx------ 1 root root 64 Apr 16  2021 0 -> socket:[8297]

The one that is not output is

lr-x------ 1 root root 64 May  2 05:24 3 -> /proc/kmsg
lrwx------ 1 root root 64 May  2 05:24 1 -> socket:[122587394]
lrwx------ 1 root root 64 May  2 05:24 0 -> socket:[122587391]

The rsyslog.conf is the same for both servers.

# rsyslog v5 configuration file

For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html

If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

MODULES

$ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability

Provides UDP syslog reception

#$ModLoad imudp #$UDPServerRun 514

Provides TCP syslog reception

#$ModLoad imtcp #$InputTCPServerRun 514

GLOBAL DIRECTIVES

Use default timestamp format

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

File syncing capability is disabled by default. This feature is usually not required,

not useful and an extreme performance hit

#$ActionFileEnableSync on

Include all config files in /etc/rsyslog.d/

$IncludeConfig /etc/rsyslog.d/*.conf

RULES

Log all kernel messages to the console.

Logging much else clutters up the screen.

#kern.* /dev/console

Log anything (except mail) of level info or higher.

Don't log private authentication messages!

*.info;mail.none;authpriv.none;cron.none /var/log/messages

The authpriv file has restricted access.

authpriv.* /var/log/secure

Log all the mail messages in one place.

mail.* -/var/log/maillog

Log cron stuff

cron.* /var/log/cron

Everybody gets emergency messages

.emerg

Save news errors of level crit and higher in a special file.

uucp,news.crit /var/log/spooler

Save boot messages also to boot.log

local7.* /var/log/boot.log

### begin forwarding rule

The statement between the begin ... end define a SINGLE forwarding

rule. They belong together, do NOT split them. If you create multiple

forwarding rules, duplicate the whole block!

Remote Logging (we use TCP for reliable delivery)

An on-disk queue is created for this action. If the remote host is

down, messages are spooled to disk and sent when it is up again.

#$WorkDirectory /var/lib/rsyslog # where to place spool files #$ActionQueueFileName fwdRule1 # unique name prefix for spool files #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown #$ActionQueueType LinkedList # run asynchronously #$ActionResumeRetryCount -1 # infinite retries if host is down

remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional

#. @@remote-host:514

We are unclear as to why the output has suddenly stopped.

OS: CentOS6 rsyslog: 5.8.10

rihm
  • 1
  • All your actual rsyslog configuration is in /etc/rsyslog.d/*.conf files. Look in there and specifically look for & stop statements, anything that happens to be below this statement (or to be included after the file with this statement) would be unreachable. It is likely that some updates introduced this change for some specific config file in there and now your main configuration isn't reachable anymore. – Peter Zhabin May 03 '22 at 08:10
  • Thank you. However, I checked under /etc/rsyslog.d/ and there are no files that have been recently modified. – rihm May 04 '22 at 21:51
  • The most common thing if it suddenly "stops working" is, that you have no disk space left. But it could also be an ownership problem. Rsyslog starts running as root, but after some time it drops privileges and runs as syslog, which may not have the rights needed (See: this answer). – eDonkey May 11 '22 at 07:44
  • This issue has been resolved. The reason was that there was a setting in a file under /etc/rsyslog.d to forward to another server, but The reason seemed to be that there was a setting in the file /etc/rsyslog.d to forward the logs to another server, but the destination became unknown because the target server was stopped. By deleting this configuration file and restarting the server, the logs are now being output. – rihm May 17 '22 at 03:56

0 Answers0