I am trying to start a Filebeat in container to send logs to the ELK cluster.
This is a standalone container only running Filebeat. Main application is running separately and Filebeat is monitoring the log through a shared volume.
This is my Dockerfile:
FROM centos:8
RUN curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-7.15.1-x86_64.rpm
RUN rpm -vi filebeat-oss-7.15.1-x86_64.rpm
RUN yum install -y initscripts
RUN curl -L https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_amd64 --output /usr/local/bin/dumb-init
RUN chmod +x /usr/local/bin/dumb-init
COPY filebeat.yml /etc/filebeat/filebeat.yml
COPY . /etc/filebeat/
ENTRYPOINT ["/usr/local/bin/dumb-init", "--"]
CMD ["/etc/filebeat/scripts/startup.sh"]
And the startup.sh script:
#!/bin/bash
set -e
echo "Starting Filebeat..."
/etc/init.d/filebeat start
When I start the container in interactive mode and run the startup script, everything is working fine and I can see the logs in ELK.
However, if I just run the container, it starts fine but immediately terminates after completing the startup script:
Starting Filebeat...
Starting filebeat: 2021-10-25T09:42:35.514Z INFO instance/beat.go:665 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2021-10-25T09:42:35.514Z DEBUG [beat] instance/beat.go:723 Beat metadata path: /var/lib/filebeat/meta.json
2021-10-25T09:42:35.517Z INFO instance/beat.go:673 Beat ID: 11cc47eb-cc1f-497e-89c2-633d5fc96b85
2021-10-25T09:42:35.518Z DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:128 add_cloud_metadata: starting to fetch metadata, timeout=3s
2021-10-25T09:42:35.521Z DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:165 add_cloud_metadata: received disposition for openstack after 1.9111ms. result=[provider:openstack, error=failed requesting openstack metadata: Get "https://169.254.169.254/2009-04-04/meta-data/instance-id": EOF, metadata={}]
2021-10-25T09:42:38.520Z DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:172 add_cloud_metadata: timed-out waiting for all responses
2021-10-25T09:42:38.521Z DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:131 add_cloud_metadata: fetchMetadata ran for 3.0027159s
2021-10-25T09:42:38.521Z INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:101 add_cloud_metadata: hosting provider type not detected.
2021-10-25T09:42:38.521Z DEBUG [processors] processors/processor.go:120 Generated new processors: add_cloud_metadata={}
2021-10-25T09:42:38.521Z INFO [beat] instance/beat.go:1014 Beat info {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "11cc47eb-cc1f-497e-89c2-633d5fc96b85"}}}
2021-10-25T09:42:38.521Z INFO [beat] instance/beat.go:1023 Build info {"system_info": {"build": {"commit": "5ae799cb1c3c490c9a27b14cb463dc23696bc7d3", "libbeat": "7.15.1", "time": "2021-10-07T20:57:21.000Z", "version": "7.15.1"}}}
2021-10-25T09:42:38.521Z INFO [beat] instance/beat.go:1026 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":6,"version":"go1.16.6"}}}
2021-10-25T09:42:38.523Z INFO [beat] instance/beat.go:1030 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-10-19T19:10:29Z","containerized":true,"name":"12c212ae1671","ip":["127.0.0.1/8","172.17.0.2/16"],"kernel_version":"5.10.25-linuxkit","mac":["02:42:ac:11:00:02"],"os":{"type":"linux","family":"redhat","platform":"centos","name":"CentOS Linux","version":"8","major":8,"minor":4,"patch":2105},"timezone":"UTC","timezone_offset_sec":0}}}
2021-10-25T09:42:38.523Z INFO [beat] instance/beat.go:1059 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 19, "ppid": 18, "seccomp": {"mode":"filter","no_new_privs":false}, "start_time": "2021-10-25T09:42:35.000Z"}}}
2021-10-25T09:42:38.523Z INFO instance/beat.go:309 Setup Beat: filebeat; Version: 7.15.1
2021-10-25T09:42:38.524Z DEBUG [beat] instance/beat.go:335 Initializing output ...
2021-10-25T09:42:38.524Z DEBUG [publisher] pipeline/consumer.go:148 start pipeline event consumer
2021-10-25T09:42:38.524Z INFO [publisher] pipeline/module.go:113 Beat name: 12c212ae1671
Config OK
[ OK ]
So, after starting Filebeat, I need a way to keep the container running. What is the proper way to keep a container like this running used for log monitoring?
Thank you.
