大网数据广告活动
服务器运维问答

How to forward Docker container logs to ELK?

· 52CE 我爱测

I would like to know what is the easiest way to forward my docker container logs to an ELK server, so far the solutions I have tried after having searched the internet didn't work at all. Basically I have a docker image that I ru…

问题原文

原帖链接 · ndarkness · CC BY-SA 3.0

I would like to know what is the easiest way to forward my docker container logs to an ELK server, so far the solutions I have tried after having searched the internet didn't work at all.

Basically I have a docker image that I run using docker-compose, this container does not log anything locally (it is composed of different services but none of them are logstash or whatever) but I see logging through docker logs -tf imageName or docker-compose logs. Since I am starting the containers with compose I cannot make use (or at least I don't know how) of the --logs-driver option of docker.

Thus I was wondering if someone may enlighten me a bit regarding how to forward that logs to an ELK container for example.

Thanks in advance,

Regards

SOLUTION:

Thanks to madeddie I could achieve to solve my issue in the following way, mention that I used the basic ELK-stack-in-containers which madeddie suggested in his post.

First I update the docker-compose.yml file of my container to add entries for the logging reference as madeddie told me,I included an entry per service, a snippet of my docker-compose looks like this

   version: '2'
    services:
      mosquitto:
        image: ansi/mosquitto
        ports:
          - "1883:1883" # Public access to MQTT
          - "12202:12202"  #mapping logs
        logging: 
           driver: gelf
           options: 
             gelf-address: udp://localhost:12202
    redis:
     image: redis
     command: redis-server --appendonly yes
     ports:
       - "6379:6379" # No public access to Redis
       - "12203:12203" #mapping logs
     volumes:
       - /home/dockeruser/redis-data:/data
     logging: 
       driver: gelf
       options: 
         gelf-address: udp://localhost:12203 

Secondly, I had to use a different port number per sevice in order to be able to forward the logs.

Finally,I updated my elk container docker-compose.yml file to map each of the upd port where I was sending my logs to the one that logstash listens to

logstash:
  build: logstash/
  command: logstash -f /etc/logstash/conf.d/logstash.conf
  volumes:
    - ./logstash/config:/etc/logstash/conf.d
  ports:
    - "5000:5000"
    - "12202:12201/udp" #mapping mosquitto logs
    - "12203:12201/udp" #mapping redis logs

This configuration and adding the entry of gelf {} in logstash.conf made it work, it is important as well to set up properly the IP address of the docker service.

REgards!

已采纳解答

原帖链接 · madeddie · CC BY-SA 3.0

Docker compose has the logging keyword: source

logging: 
  driver: syslog
  options: 
    syslog-address: "tcp://192.168.0.42:123"

So if you know where to go from there, go for it.

If not, I can advice you to look into the gelf logging driver for docker and the logstash gelf input plugin

If you would for instance use this basic ELK-stack-in-containers setup, you would update the docker-compose file and add port - "12201:12201/udp" to logstash.

Edit the logstash.conf input section to:

input {
    tcp {
        port => 5000
    }
    gelf {
    }

}

Then configure your containers to use logging driver gelf (not syslog) and the option gelf-address=udp://ip_of_logstash:12201 (instead of syslog-address).

The only magic you will have to take care of is how Docker will find the IP address or hostname of the Logstash container. You could solve that through docker-compose naming, Docker links or just manually.

Docker and ELK are powerful, flexible, but therefore also big and complex beasts. Prepare to put in some serious time, in reading and experimentation.

Don't be afraid to open new (and preferably very specific) questions you come across while exploring all this.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2016-08-11)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 3.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。