FluentBit Examples

FluentBit is a lightweight and high-performance log processor and forwarder. These examples demonstrate various FluentBit deployment scenarios, from basic setups to advanced configurations with custom processing.

Simple Deployment

Basic FluentBit configuration for standard log collection:

fluentbit-simple-values.yaml

High Availability with Aggregator

FluentBit deployment with aggregator for improved reliability and load distribution:

fluentbit-with-aggragator-values.yaml

Custom Lua Script Processing

Advanced FluentBit configuration with custom Lua script for specialized log processing:

fluentbit-custom-lua-script-values.yaml

# This is a YAML-formatted file.
# All parameters specify only as example

fluentbit:
  install: true
  systemLogType: varlogsyslog
  containerLogging: true

  graylogOutput: true
  graylogHost: 1.2.3.4
  graylogPort: 12201

  customLuaScriptConf:
    # In FluentBit config will be create the Lua script with the name "convert_to_utc.lua"
    convert_to_utc.lua: |
      function convert_to_utc(tag, timestamp, record)
          local date_time = record["pub_date"]
          local new_record = record
          if date_time then
              if string.find(date_time, ",") then
                  local pattern = "(%a+, %d+ %a+ %d+ %d+:%d+:%d+) ([+-]%d%d%d%d)"
                  local date_part, zone_part = date_time:match(pattern)
                  if date_part and zone_part then
                      local command = string.format("date -u -d '%s %s' +%%Y-%%m-%%dT%%H:%%M:%%SZ", date_part, zone_part)
                      local handle = io.popen(command)
                      local result = handle:read("*a")
                      handle:close()
                      new_record["pub_date"] = result:match("%S+")
                  end
              end
          end
          return 1, timestamp, new_record
      end

fluentd:
  install: false

This configuration demonstrates:

• Custom Lua script for date/time conversion to UTC
• Graylog output configuration
• Custom log processing logic
• Integration with external commands for date parsing

High Availability with Custom Lua Scripts

Combined high availability and custom processing configuration:

fluentbit-ha-custom-lua-script-values.yaml

# This is a YAML-formatted file.
# All parameters specify only as example

fluentbit:
  install: true
  systemLogType: varlogsyslog
  containerLogging: true

  customLuaScriptConf:
    split.lua: |
      function cb_split(tag, timestamp, record)
        if record["x"] ~= nil then
          return 2, timestamp, record["x"]
        else
          return 2, timestamp, record
        end
      end

  aggregator:
    graylogOutput: true
    graylogHost: 1.2.3.4
    graylogPort: 12201

    customLuaScriptConf:
      # In FluentBit config will be create the Lua script with the name "convert_to_utc.lua"
      convert_to_utc.lua: |
        function convert_to_utc(tag, timestamp, record)
            local date_time = record["pub_date"]
            local new_record = record
            if date_time then
                if string.find(date_time, ",") then
                    local pattern = "(%a+, %d+ %a+ %d+ %d+:%d+:%d+) ([+-]%d%d%d%d)"
                    local date_part, zone_part = date_time:match(pattern)
                    if date_part and zone_part then
                        local command = string.format("date -u -d '%s %s' +%%Y-%%m-%%dT%%H:%%M:%%SZ", date_part, zone_part)
                        local handle = io.popen(command)
                        local result = handle:read("*a")
                        handle:close()
                        new_record["pub_date"] = result:match("%S+")
                    end
                end
            end
            return 1, timestamp, new_record
        end

fluentd:
  install: false

This advanced configuration provides:

• High availability setup with multiple instances
• Custom Lua scripts for log transformation
• Enhanced reliability and processing capabilities
• Scalable log processing pipeline

Key Configuration Parameters

Parameter	Description	Use Case
fluentbit.install	Enable FluentBit deployment	All scenarios
fluentbit.systemLogType	System log source type	varlogsyslog, journald
fluentbit.containerLogging	Enable container log collection	Container environments
fluentbit.graylogOutput	Enable Graylog output	Graylog integration
fluentbit.customLuaScriptConf	Custom Lua processing scripts	Advanced log processing

Use Cases

• Simple Deployment: Basic log collection and forwarding
• High Availability: Mission-critical environments requiring redundancy
• Custom Processing: Complex log transformation requirements
• Hybrid Configurations: Combining HA with custom processing logic