Docs / Log Ingestion / Kubernetes Pods

Kubernetes Log Ingestion

Integrate logs from containerized workloads running in Kubernetes clusters using deployment manifest sidecars or cluster-wide daemonsets.

1. Kubernetes Ingestion Patterns

Orchestrated workloads running in Kubernetes clusters can stream logs via two standard structures:

Sidecar Deployment (Recommended): Running a lightweight `sprint-log-shipper` container alongside the application container in the same pod. It reads log output using a shared `emptyDir` volume.
Node DaemonSet: Running a single agent pod on every cluster node. The shipper mounts the host path `/var/log/pods` directory to capture all container output.

2. Pod Sidecar Pattern

Configure a shared `emptyDir` volume in your Deployment file, mapping it to both the main application container (e.g. Nginx writing logs) and the log shipper container:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      volumes:
        # Define a shared temporary memory volume
        - name: shared-logs
          emptyDir: {}
        # Mount the shipper config using a Kubernetes ConfigMap
        - name: shipper-config
          configMap:
            name: log-shipper-config
      containers:
        # 1. Main Application Container
        - name: nginx
          image: nginx:alpine
          volumeMounts:
            - name: shared-logs
              mountPath: /var/log/nginx

        # 2. Log Shipper Sidecar Container
        - name: shipper
          image: python:3.10-alpine
          command: ["sh", "-c"]
          args:
            - |
              wget -qO /usr/local/bin/log_shipper.py https://sprint-logparser.dev.5starcompany.com.ng/download/shipper
              python /usr/local/bin/log_shipper.py /etc/log-shipper/config.json
          volumeMounts:
            - name: shared-logs
              mountPath: /var/log/app
            - name: shipper-config
              mountPath: /etc/log-shipper

Define the shipper ConfigMap manifest `configmap.yaml` containing the configuration settings:

apiVersion: v1
kind: ConfigMap
metadata:
  name: log-shipper-config
data:
  config.json: |
    {
      "api_url": "https://sprint-logparser.dev.5starcompany.com.ng/api/projects/ingest",
      "api_token": "YOUR_WORKSPACE_INGESTION_TOKEN",
      "log_file": "/var/log/app/access.log",
      "log_type": "nginx_access"
    }

3. DaemonSet Ingestion Pattern

For cluster-wide log collection where stdout/stderr of pods is automatically tracked by the container runtime, you can configure a DaemonSet mapping to host path `/var/log/pods`. This allows a single instance of the shipper per node to capture and forward all active pod stdout.