The Missing Logs

Step 1 — Understand the symptoms

"The application is generating logs" only confirms the very first link in a much longer chain. A log line's actual journey looks like this:

Application
    ↓
Container / Runtime (stdout/stderr)
    ↓
Log Collector (node-level agent)
    ↓
Forwarder / Sink
    ↓
Central Logging Platform
    ↓
Search / Dashboard

Confirming logs exist at the very first step tells you nothing about the other five. A total, hours-long absence in the central platform — not a reduction, a complete absence — points at a single stage somewhere in that chain that's either stopped working entirely or is silently dropping everything for this one service.

Step 2 — Identify the likely bottleneck

A complete, total loss (not degraded, not partial) for one specific service, while other services log normally, narrows this considerably. If it were a central-platform-wide issue, every service would be affected, not just one — that points the investigation toward something specific to this service's pipeline path: a collector configuration issue for this node or namespace, a filter or routing rule that's silently excluding this service, a permissions issue between the forwarder and the central platform for this specific log stream, or a recent change to this service's logging configuration or labels that broke how it's being picked up.

Step 3 — Investigation

Walk the pipeline stage by stage rather than guessing:

• Application / container — already confirmed logs exist here (kubectl logs or exec into the pod).
• Log collector — is the node-level agent actually running and healthy on the node this pod is scheduled on? Check its own logs and metrics for errors specific to this pod or namespace.
• Filtering and routing rules — many log pipelines have include/exclude rules by namespace, label, or log level. A recent change to this service's labels or a broad exclusion rule elsewhere in the pipeline is a common, easy-to-miss cause.
• Forwarder / sink — is it successfully shipping data, and does it show any errors or backpressure specific to this service's stream?
• Permissions and quotas — a forwarder that's lost write access to the destination, or hit a per-service ingestion limit, can silently drop logs rather than erroring loudly.
• Central platform ingestion — confirm the index or destination this service's logs should land in actually exists and is receiving data from anything, to rule out a platform-side ingestion problem scoped to that destination.

Step 4 — Recommended action

Fix the specific stage the investigation points to, not the whole pipeline speculatively. If it's a filtering rule, correct the rule and add a check that flags when a previously-logging service goes silent for a configurable window — for a total log loss like this, an automated dead man's switch on log volume per service would have caught it in minutes instead of it being found incidentally days later.

Don't restart the application or "just redeploy it" as a first move — the application was already confirmed to be writing logs correctly. Restarting it addresses a stage of the pipeline that was never actually broken, and it won't fix a collector, forwarder, or routing issue.

Step 5 — Engineering lesson

"The application is generating logs" and "the logs are searchable in the central platform" are claims about two completely different systems, separated by several hops that can each fail independently and silently. A missing-logs incident is a pipeline-health problem first, and an application problem only if every other stage checks out — troubleshooting from the wrong end of that chain wastes time confirming something that was never actually in question.