--- title: "Kubernetes CronJob monitoring and alerts | Logdash" description: "Alert when a Kubernetes CronJob fails or stops: a 10-second watcher on lastSuccessfulTime with Telegram alerts, or the kube-state-metrics rule if you run Prometheus." url: https://logdash.io/cron-monitoring/kubernetes-cronjob --- # Kubernetes CronJob monitoring Kubernetes records when each CronJob last succeeded in status.lastSuccessfulTime and never alerts on it, so either alert on that timestamp in Prometheus through kube-state-metrics, or run a small watcher that pings an external monitor every 10 seconds while it is fresh. A CronJob can fail quietly in more ways than a crontab line. The pod exits non-zero until the Job hits its backoffLimit. The image tag was deleted and the pod sits in ImagePullBackOff. Someone set suspend to true during an incident and forgot. The controller was down past startingDeadlineSeconds and skipped the run. kubectl get cronjob shows a LAST SCHEDULE column, and nothing pages anyone. One field covers all of these: `status.lastSuccessfulTime`, which the CronJob controller sets when a Job completes. Every failure above stops it moving. So monitoring a CronJob means alerting when that timestamp is older than the schedule plus some slack. ## Kubernetes CronJob monitoring with Prometheus If kube-state-metrics already runs in the cluster, it is two rules. `time() - kube_cronjob_status_last_successful_time > 90000` fires when a job has not succeeded in 25 hours, and `kube_job_status_failed > 0` catches each failed Job, with a reason label. Alertmanager routes both. For a team that already operates that stack, this is the answer. For a team that does not, it is three components to run, and they go down with the cluster they are meant to watch. ## Kubernetes CronJob failed alert without Prometheus The Logdash version is one Deployment that reads lastSuccessfulTime every 10 seconds and posts to a push monitor while it is fresh. Push monitors are on Pro, $15 a month, and Pro checks each one every 15 seconds: no ping in that window, the monitor goes down. That is why the CronJob does not call Logdash itself. A curl at the end of a nightly job is one ping a day against a 15-second window. Read-only access to CronJobs ```bash kubectl create serviceaccount cron-watch kubectl create role cron-watch --verb=get --resource=cronjobs kubectl create rolebinding cron-watch --role=cron-watch --serviceaccount=default:cron-watch ``` cron-watch.yaml ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: cron-watch spec: replicas: 1 selector: matchLabels: { app: cron-watch } template: metadata: labels: { app: cron-watch } spec: serviceAccountName: cron-watch containers: - name: watch image: alpine/k8s:1.36.5 env: - { name: JOB, value: nightly-report } - { name: MAX_AGE, value: "90000" } # 25 hours in seconds - { name: PING, value: https://api.logdash.io/ping/68b4c1f0e3a2d5c7b9f01234 } command: ["/bin/sh", "-c"] args: - | while true; do kubectl get cronjob "$JOB" -o json \ | jq -e "now - (.status.lastSuccessfulTime | fromdateiso8601) < $MAX_AGE" >/dev/null \ && curl -fsS -m 5 -X POST "$PING" sleep 10 done ``` The alpine/k8s image ships kubectl, jq and curl. jq -e exits non-zero when the timestamp is stale or missing, so a CronJob that has never succeeded sends nothing. Set MAX\_AGE to the schedule interval plus the lateness you accept. Because the watcher runs in the cluster, a dead cluster or a broken network goes quiet too, and that reaches you as well. 1. **Create a push monitor** Add a service in Logdash on Pro, set the monitor to push and copy the monitor id from the ping URL. Name it after the CronJob, since the alert shows the name. 2. **Apply the watcher** Run the three kubectl lines, put the CronJob name, window and monitor id into the env block and apply the file. If the job succeeded inside the window, the monitor goes green within 15 seconds. 3. **Break it on purpose** Scale the watcher to zero replicas. Within 30 seconds a Telegram message lands saying the monitor is down, status code 0, did not receive call for this time range. Scale it back and the up message follows. ### Logdash vs Prometheus for CronJobs | Feature | Logdash | Prometheus and kube-state-metrics | | --------------------------- | ---------------------------------------- | ----------------------------------------------- | | Setup if you already run it | RBAC, a Deployment and a monitor per job | Two alert rules | | Setup from zero | One Deployment, alerting lives outside | kube-state-metrics, Prometheus and Alertmanager | | New CronJobs | One more watcher and monitor each | Covered by the same rule | | Why it failed | Only that the job went stale | Failure reason as a label | | Whole cluster down | Alert within 30 seconds | Silent unless you add an outside check | | Cost | Pro, $15 a month | Open source, you run and store it | ### When Prometheus is the better pick - You already run Prometheus with kube-state-metrics. Two rules beat a new Deployment. - You have dozens of CronJobs. One rule covers all of them, including the ones added next month. - You want to know why a Job failed. The reason label separates BackoffLimitExceeded from DeadlineExceeded. - Alerts have to reach Slack or PagerDuty. Logdash sends Telegram and webhooks only. ### Which metrics does Kubernetes CronJob monitoring use in Prometheus? kube\_cronjob\_status\_last\_successful\_time for staleness and kube\_job\_status\_failed for failed runs, both from kube-state-metrics. Alert when time() minus the first exceeds the schedule interval plus slack. ### How do I get a Kubernetes CronJob failed alert? Watch status.lastSuccessfulTime, not pod events. A failed, suspended or never-scheduled job stops it moving. Alert on it in Prometheus, or ping a Logdash push monitor while it is fresh and get a Telegram alert when it stops. ### What is the simplest kube cronjob alerting setup? Without Prometheus, one Deployment running kubectl, jq and curl in a loop, plus a read-only role. With Prometheus, two rules on kube-state-metrics. Both alert on a stale lastSuccessfulTime. ### Does Kubernetes CronJob monitoring need a Pro plan in Logdash? Yes. Push monitors are a Pro feature at $15 a month, which covers 50 services with one monitor each. HTTP uptime checks are on every plan, including the free one. ## Keep reading [All cron monitoring guides](https://logdash.io/cron-monitoring): One guide per scheduler, each with the code that proves a job ran and the alert for when it did not. [GitHub Actions monitoring](https://logdash.io/cron-monitoring/github-actions): GitHub shows you the runs that failed but never the runs that did not start, so watch a scheduled workflow from outside: the last step reports success, and a monitor alerts you when that report goes stale. [Windows Task Scheduler monitoring](https://logdash.io/cron-monitoring/windows-task-scheduler): Task Scheduler keeps each task's last run time and last run result but never alerts on them, so monitor it with a small PowerShell watcher that pings an external monitor every 10 seconds while the last run is recent and clean. [Scheduled task monitoring](https://logdash.io/cron-monitoring/scheduled-tasks): Scheduled task monitoring means the task leaves proof after every clean run, whatever scheduler starts it, and a service outside the box alerts you when that proof gets older than the schedule allows. [Uptime monitoring](https://logdash.io/features/monitoring): HTTP checks as often as every 15 seconds. When one fails, Telegram tells you before your users do.