How Nova investigates
Nova works an incident the way an on-call engineer does: look first, draw on experience, test the read that best tells the possible causes apart, and let a person decide every change. The card's plan shows each step as it happens.
The steps
- Triage. Nova sets the incident's type, such as availability or performance, and its severity, and sizes the investigation as standard or deep. The card shows the size and its price before any work starts. Triage never chooses a script.
- First facts. Nova reads what changed first. Before forming any cause, it states whether the app is serving, since when, what changed last (the version and when it was deployed) and who is affected, each with its source. What it cannot tell is said as unknown. When a change lines up with the start of the incident, rolling it back is the first way to restore service Nova offers (see Fixes and mitigations).
- Past incidents. Nova looks for incidents like this one: this app's past cases first, then similar ones elsewhere in your organization. A cause Nova confirmed, with a fix that was checked healthy, counts most. If the same symptoms return after a fix, the card says the earlier fix did not hold, or which later deploy undid it. A past case can move a cause up the list; it is never evidence.
- Possible causes. Nova draws on built-in knowledge of common failures. It is matched against the alert or question, and again once the first facts are known, and lends the failure modes worth testing. A generic set of causes always applies. It is knowledge, not a script: Nova chooses every read itself.
- Reads that tell causes apart. Each round, Nova picks the read most likely to separate the open causes, and judges every result against every cause: supports, contradicts, or says nothing about it. A standard investigation makes one read a round; a deep one reads up to three open causes at once. Reads from different skills run side by side, and a skill that asks Nova to slow down holds up only its own reads. A read that needs approval is asked for in place while other reads continue.
- A challenge. Before calling a cause found, Nova looks for anything the evidence leaves unexplained, a missing link, or a rival cause it has not ruled out. An open question becomes the next read, and the cause stays Likely until it is answered. If what is left is something no connected source can read, the cause stands as found, the card names what was not checked, and Nova may ask you to confirm it: Yes, it's fine, No (Nova investigates again with that rival open) or Don't know.
- Conclusion. One of three answers.
The three answers
| Answer | What it means |
|---|---|
| Cause found | Observed evidence supports this cause and no other open cause, and the challenge found no gap. |
| Likely cause | The evidence points to this cause but does not set it apart, or a read that would confirm it was declined or unavailable. The card says what would confirm it. |
| No cause found | Nothing Nova could read pointed to a cause. The card shows what it checked. |
Every answer lists what Nova could not check and why: withheld, blocked by policy, or unavailable. A gap is not evidence that nothing is wrong there. Running out of steps or time never upgrades an answer: a cause is Cause found only if what Nova read before it stopped confirms it.
Built-in failure knowledge
Nova knows how common Kubernetes app failures show themselves, and what rules each one out. It tests them when the incident matches:
| Failure | What Nova looks for |
|---|---|
| A process that exits cleanly | A container that finishes and exits with code 0 (Completed), so it is restarted in a loop |
| An error at start | A non-zero exit code with the app's own error, or a command that does not exist in the image |
| Out of memory | A container killed at its memory limit (OOMKilled, exit code 137) |
| Image pull | An image reference that is wrong or cannot be pulled |
| Probes | A liveness or readiness probe that checks a path or port the app does not serve |
| Missing configuration | A secret or config map the app references that does not exist, a wrong port, or a non-root container with no writable path |
| A dependency that is down | A database, queue or other service the app needs at start or on every request |
| A regression after a deploy | A new version, a changed setting or an unfinished rollout just before the failure began |
| Nothing serving | No ready pods, an app scaled to zero or paused, or a cluster the platform cannot reach |
| Node pressure and scheduling | Pods that stay pending because no node has room or matches their constraints, or are evicted under memory or disk pressure |
Outside these, Nova still investigates with a generic set of causes.
What Nova found in practice
Typical incidents, and the evidence that decides each one:
| Incident | Answer | What decided it |
|---|---|---|
| An nginx app restarting in a loop | Cause found: the container runs as non-root and cannot write its cache path | The log line chown("/var/cache/nginx/client_temp", 101) failed (1: Operation not permitted), which also ruled out a probe, a missing secret and a bad image |
| A job deployed as a service, restarting | Cause found: a one-shot command exits with code 0 and is restarted | App status reason Completed, exit code 0, the --once command in the configuration, and the deploy that set it |
| An app restarting with no recent deploy | Cause found: killed at its 512Mi memory limit | App status reason OOMKilled, exit code 137; a deploy six days earlier and a probe (exit 143) ruled out |
| An app failing after a deploy, logs declined | Cause found, without logs: the deploy set a model runtime the cluster does not have | The platform's own status and the configuration; the declined read listed as not read |
| A certificate not renewing | Cause found on the add-on's own cluster: a CPU request no node can fit | 0/3 nodes are available: 3 Insufficient cpu and the add-on's history, revision 4 |
| An error-rate alert on a healthy app | No cause found, with no fix offered | Every read quiet (3/3 replicas ready); the card names the monitoring skill that would show more |
An add-on incident is read on the cluster the add-on runs on: the alert's cluster label says where, and without it Nova lists your clusters and their add-ons to find it.
What counts as evidence
- Only what a read returned. An empty or missing result supports and contradicts nothing.
- Quoted, with source and time. Each piece of evidence is copied from what the read returned, and must be about the app under investigation.
- Support every cause shares counts for none. A symptom common to all the possible causes does not favour any of them.
- What a person says counts, up to a point. Your answer to Nova's question is weighed like a read, but on its own it can make a cause Likely at most.
- No percentages. A number would suggest a precision the evidence does not have. The three answers are what Nova can stand behind.
Where Nova looks
Nova's first reads come from AstroPulse: app status, configuration, events, platform events and deploy history. After that it can also use read tools from your other connected skills, including your own MCP servers, when they help tell causes apart. Only tools your guardrails classify as reads are used while investigating. A tool on your own MCP server asks a person before every call until an admin trusts that server (see Configure). With GitHub connected, Nova can link a deploy to the commit and pull request it was built from.
Standard or deep
Nova sizes each investigation at triage, from its severity, the causes still open after the first reads, and whether those reads already show the failure. The card says the size and price before any work starts.
| Standard | Deep | |
|---|---|---|
| When | One likely cause, or the failure is already visible | Two or more open causes, a critical or major incident with no visible failure yet, or any investigation an alert started |
| Reads a round | One | Up to three, one for each open cause |
| Steps and time | Your organization's settings: 8 steps and 10 minutes by default | Twice your settings, up to 20 steps and 60 minutes |
| Cost | 5 Nova units | 10 Nova units |
An investigation costs 1 unit, whatever its size, only if Nova can't read anything: nothing it tried to read returned data. Once a read has returned data, it costs its full price, whether it ends with Cause found, Likely cause or No cause found, and whether it runs to the end or someone stops it. With fewer than 10 units left, a deep investigation runs as a standard one and the card says so. An alert-started investigation is never made smaller, so with fewer than 10 units left it is refused: see If it's refused and Nova Units.
How a run ends
An investigation stops when one of these happens, and the card says which in plain words:
- It finds the cause.
- It converges. A round tested every open cause and moved none of them, so more reads of the same kind would not help.
- It has a supported cause to report. Once it has one and a short time has passed (90 seconds for standard, 5 minutes for deep), Nova reports its first findings rather than start more steps.
- Nothing is left to read, or what is left needs access nobody gave.
- The app you named is not on the platform. The card says so, with any apps whose names are close.
- It runs out of steps or time. Time spent waiting on a person does not count.
- A person stops it.
Whatever stops the run, Nova concludes with what it has, and the answer is never upgraded because a limit ran out. A skill that fails twice in a row because it is unreachable or erroring is not used again in that run, and the card says what could not be read.
How Nova learns
Each investigation's cause, what it ruled out and how it ended become past cases for your organization. On the next incident on the same app, they shape the order Nova tests causes in. When you take a fix over, your answer to what fixed it labels the case. With Operational Memory (Enterprise), Nova also recalls your own runbooks.
Limits and caveats
- Nova sees what your skills can read. If a signal lives in a tool that is not connected, Nova cannot use it.
- Built-in experience covers common Kubernetes app failures. Outside them, Nova still investigates from first principles with a generic set of causes.
- Steps and time bound every investigation. When they run out, the answer is often Likely cause.
- Declined reads cost certainty. A cause that needed a declined read stays Likely.
- Fixes need a confirmed cause. For a likely cause, Nova shows a runbook, not a button.
- Resolved means healthy after the check window, not that the cause can never return. A recurrence is flagged on the next investigation.
FAQ
Does Nova change anything without approval? No. An investigation always asks before it changes anything, whatever your Changes & Remediation rule says, and the change still runs through your guardrails.
Can Nova read logs without asking? Only for apps an admin lists under Apps whose logs Nova reads without asking, or if your organization sets Sensitive Data Access to Allowed. Otherwise it asks for each app in each investigation.
What if nobody answers? A read not answered by the deadline counts as declined. A change not approved in time expires, nothing changes, and the card reads Approval expired.
What if two people answer at once? The first answer is applied. Everyone else is told someone answered first, and nothing runs twice.
Can Nova use my monitoring tools? Yes, on Enterprise, through the Datadog skill or your own MCP server. Their tools go through the same guardrails.
What if Nova is wrong? Decline or reject the fix, or choose I'll fix it myself. When Nova asks what fixed it, correct its guess; the next investigation learns from that.
Can alert text steer Nova? No. Alert labels and annotations reach Nova as quoted data, not instructions.
What does it cost? 5 Nova units for a standard investigation and 10 for a deep one, or 1 unit if Nova can't read anything. The card shows the size and price before work starts. See Nova Units.
Next
- Configure: approvers, limits and cost.