The platform underneath a data incident
Data problems often start below the data. Varqeni reads what Infrastructure Operations sees, at the moment a pipeline incident opens, on the things that pipeline runs on.
AvailabilityAvailable to everyone with early access to Varqeni.
Steps
- Open a pipeline investigation. Platform underneath appears in the Summary when something it runs on is down, for example the Kafka Connect worker: a Kubernetes deployment has unavailable replicas.
- The same line is on the page card, in the timeline and in the write-up, and in a data check that finds rows lost or duplicated.
- It is context only. The cause, its confidence and the recommended actions are decided first and are not changed by it, and the autopilot never acts on it.
What you should see: In the lab it was right in 32 of 32 blind trials: a Kafka Connect worker or schema registry with no node to run on named underneath, and a connector stopped or paused on purpose, where nothing underneath was named.
What it can and cannot do
Can
- Name the node, pod or service underneath a pipeline incident when something the pipeline runs on is down.
Cannot
- Change the cause or the fix because of it, or see a fault the metrics cannot.
If it does not work
- Nothing shown: nothing the pipeline runs on was down when the incident opened. A fault that cleared before then, or one the metrics cannot see, is not named.
Related articles
- Read an investigationOpen an investigation to see the cause of record, the model review, what was ruled out and what a person should do next.
- Read the estate under InfrastructureUnder Infrastructure, the console shows hosts, Kubernetes, data services, Kafka, autoscalers, load balancers, storage pools and certificates, with any findings.
- What each cause meansThe causes Varqeni names for a change data capture pipeline, in plain words.
Still stuck? Contact support or email info@fortisdatagroup.com.