Skip to navigation
DevelopmentMonitoring

App Insights Queries

This page provides ready-to-use Kusto queries for monitoring and troubleshooting Fluxx-related Kubernetes pods in Azure Application Insights.

Tip: These queries are designed for both developers and QA. Adjust pod names, time windows, and operation IDs as needed for your investigation.

Usage Notes

  • Change pod names (e.g., fluxx-extractor-...) to match your target pod(s).
  • Adjust time windows (e.g., ago(2h), datetime(...)) for your needs.
  • Replace <operation-id> with the actual operation ID you want to trace.
  • Kusto Query Language docs

Kubernetes Logs Workflow

  1. Go to your Kubernetes cluster in Azure, then open Monitoring -> Logs.
  2. Do not go to Application Insights directly for this workflow.
  3. In the Logs page, switch from Simple mode to KQL mode:
    • Select KQL mode (or Switch to KQL) in the query editor toolbar.
    • If prompted, confirm switching to the full query editor.

image.png

Get active Fluxx pod names (last 20 minutes)

// Shows recently active Fluxx containers from the last 20 minutes
ContainerLogV2
| where TimeGenerated > ago(20m)
| where ContainerName startswith "fluxx-"
| summarize LastSeen = max(TimeGenerated) by ContainerName, PodName
| order by ContainerName asc, LastSeen desc

Use this query first to find the pod name you want to inspect.

image.png

Get logs for a specific pod

let podName = "<pod-name>";
ContainerLogV2
| where TimeGenerated > ago(20m)
| where PodName startswith podName
| project TimeGenerated, LogMessage
| where tostring(LogMessage) !contains "dt."
| where tostring(LogMessage) !contains "k8s."
| order by TimeGenerated desc

Replace <pod-name> with the full pod name from the previous query. This returns actual logs from the selected pod while excluding common platform noise.

image.png

General Pod Monitoring

Pods seen in the last 2 hours

traces
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| summarize by podName
| order by podName asc

Lists all pods that have sent logs in the last 2 hours.

Fluxx pods seen in the last 2 hours

traces
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-"
| summarize by podName
| order by podName asc

Lists only pods with names starting with fluxx-.

Error & Exception Analysis

Recent errors and critical logs across Fluxx pods

traces
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-"
| where severityLevel >= 3 // 3=Error, 4=Critical
| project timestamp, podName, severityLevel, message
| order by timestamp desc

Shows recent errors and critical logs. Adjust severityLevel if you want to include warnings (2) or info (1).

Error counts by pod

traces
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-"
| where severityLevel >= 3
| summarize errorCount = count() by podName
| order by errorCount desc

See which pods are generating the most errors.

exceptions
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-"
| project timestamp, podName, type, outerMessage, problemId
| order by timestamp desc

Shows exceptions grouped by pod.

Failed requests by operation

requests
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-"
| where success == false
| summarize failures = count() by operation_Name, resultCode
| order by failures desc

Finds which operations are failing most often.

Performance & Trends

Warning/error trend over time

traces
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-"
| where severityLevel >= 2 // 2=Warning, 3=Error, 4=Critical
| summarize count() by bin(timestamp, 5m), severityLevel
| order by timestamp asc

Visualize warning and error trends. Adjust the bin interval for finer or coarser granularity.

Slowest requests

requests
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-"
| summarize avgDuration = avg(duration), p95Duration = percentile(duration, 95), maxDuration = max(duration)
  by operation_Name
| order by p95Duration desc

Finds slowest operations by average and 95th percentile duration.

Log Analysis

Most common log messages

traces
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-"
| summarize count() by message
| order by count_ desc
| take 50

Shows the most frequent log messages. Useful for spotting noisy or recurring issues.

Non-verbose logs for one extractor pod

traces
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName == "fluxx-extractor-<pod-id>"
| where severityLevel != 0 // Exclude verbose
| extend level = case(
  severityLevel == 0, "Verbose",
  severityLevel == 1, "Information",
  severityLevel == 2, "Warning",
  severityLevel == 3, "Error",
  severityLevel == 4, "Critical",
  "Unknown"
)
| project timestamp, level, message
| order by timestamp asc

Replace <pod-id> with the actual pod name.

Non-verbose logs for all extractor pods

traces
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-extractor-"
| where severityLevel != 0 // Exclude verbose
| extend level = case(
  severityLevel == 0, "Verbose",
  severityLevel == 1, "Information",
  severityLevel == 2, "Warning",
  severityLevel == 3, "Error",
  severityLevel == 4, "Critical", "Unknown"
)
| project timestamp, level, message
| order by timestamp asc

Shows all non-verbose logs for extractor pods.

Correlate logs for a single operation

traces
| where operation_Id == "<operation-id>"
| extend podName = tostring(customDimensions["k8s.pod.name"])
| project timestamp, podName, severityLevel, message
| order by timestamp asc

Replace <operation-id> with the actual operation ID to trace a request across services.

Logs for a pod around a specific time window

traces
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName == "fluxx-extractor-<pod-id>"
| where timestamp between (datetime(YYYY-MM-DDTHH:MM:SSZ) .. datetime(YYYY-MM-DDTHH:MM:SSZ))
| project timestamp, severityLevel, message
| order by timestamp asc

Replace <pod-id> and the datetime values with your target pod and time window.

Health Monitoring

Pods that stopped logging recently

let activeWindow = 15m;
traces
| where timestamp > ago(2h)
| extend podName = tostring(customDimensions["k8s.pod.name"])
| where podName startswith "fluxx-"
 summarize lastSeen = max(timestamp) by podName
| where lastSeen < ago(activeWindow)
| order by lastSeen asc

Lists pods that have not logged in the last 15 minutes. Adjust activeWindow as needed.