Sentiment Tracking
Sentiment tracking analyses user messages in AI agent conversations to measure satisfaction and detect negative trends. Use it to monitor conversational quality at scale and catch issues before they impact user experience.
Metrics
NodeLoom tracks the following sentiment metrics for each AI agent workflow:
| Metric | Type | Description |
|---|---|---|
| Total messages | Counter | Total number of user messages analysed across all conversations. |
| Positive count | Counter | Messages classified as positive (satisfaction, gratitude, approval). |
| Negative count | Counter | Messages classified as negative (frustration, complaints, dissatisfaction). |
| Neutral count | Counter | Messages classified as neutral (factual questions, instructions, acknowledgements). |
| Average score | Float (0--1) | Running average sentiment score across all messages. 0 = fully negative, 0.5 = neutral, 1 = fully positive. |
Classification model
Alert Types
Sentiment alerts are triggered when metrics cross configurable thresholds:
| Alert Type | Description |
|---|---|
| Negative spike | A high proportion of messages in the evaluation window were classified as negative. Indicates a sudden surge in user dissatisfaction. |
| Score drop | The average sentiment score dropped significantly compared to the previous period. Indicates a gradual decline in conversation quality. |
| High volume | The total number of messages exceeds a threshold, which may indicate an agent loop, spam, or a misconfigured workflow generating excessive messages. |
Configurable Thresholds
Each team can customise sentiment thresholds from the workspace monitoring settings:
| Setting | Description |
|---|---|
| Negative spike threshold | Percentage of negative messages in the evaluation window that triggers an alert. |
| Score drop threshold | Percentage decrease in average score compared to the previous period. |
| Evaluation window | The rolling time window used to calculate alert conditions. |
| High volume threshold | Message count that triggers a high-volume alert. |
| Minimum sample size | Minimum messages in the window before alerts can fire. Prevents false positives from small samples. |
Tuning for your use case
Trends
The sentiment dashboard provides trend views at multiple granularities:
| View | Period | Use Case |
|---|---|---|
| Daily | Last 30 days | Spot day-to-day fluctuations and correlate with deployments or incidents. |
| Weekly | Last 12 weeks | Identify weekly patterns (e.g., Monday spikes) and measure improvement over time. |
Each trend view shows the sentiment score line chart, positive/negative message counts as stacked bars, and any triggered alerts overlaid as markers.
Per-Workflow Breakdown
Sentiment metrics are tracked per AI agent workflow, so you can compare performance across different agents. The monitoring dashboard shows a sortable table of all agent workflows with their current average score, message counts, and alert status. Use this to identify which agents need prompt improvements or additional tools.
Notifications
Sentiment alerts use the same notification channels as anomaly detection and drift alerts -- email and webhook. Configure preferences per team from the workspace monitoring settings.
Next Steps
- Anomaly Detection -- per-execution anomaly scoring and alerting.
- Drift Alerts -- detect gradual performance degradation.
- Token Usage -- monitor and control AI token consumption.