Agent Analytics

Track your AI agents' performance, understand usage patterns, and optimize costs with comprehensive analytics. Monitor success rates, response times, and team productivity gains from automation.

Understanding Agent Analytics

Why Monitor Agent Performance?

Agent analytics help you:

• Measure ROI - Quantify time and cost savings from automation
• Identify Issues - Spot failing agents or poor performance
• Optimize Configuration - Data-driven decisions for better results
• Track Costs - Monitor API usage and associated expenses
• Improve Over Time - Continuous improvement based on metrics

Analytics Dashboard Overview

Access agent analytics from:

1. Navigate to AI Agents in the sidebar
2. Select an agent
3. Click the Analytics tab

Key metrics displayed:

• Total actions performed
• Success rate percentage
• Average response time
• Cost per action
• Most common operations
• Usage trends over time

Core Metrics

Action Success Rate

Percentage of agent actions that completed successfully.

Calculation: (Successful actions / Total actions) × 100

Good success rates:

• 95-100%: Excellent - Agent is well-configured
• 85-94%: Good - Minor issues or edge cases
• 70-84%: Fair - Needs optimization
• Below 70%: Poor - Requires immediate attention

View details:

• Click on success rate metric
• See breakdown by action type
• Filter by date range
• Export detailed logs

Response Time

Average time from agent trigger to action completion.

Typical response times:

• Simple tasks (labeling): 1-3 seconds
• Task creation: 2-5 seconds
• Complex analysis: 5-15 seconds
• Knowledge base queries: 3-10 seconds

Factors affecting speed:

• AI model selected (GPT-4 slower than GPT-3.5)
• Knowledge base size
• Complexity of instructions
• API response times

Improving response time:

• Use faster models for simple tasks
• Reduce knowledge base size
• Simplify agent instructions
• Upgrade to Pro for better performance

Actions Per Day

Total number of actions your agent performs daily.

Use this metric to:

• Track adoption and usage
• Identify usage spikes or drops
• Plan capacity and costs
• Validate automation value

Compare periods:

• View daily, weekly, or monthly trends
• Compare to previous periods
• Identify seasonal patterns
• Measure growth over time

Cost Per Action

Average cost in API credits per agent action.

Cost factors:

• AI model selected (premium models cost more)
• Knowledge base retrieval
• Response length
• Number of retries

Optimize costs:

• Use appropriate model for task complexity
• Reduce unnecessary knowledge base queries
• Set concise response length requirements
• Fix errors to reduce retry costs

Performance Analytics

Action Type Breakdown

See which actions your agent performs most:

Common action types:

• Task creation
• Task updates
• Label assignment
• Priority setting
• Team member assignment
• Comment posting
• Status changes

View breakdown:

1. Navigate to Analytics → Actions
2. See pie chart of action distribution
3. Click any action type for details
4. Filter by date range or project

Use insights to:

• Identify primary use cases
• Validate agent purpose alignment
• Discover unexpected usage patterns
• Optimize for most common actions

Success vs. Failure Analysis

Understand why agent actions fail:

Common failure reasons:

• Invalid data (missing required fields)
• Permission issues (insufficient access)
• Rate limits exceeded
• External API errors
• Knowledge base queries timeout
• Malformed agent responses

Analyze failures:

1. Click Failed Actions in analytics
2. Review error messages and details
3. Identify patterns in failures
4. Export failure logs for deeper analysis

Address failures:

• Fix permission issues
• Add data validation to instructions
• Improve knowledge base organization
• Adjust rate limits
• Refine agent instructions

Time-Based Performance

Track performance trends over time:

Available views:

• Last 24 hours (hourly breakdown)
• Last 7 days (daily breakdown)
• Last 30 days (daily breakdown)
• Last 90 days (weekly breakdown)
• Custom date range

Identify patterns:

• Peak usage times
• Performance degradation
• Sudden failure spikes
• Usage growth or decline
• Seasonal variations

Example insights:

• "Agent fails more often on Monday mornings" → might need better error handling for weekend data
• "Response time increases at month-end" → likely processing more complex tasks
• "Success rate dropped after recent update" → configuration issue to investigate

Usage Statistics

Total Actions Over Time

Track cumulative agent actions:

Visualizations:

• Line graph showing total actions
• Stacked area chart by action type
• Bar chart comparing periods
• Cumulative sum over time

Compare agents:

• View all agents on one chart
• Identify most-used agents
• Compare efficiency between agents
• Benchmark performance

Active vs. Idle Time

See when your agent is working:

Metrics:

• Active hours per day
• Peak usage times
• Idle periods
• Usage consistency

Use cases:

• High idle time: Consider expanding agent responsibilities
• Consistent usage: Agent solving real problems
• Peak times identified: Optimize infrastructure for those periods
• Sporadic usage: May not be well understood or trusted by team

User Engagement

Track which team members use or benefit from the agent:

Metrics available:

• Number of users triggering agent
• Tasks created/updated per user
• User satisfaction ratings
• Adoption rate over time

Improve engagement:

• Share success stories with team
• Provide training on agent capabilities
• Gather feedback from low-engagement users
• Adjust agent behavior based on needs

Cost Tracking

Total Cost Overview

Monitor cumulative spending on agent operations:

Cost components:

• AI model API calls
• Knowledge base processing
• Storage for agent data
• Integration API calls

View by:

• Per agent
• Per time period
• Per action type
• Per project

Cost Trends

Understand spending patterns:

Daily cost tracking:

• Graph showing daily costs
• Compare to previous periods
• Identify cost spikes
• Project future costs

Budget alerts:

1. Set monthly budget for agent
2. Receive notifications at 50%, 75%, 90% of budget
3. Agent pauses automatically at 100% (optional)
4. Review and adjust budget as needed

Cost Optimization Insights

Automated recommendations to reduce costs:

Common suggestions:

• "Switch to GPT-3.5 for simple categorization (60% cost reduction)"
• "Reduce knowledge base queries by caching common information"
• "Remove unused agent that cost $15 last month"
• "Batch operations to reduce API calls"

Implement recommendations:

1. Review suggestion details
2. Estimate impact on performance
3. Apply change
4. Monitor results
5. Roll back if needed

Cost per Productivity Gain

Calculate ROI of your agents:

Formula: Time saved by agent × Hourly rate / Agent cost

Example calculation:

• Agent processes 100 emails/day
• Each email took 2 minutes manually = 200 minutes saved
• Team member hourly rate: $50
• Daily value: (200/60) × $50 = $167
• Agent daily cost: $5
• ROI: ($167 / $5) = 3,340% or 33x return

Track ROI:

1. Navigate to Analytics → ROI Calculator
2. Input time saved per action
3. Set team member hourly rate
4. View calculated ROI
5. Export report for stakeholders

Advanced Analytics

Conversion Funnels

Track agent performance through multi-step workflows:

Example funnel:

1. Email received → 100%
2. Email processed → 95%
3. Task created → 90%
4. Task assigned → 85%
5. Task completed → 70%

Identify drop-offs:

• Where does the process fail most?
• Which steps need optimization?
• Are agents passing appropriate tasks forward?

A/B Testing Agents

Compare two agent configurations:

Setup A/B test:

1. Create two versions of an agent
2. Split traffic 50/50
3. Run for defined period (1-2 weeks)
4. Compare performance metrics
5. Deploy winning configuration

Test scenarios:

• Different AI models (GPT-4 vs Claude 3)
• Temperature settings (0.3 vs 0.7)
• Instruction variations
• Different knowledge bases

Metrics to compare:

• Success rate
• Response time
• Cost per action
• User satisfaction
• Quality of outputs

Custom Reports

Create tailored analytics reports:

Build custom reports:

1. Navigate to Analytics → Custom Reports
2. Select metrics to include
3. Choose date range
4. Add filters (project, user, action type)
5. Save report template
6. Schedule automated delivery

Report types:

• Executive summary (high-level metrics)
• Technical deep-dive (detailed logs and errors)
• Cost analysis (spending breakdown)
• ROI report (productivity gains)
• Compliance report (audit trail)

Data Export

Export analytics data for external analysis:

Export formats:

• CSV (spreadsheet analysis)
• JSON (programmatic access)
• PDF (sharing with stakeholders)
• Excel (advanced analysis with charts)

Export process:

1. Select date range
2. Choose metrics to export
3. Select format
4. Click Export
5. Download file

API access:

# Get agent analytics via API
curl https://api.verk.com/v1/agents/{agent_id}/analytics \
 -H "Authorization: Bearer YOUR_TOKEN" \
 -d '{
 "start_date": "2024-01-01",
 "end_date": "2024-01-31",
 "metrics": ["success_rate", "response_time", "cost"]
 }'

Team-Wide Analytics

Organization-Level Metrics

View analytics across all agents:

Aggregate metrics:

• Total actions across all agents
• Combined cost savings
• Overall success rates
• Most effective agents
• Adoption by team members

Compare agents:

• Side-by-side performance comparison
• Benchmark against organization average
• Identify top performers
• Find underutilized agents

Agent vs. Human Work

Understand the balance between automated and manual work:

Metrics tracked:

• Tasks created by agents vs. humans
• Tasks updated by agents vs. humans
• Time saved through automation
• Work patterns and trends

Visualizations:

• Stacked bar chart (agent vs human work)
• Trend lines over time
• Percentage automation by project
• Team member workload distribution

Use insights to:

• Identify opportunities for more automation
• Validate agent value to stakeholders
• Balance automation with human oversight
• Plan future automation initiatives

Team Productivity Impact

Measure how agents affect team performance:

Key indicators:

• Task completion velocity
• Time to task completion
• Workload distribution
• Team satisfaction scores

Before vs. After analysis:

• Baseline metrics before agent deployment
• Current metrics with agent active
• Calculate improvement percentage
• Project long-term impact

Alerts and Notifications

Performance Alerts

Get notified when agent performance changes:

Alert triggers:

• Success rate drops below threshold (e.g., 85%)
• Response time exceeds limit (e.g., 30 seconds)
• Error rate spikes above normal
• Agent stops working entirely
• Unusual usage patterns detected

Configure alerts:

1. Agent settings → Alerts
2. Choose metrics to monitor
3. Set threshold values
4. Select notification channels (email, Slack)
5. Enable/disable alerts

Cost Alerts

Monitor spending and prevent budget overruns:

Budget notifications:

• Daily spending limit
• Monthly budget threshold
• Cost per action limit
• Unusual cost spikes

Auto-pause options:

• Pause agent at budget limit
• Require approval to continue
• Switch to cheaper model automatically
• Notify admin for decision

Anomaly Detection

Automatic detection of unusual patterns:

Anomalies flagged:

• Sudden success rate drop
• Unexpected action type
• Unusual usage time
• Cost spike
• Performance degradation

Anomaly notifications:

• Email digest of anomalies
• Real-time Slack alerts
• In-app notification badge
• Weekly summary report

Best Practices

Monitor Regularly

Establish a monitoring routine:

Daily:

• Check critical agent status
• Review any failure alerts
• Verify cost tracking

Weekly:

• Review success rates and trends
• Analyze top actions performed
• Check cost vs. budget
• Review user feedback

Monthly:

• Comprehensive performance review
• ROI calculation and reporting
• Identify optimization opportunities
• Plan improvements for next month

Set Meaningful Baselines

Establish benchmarks for comparison:

Initial baselines:

• First week metrics
• Expected performance ranges
• Acceptable cost parameters
• Success rate targets

Update baselines:

• Quarterly based on improvements
• After significant configuration changes
• As usage patterns evolve
• When expanding agent responsibilities

Act on Insights

Don't just collect data—use it:

Action plan:

1. Review weekly metrics
2. Identify 1-2 areas for improvement
3. Implement changes
4. Monitor impact for 1-2 weeks
5. Iterate based on results

Common optimizations:

• Adjust agent instructions
• Change AI model
• Update knowledge base
• Modify tool permissions
• Refine trigger conditions

Share Results

Communicate agent value to stakeholders:

Monthly reports:

• Executive summary (1 page)
• Key metrics and trends
• Cost savings calculation
• Success stories
• Plans for next month

Quarterly reviews:

• Comprehensive performance analysis
• ROI across all agents
• Team feedback summary
• Strategic recommendations
• Budget requests for expansion

Continuous Improvement

Use analytics for ongoing optimization:

Improvement cycle:

1. Measure: Track current performance
2. Analyze: Identify improvement opportunities
3. Hypothesize: Predict impact of changes
4. Test: Implement changes carefully
5. Validate: Measure results
6. Iterate: Continue refining

Track improvements over time:

• Document what changes were made
• Record before/after metrics
• Calculate impact of each change
• Share learnings with team

Troubleshooting Analytics

Missing or Incomplete Data

Common causes:

• Analytics not enabled (check settings)
• Date range selected has no activity
• Agent recently created (limited history)
• Data retention period exceeded

Solutions:

1. Verify analytics are enabled in agent settings
2. Expand date range to include activity
3. Wait for more data to accumulate
4. Check organization data retention policy

Inaccurate Metrics

Common causes:

• Time zone mismatch
• Caching delays (up to 15 minutes)
• Filtered view applied
• Partial data export

Solutions:

1. Check time zone settings in profile
2. Refresh page to update cached data
3. Clear all filters and reapply
4. Export full dataset for verification

Performance Dashboard Loading Slowly

Common causes:

• Large date range selected
• Too many agents displayed
• Complex custom report
• Browser cache issues

Solutions:

1. Reduce date range to recent period
2. View agents individually
3. Simplify custom report
4. Clear browser cache
5. Use data export for heavy analysis

Related Documentation

• Agent Configuration - Optimize agent setup
• Creating AI Agents - Get started with agents
• Team Performance Analytics - Broader team metrics
• Custom Reports - Advanced reporting

---

Questions about agent analytics? Check our FAQ or contact support.