StatsD All Up in This

Why?

Knowing is better than guessing

What Are We Doing Already?

Huh?

Aggregated Data Over Time

Three types of data it can record

What is StatsD?

What is Graphite?

Word Problems

Delayed Job Queues

Problem

How queued do our delayed job servers get?

Proposal

Every minute, look at the queued jobs, and record how many jobs are at each priority level

Send Stats

prioties.each do |priority, count|
  # e.g., "delayed_jobs.10", 45
  Metrics.record_number("delayed_jobs.#{priority}", count)
end

Query

stats.gauges.ngin_production.delayed_jobs.*
Example graph of delayed jobs queues

Ngin Web Server Capacity

Problem

Over time, how many of our available web server workers are in use? How close to capacity are we during peak traffic?

Proposal

Every minute, on each server, query the passenger-status script, and record how many active, booted, and maximum workers are available and sum them up.

Send Stats

Metrics.record_number("passengers.#{server_id}.active", stats.active)
Metrics.record_number("passengers.#{server_id}.booted", stats.booted)
Metrics.record_number("passengers.#{server_id}.max", stats.max)
Metrics.record_number("passengers.#{server_id}.queued", stats.queued)

Query

sumSeries(stats.gauges.ngin_production.passengers.*.active)
Example graph of Ngin Passenger capacity

Completed Games

Problem

How many games are completed every night, and which sports are they for, and what type of scoring do they use?

Proposal

Every time a game is completed, record its sport and its scoring type.

Send Stats

Metrics.record_counter("games.completed-by-sport.#{sport.name}")
Metrics.record_counter("games.completed-by-scoring-type.#{scoring_type}")

Query

stats_counts.ngin_production.games.completed-by-sport.*
Example graph of games completed, by sport
Example graph of games completed, by scoring type

Further Reading