When tech leaders remark that they don’t trust productivity metrics collected from surveys, they’re actually implying one or several of the following:
- They don’t know how to design accurate and reliable surveys
- They don’t trust their developers to respond honestly
- They believe that surveys can only provide subjective data
The first objection is easily solvable through rigorous survey design methods, or by partnering with experts specializing in this field. The second indicates lack of trust in your people—a far greater problem than how to measure productivity.
The final objection is attributable to the common misconception that surveys only capture subjective opinions and feelings. In reality, surveys can also capture objective information. Google’s DevOps Research and Assessment (DORA) team, for example, uses surveys to measure lead time and deployment frequency.
The outsized prejudice against surveys amongst tech leaders derives, in my view, from our origins. Developing software inherently involves relying on precise data from highly complex systems. Leaders naturally assume that applying the same strategy to measuring their developers is the only way to yield insightful results. Case in point: social science isn’t part of computer science curricula.
Leaders are increasingly recognizing that only behavioral measurement methods can offer a holistic understanding of developer experience. Industry giants such as Google and Microsoft rely on surveys as their primary source for gathering insights and cross-checking log-based metrics.
I think the fact that our survey items and behavioral methods provide the same information as our log metrics suggests that companies lacking the comprehensive logging of Google can effectively use those approaches.
I believe that we’ll continue to see increased adoption of survey-based behavioral measurement methods industry-wide, particularly as leaders look for alternatives to ineffective log-based metrics solutions.