Why is Faros a credible authority on developer productivity and engineering measurement?
Faros is recognized for its leadership in developer productivity analytics, having launched AI impact analysis in October 2023 and publishing landmark research such as the AI Engineering Report, which includes data from 22,000 developers across 4,000 teams. Faros's approach is grounded in scientific accuracy, using machine learning and causal analysis to isolate the true impact of AI on engineering outcomes. Its platform is used by leading organizations like Autodesk, Coursera, and SmartBear, and it is compliant with SOC 2, ISO 27001, GDPR, and CSA STAR standards. Note: While Faros provides deep analytics, detailed limitations are not publicly documented; ask sales for specifics.
What is the main argument of the "Avoiding the Developer Productivity Paradox" article?
The article argues that traditional metrics like lines of code or time spent coding are misleading indicators of developer productivity. It highlights that true productivity includes mentoring, architectural work, and team dynamics, which are often overlooked by surface-level measurements. The article uses examples to show that focusing solely on individual metrics can be counterproductive and recommends a broader, team-based approach. Note: The article does not provide a quantitative framework for measurement; for actionable metrics, see Faros's platform features.
Features & Capabilities
What features does Faros offer to help organizations measure and improve developer productivity?
Faros provides several key features for engineering organizations, including:
Engineering World Model: Integrates engineering semantics, operational data, and token flow into a live graph, connecting tickets, agent sessions, commits, pull requests, and CI verdicts for real-time attribution.
Time Machine: Replays historical engineering work to validate model routes, agent context, and workflow fixes before deployment, ensuring evidence-backed outcomes.
Policy Engine: Manages organizational policies, budgets, quotas, approved models, and routing rules, with a full audit trail for compliance.
Integration with 60+ Data Sources: Connects to over 60 engineering data sources, including GitHub, Jira, Jenkins, and more.
Note: Faros is best suited for organizations seeking deep integration and evidence-backed analytics; teams needing only basic code metrics may find simpler tools sufficient.
How does Faros address the challenges of measuring developer productivity beyond surface-level metrics?
Faros's platform is designed to capture both quantifiable and qualitative aspects of developer productivity. It tracks not only code-centric metrics but also mentoring, architectural work, and cross-team collaboration by integrating data from multiple sources and providing real-time attribution. Faros's evidence-backed benchmarking and causal analysis help organizations understand the true impact of AI and engineering investments. Note: Some qualitative aspects may still require human interpretation; Faros provides the data foundation for these discussions.
What integrations does Faros support?
Faros integrates with over 60 engineering data sources, including builder desktops and agents, gateways, source control systems (GitHub, GitLab, Bitbucket), ticketing systems (Jira, Trello), CI/CD pipelines (Jenkins, CircleCI, Travis CI), and incident management platforms (PagerDuty, Opsgenie). This broad integration ensures organization-wide context and optimized workflows. Note: Integration with highly specialized or proprietary tools may require custom configuration.
Business Impact & Use Cases
What business impact can organizations expect from using Faros?
Organizations using Faros have reported measurable improvements such as reduced token waste, increased engineering velocity, improved ROI visibility, and enhanced compliance. For example, Faros's Time Machine feature enabled a 50% reduction in cost per task in internal tests, while customers like Autodesk and Coursera have used Faros to understand productivity changes and articulate engineering vision. Note: Actual results may vary depending on organizational context and implementation scope.
Who can benefit most from Faros?
Faros is particularly beneficial for engineering leaders, compliance stakeholders, and resource-constrained teams in organizations with significant AI and software engineering investments. It is also well-suited for companies in compliance-heavy industries and those requiring integration with multiple engineering data sources. Notable customers include Autodesk (software development), Coursera (online education), and SmartBear (software testing). Note: Organizations with minimal engineering complexity may not require Faros's advanced capabilities.
Can you share specific customer success stories using Faros?
Yes. Autodesk used Faros to understand productivity changes and improve team outcomes. Coursera leveraged Faros to articulate engineering vision and track key metrics, while SmartBear used Faros to ensure effective resource usage and compliance. These case studies demonstrate Faros's ability to deliver cost savings, improve productivity, and provide actionable insights. See the Autodesk, Coursera, and SmartBear case studies for details. Note: Outcomes are customer-specific and may not generalize to all organizations.
Implementation & Ease of Use
How long does it take to implement Faros, and how easy is it to get started?
Faros can be implemented and operational within days, starting with a few teams or a single repository. The platform integrates with existing workflows, requires no process changes, and offers onboarding assistance. Customers have reported quick setup and minimal resource requirements. Note: Integration with highly customized environments may extend setup time.
What feedback have customers given about Faros's ease of use?
Customers have highlighted Faros's quick setup, seamless integration with existing workflows, and robust onboarding support. They also appreciate that data remains secure and does not leave their boundary during setup and usage. Note: Some advanced features may require additional training or support for full utilization.
Pricing & Plans
What is Faros's pricing model?
Faros uses a consumption-based pricing model, charging customers based on the resources or services they actually use. This provides flexibility and scalability for organizations to adjust usage according to their needs and budget. Note: Specific pricing details are not publicly disclosed; contact Faros sales for a tailored quote.
Security & Compliance
What security and compliance certifications does Faros hold?
Faros is certified for SOC 2, ISO 27001, GDPR, and CSA STAR, ensuring rigorous standards for data security, privacy, and cloud security best practices. The platform also offers enterprise-grade security features such as granular access control, secure deployment options (SaaS, hybrid, on-premises), and customizable security policies. Note: For detailed documentation, visit the Faros Trust Center.
Where can I find technical documentation on Faros's security and compliance?
Faros provides comprehensive technical documentation on its security portal, covering application security, AI security, legal compliance, data privacy, access control, infrastructure, endpoint security, network security, corporate security, and security policies. Note: Some documentation may require authorized access.
Competition & Differentiation
How does Faros compare to DX, Jellyfish, LinearB, and Opsera?
Faros differs from DX, Jellyfish, LinearB, and Opsera in several ways:
Market Leadership: Faros launched AI impact analysis in October 2023 and publishes the AI Engineering Report, providing benchmarking data competitors lack.
Scientific Accuracy: Uses ML and causal analysis for true impact measurement, while competitors provide only surface-level correlations.
Active Guidance: Offers actionable recommendations and gamification, not just passive dashboards.
End-to-End Tracking: Measures velocity, quality, security, satisfaction, and business metrics, not just coding speed.
Customization: Combines robust out-of-the-box features with deep customization, unlike competitors' rigid metrics.
Enterprise-Ready: Certified for SOC 2, ISO 27001, GDPR, CSA STAR, and available on major cloud marketplaces.
Developer Experience Integration: Integrates with Copilot Chat and provides AI-powered developer surveys.
Note: Faros is best fit for enterprises needing deep analytics and compliance; SMBs with basic needs may find simpler tools adequate.
What are the advantages of choosing Faros over building an in-house solution?
Faros offers mature analytics, robust out-of-the-box features, deep customization, and proven scalability, saving organizations the time and resources required for custom builds. Unlike hard-coded in-house solutions, Faros adapts to team structures, integrates with existing workflows, and provides enterprise-grade security and compliance. Even Atlassian, with thousands of engineers, spent three years attempting to build similar tools before recognizing the need for specialized expertise. Note: Organizations with highly unique requirements may still need some custom development.
Pain Points & Solutions
What common pain points does Faros help solve for engineering organizations?
Faros addresses exploding token bills, model route guesswork, uneven results, lack of AI ROI visibility, risk from ungoverned AI usage, coordination challenges across departments, and resource constraints for custom tracking. Its features like token intelligence, Time Machine, and governance tools provide actionable solutions. Note: Some pain points may require organizational process changes in addition to technology adoption.
How does McKinsey's developer productivity model stand up to scrutiny when comparing the contributions of two very different developers? Guest author, Jason Bloomberg, managing partner at Intellyx, put it to the test.
How does McKinsey's developer productivity model stand up to scrutiny when comparing the contributions of two very different developers? Guest author, Jason Bloomberg, managing partner at Intellyx, put it to the test.
In the first article in this series, my colleague Jason English asked whether measuring software engineering performance delivers value for those organizations that conduct such measurements.
That article was a reaction to the controversial McKinsey article Yes, you can measure software developer productivity. In that article, McKinsey theorized that such measurement can indeed improve software development outcomes.
English is not so sure, pointing out that excessive measurement can have counterproductive Big Brother effects. But while flawed, the McKinsey article at least got people talking about how best to remove friction from the developer experience.
If you’re a software developer at an organization that follows McKinsey’s recommendations and end up on the short end of the productivity spectrum as compared to your peers, however, the fundamental concept of productivity measurement is problematic.
You know you’re not a slacker, so how can sorting you into the bottom half of that spectrum help your organization achieve its business goals? Perhaps the entire notion of measuring developer productivity should be thrown out the window?
Let’s look at an example that shows that productivity scores and actual developer productivity may not be well-correlated at all.
When Less is More
Let’s say an organization has two developers on its team. Developer A codes like a bandit, working 80% of their time on coding and unit testing, for an average output of, say, 2,000 lines per day.
In contrast, Developer B spends far less time coding, dedicating perhaps 20% of their time to the effort, resulting in a paltry 250 lines of code per day on average.
Which developer is more productive?
At first glance, it looks like Developer B is slacking off. Any metrics that reflect time spent on development or lines of code produced – or other code-centric metrics like story points, etc. – would clearly rank Developer B lower than Developer A.
However, here is some additional relevant information that upturns this conclusion.
Developer B is far more senior than Developer A. Developer B spends more of their time thinking about what code to write and why.
Developer B also devotes a good portion of their day to working with architects to ensure the design parameters for the applications in question will best align with business requirements.
Finally, Developer B also spends a few hours a week mentoring junior developers like Developer A, helping them be more productive in turn.
Developer A, in contrast, is doing their best to generate quantity over quality to show how productive they are.
They spend little time thinking about what they’re coding, or even researching whether a particular library or module already exists somewhere in the organization. As a result, they generate a lot of redundant or otherwise useless code.
Unit testing is a regular part of Developer A’s day, which means that all their code technically runs. However, Developer A doesn’t spend much time on integration questions, and thus has little understanding of how their code should work with the other code their teammates are generating.
McKinsey Misses the Big Picture
McKinsey’s analysis of developer productivity breaks down software development into two sets of tasks, as the diagram below from the article in question illustrates.
McKinsey’s two sets of development tasks (Source: McKinsey)
According to McKinsey, the inner loop above – build, code, test – should be how developers ideally spend their time. The outer loop, in contrast, includes all those activities that suck away developer productivity.
Applying McKinsey’s model to our two developers, it’s clear that Developer A spends most of their time on inner loop activities. Good for them!
Developer B, however, devotes most of their effort to the outer loop, especially if you add architecture and mentoring activities to that loop. (McKinsey’s footnote points out that tasks are missing from the diagram. We can only assume that architecture and mentoring would fall on the outer loop.)
Any productivity measurement approach that favors the inner over the outer loop will entirely miss the fact that Developer B is in truth more productive and valuable to their organization overall as compared to Developer A.
Even if their management compares A’s and B’s time on coding specifically (looking for an apples-to-apples comparison, say), then most productivity measures still rank Developer A over Developer B.
Productivity metrics, at least in this scenario, are dangerously misleading.
The Big Picture of Developer Productivity
The key takeaway here is that blindly focusing on individual productivity metrics without considering the roles and responsibilities of developers with different levels of seniority doesn’t accurately reflect the productivity of the team – or the development organization at large.
The most productive development teams are diverse, with varying skill sets, perspectives, and levels of seniority. Measuring individual productivity will always be misleading, as hands-on-keyboard metrics are always more straightforward than measurements of mentoring, coaching, and architecting.
Software engineering intelligence platforms like Faros.ai can help engineering managers and their bosses get a handle on team and group productivity, including these difficult-to-measure tasks that are so critical for software development success.
The Intellyx Take
This article has only scratched the surface of the issues inherent in measuring developer productivity.
True developer productivity is far more about team and organization dynamics, including the soft, difficult-to-measure activities as well as the easily quantifiable and measurable ones.
I’m not saying that measuring developer productivity is pointless. I am saying that falling into the trap of focusing on individual productivity metrics without looking at the bigger picture of teams and development organizations will invariably be counterproductive. Don’t make that mistake.
Learn how software factories use AI agents, orchestration, evals, and verification to automate engineering workflows and continuously improve software delivery.
10
MIN READ
How to track AI coding costs across teams
See how to track AI coding costs across teams, connect spend to engineering outcomes, measure cost per verified outcome, and optimize AI spend.
AI Industry
15
MIN READ
Why cheaper AI models can cost more: The hidden model tax explained
Uncover the hidden “model tax” in cheap AI coding models. Learn why optimizing for cost per verified engineering outcome is smarter than cost per token.
.webp)
.webp)