The Definitive Answer to ‘Has this Code Shipped?’

If you work in tech, you probably hear these questions more often than you’d like:

• Did this change go out? Where is it now?
• What did we release this past week/month/quarter?
• If I deploy this service, what’s going to be shipped?

Whether you’re the person posing the question (customer success, product, marketing, business leader) or the one being asked (engineering manager, release manager), it’s frustrating nonetheless.

We should, by now, have an automated and self-serve way to get these questions answered definitely.

But for most organizations, that is not the case. Even if a ticket is done or a PR is complete, it’s often not quite clear whether the code shipped and value has been delivered.

Why Is Tracing Code Changes So Difficult?

Why did a product manager at a leading US bank have to drive to an ATM to see if their new ATM feature was finally live?!

Because it’s quite difficult to track the journey of new functionality through disconnected systems, especially as it’s changing shape along the way. Here’s why:

1. Functionality, whether a bug or feature, traditionally starts as a task in a work management system like Jira. This initial task phase describes what needs to be done and the why behind it.
2. From there, an engineer will translate it into code tracked through commits and PRs in version control systems like GitHub or GitLab.
3. Code is eventually merged and packaged into artifacts that are deployed using a deployment system, e.g., Circle CI or Jenkins. These deployed artifacts take the functionality through different environments like dev and QA before finally delivering it to customers in production. In large organizations or complex systems, the deployment pipeline may involve multiple stages, environments, and checks.

Tracing the code changes up and down this toolchain requires strong integration between the tools and an understanding of the relationships between the various artifacts that encapsulate them.

The larger teams become and the more distributed geographically and architecturally, the harder it becomes to just know. With a microservices architecture, different services might be deployed independently. This can make it challenging to know if a specific feature, which might span multiple services, is fully live.

Further complicating matters, organizations (and even groups within them) have different cadences for advancing code from dev to production, restricted by code cadences and policies.

And, while a powerful tool for controlled releases, the wide adoption of feature flags also introduces uncertainty. A feature might be deployed to production but turned off, leading to confusion about its live status.

Some companies try to solve this problem with better labeling throughout all stages, however, I’ve found this to be brittle and error-prone and it only adds to an already complicated process.

Is the only solution manually verifying the issue yourself? Driving to the ATM? Even if you could afford the hassle, often you simply can’t! You don’t always have access to the software, environment, or configuration in question.

The bottom line is that if you really need to know what’s going on and where functionality is, a fair amount of digging and inference is involved.

Eliminating the Wild Goose Chase

Faros AI has solved this problem for me, and it can for you too.

As a complete and extensible software engineering intelligence platform, Faros AI knits together data from work management, source code, and deployment systems to trace code changes as they get merged, tested, built and deployed, and ultimately released.

As I’ve explained above, this is hard stuff. When a deployment happens, the deployment system can tell you which artifact went out, or, at best, the most recent commit that was released. But what else was in that artifact? Normally, you wouldn’t know.

Faros AI has made it trivial to unpack what was bundled into an artifact so you can easily unwind everything that went out with a given deployment. Each code change is traced not just through its production release; it’s also connected to its corresponding product context through the associated task and its parent (epic, feature).

Here’s how I use Faros AI to utilize this information to answer those frequent “Has this code shipped” questions.

Did this change go out?

Below is a Faros AI chart that lets me and my colleagues easily see where we are on a current feature. I can see across the Jira ticket status, PR status, and which environment the change has made it to.

In this example, my colleagues in customer success can see that the bug is still in development, waiting for a review. However, the second item — a feature — is already in our staging environment and just awaiting a production release.

With Faros, the team can get accurate information in seconds without having to ask PMs or engineers for updates on every item.

What did we release this past week/month/quarter?

Every organization has reporting cadences where it’s necessary to understand what was released in the past week, month, or quarter. This information is vital for updating documentation, notifying customers, and preparing marketing communications.

Personally, I also love to look at this information when I get back from vacation; it helps me catch up on everything I missed.

Here’s a dashboard on Faros that summarizes what’s been released over the last 30 days. Looking at the Released Tasks with Epic and Sha table, I can see:

• The ‘Mock data feed takes ‘now’ time as input’ task is done and all related commits have been released
• The ‘Update CLI’ task is being worked on incrementally; some work has been released but the overall task is still in progress.
  

Beyond a dashboard view, I utilize Faros automations to send a weekly update to our team on Slack and an email summary to leadership.

If I deploy this service, what’s going to be shipped?

With different teams contributing to the same code base, it’s important to know what I’ll be releasing when I pull the trigger.

This comes up often for us at Faros AI for services involving contractors or team members in different time zones.

Not everyone can be in the go/no-go decision about a release. Having a Faros AI dashboard to check what will go out gives me the peace of mind I need to kick off a release and the confidence to know what is about to go live.

This dashboard of “Stuff in Dev” has all the work that will go out in the next production release.

Have we closed out completed work?

Data hygiene can be a struggle, more so when the work on an epic or feature is distributed across multiple teams or contributors — each completing their work at a different pace. The unitary stories, tasks, or sub-tasks move to ‘Done’, but often the parent is forgotten in some “in progress” state.

At large organizations, it does become hard to know which epics should be closed out and when.

With Faros AI automations, you can create alerts to notify the epic owner when all the children stories are complete and the epic itself is still ‘In Progress’. This way, they can be sure to tie up any remaining activities required to close the parent.

Visibility Is a Productivity Game Changer

Our current economy has everyone trying to do more with fewer resources. GitHub Copilot is unlocking developer productivity. Software engineering intelligence platforms are doing the same for managers and leaders.

If you want visibility similar to what I have into code changes, deployments, and releases, you might want to try Faros AI. Our mission is to maximize the effectiveness and efficiency of software engineering, and that includes eliminating the scavenger hunt part of our jobs.