Terminology

Sleuth uses terms that you are probably already familiar with as a developer. Just to be sure we're all on the same page, below is a list of some common terms.

Information Architecture (IA)

Sleuth can help you track the health and status of your deploys by providing a single pane of glass through which you can view all of your change and impact sources. The Sleuth information architecture terms should already be familiar to you since we use industry-standard CI/CD nomenclature.

The Sleuth Information Architecture

In Sleuth you create a Project container, which houses all the necessary environments your team might need to create, develop, and test your applications. These environments might include production, staging, or development, and could even account for different deployment strategies such as canary, blue/green, rolling deployments, etc.

Once you've created and configured the various environments within your project, you can start adding connections to your Change Sources and Impact Sources (see Integrations for more information on connecting Change Sources and Impact Sources).

Sleuth tracks Change Sources, such as Code Deployments, Feature Flags, and Infrastructure, and constantly analyzes the information they contain to capture the state of your code before, during, and after deploys. Additionally, Sleuth intakes information provided by various Impact Sources, such as Error Rates, Uptime, and Other SLIs.

Dashboard

Combining Impact Source information with Change Source data is what drives the information you see on the Sleuth Dashboard.

You can instantly see the impact of your deploys on your entire project environment over a period of time by viewing the Trend Graph; for detailed information on individual deploys, you can view a deploy card (see below).

You can view a live version of the Sleuth Dashboard at https://app.sleuth.io/sleuth/sleuth It's what the Sleuth team uses every day to make awesome!

Deploy cards

Learn more about deploy cards here.

Size

Another significant metric assessment Sleuth provides is Size. The Size chart shows you how many large versus small deploys you have committed to your repos (changes can be Small, Medium, Large, or Gigantic). Since the overall goal of solid CI/CD practice is to deploy small and deploy often, the Size chart gives you instant insight into whether you're continuously deploying small, effective changes to your repositories instead of occasional gigantic, unstable changes, which could prove problematic if a rollback is necessary when a change proves fatal to your application.

Projects are the main entities in Sleuth. They house your code deployments, feature flags, impact sources, and any manual changes you configure. Think of them as the application you're deploying.

Code deployment

Code deployments track changes made via source code and the software development surrounding the change. Each deploy collects the code reviews, issues, code changes and authors of the change being deployed to your systems. Code can live on either GitHub or Bitbucket repos.

Feature flags

Sleuth tracks feature flags changes in LaunchDarkly by changing the values of feature flags. Each flag change collects the changes made, who made them, and the state of your other flags and the linked code version deployed at the time of the change. Feature flags are an integral part of software development, and Sleuth tracks them along with other metrics to provide you with a snapshot of your deployments' health.

Impact

Impact is the effect of your deploys over a period of time. As you perform your commits and PRs to your code repos, Sleuth is constantly ingesting the information generated by your change sources, such as code deployments, feature flags, and error and metric sources. The collective effect of the information generated by all of your change sources is what defines the impact (observability and SLI are similar terms used in software development). Sleuth detects when impact has deviated from a baseline value and keeps you informed on this deviation by constantly generating an impact reports, which are viewed in the Dashboard and deploy cards. You can also receive notifications via your chatops integration if impact deviates from the baseline, letting you and your team know right away when something is wrong.

Error impact sources, for example, track the impact from application errors on your project. With each deploy, the error rate is sampled and correlated to the change that may have caused it. Error rates are displayed on the deployment frequency graph, and any chatops integrations can send notifications of impacted deploys.

Metric impact sources track the impact from service level metrics, application and infrastructure specific metrics that are being collected from your project. With each deploy, the metric is sampled and correlated to the change that may have caused it. Example metrics include average response time of your API or the percentage of notifications sent within a 5-minute SLA.

A user might want to know if a deploy has had a positive/negative/neutral impact on their project, or wants to know how impact is trending in relation to the deploys that are occurring. They want to understand if a deploy has changed the “normal” behavior of their system so they can react appropriately.

Once a commit is performed, Sleuth samples the commit at the moment of deploy, started by defining a standard deviation, and then afterwards for up to 60 minutes. The average of the errors in that time period is used to compute the impact, which is visible on the deploy card for every commit. This feature is exclusive to Sleuth, and provides you with higher-resolution feedback other than a simple👍or 👎.

Impact is integral to the Sleuth experience, and is one of the main metrics Sleuth computes to provide you with the overall health status of your deploys.

Deploy Tracking Type

When adding a code deployment change source, you can specify when and how Sleuth should track changes in your repo.

  • Manually register each deploy: You must manually notify Sleuth when a commit has been made to your code repo. This can be done via the Sleuth API.

  • Automatically create deploys for every tag on branch: When a commit is tagged, Sleuth will automatically create a deploy.

  • Automatically create deploys for every push to branch: When a commit is pushed, Sleuth will automatically create a deploy.

Deployment Locking

The Lock feature in Sleuth prevents pull requests from merging into main branches, and generates automatic notifications via your Chat Ops integrations (e.g., Slack).

Locking a project/deployment is as easy as pressing one button

Read Sleuth CTO Don Brown's blog post on how Sleuth's locking feature can make your DevOps life easier!

Leaderboard

The Leaderboard provides a competitive component to Sleuth by endeavoring developers to deploy faster and smaller.

Scoring

The score of an author is the sum of several categories of scoring:

  • Deploy: 5 points for each deploy.

  • Author: 3 points for each deploy in which the author was involved but didn't perform the deploy.

  • Impact: 2 points for each deploy rated Healthy, 10 for each deploy rated Improved.

  • Reaction: ½ point for each reaction to the author's deploys, rounded up.

View Compare

The View Compare function provides a link to the right of the SHA that opens up the corresponding code repo (i.e., Bitbucket or GitHub), where you can view the changes between different branches.

Since the interfaces vary, consult the documentation for the corresponding code hosting service for help on using the compare function.