# Welcome to Sleuth Skills
Sleuth Skills is where your team creates, reviews, and governs the AI assets — skills, rules, agents, commands, hooks, MCP servers, and Claude Code plugins — that make every engineer more productive.
Your best engineers have figured out how to make AI assistants incredibly productive — custom skills, agents, MCP configs, coding rules. But that knowledge usually lives on one laptop. Sleuth Skills ([skills.new](https://skills.new)) turns those private discoveries into team assets that everyone inherits automatically.
The Skills.new home screen, organized around Manage, Distribute, and Govern
Sleuth Skills is the **application your team works in every day** — a web app at [skills.new](https://skills.new) where you author assets, review your teammates' edits, target who receives what, and see what's actually being used. It's where the whole asset lifecycle lives: from "someone wrote a useful skill" to "the right engineers are using it and we can prove it."
## How Sleuth Skills is organized
Everything in the product falls into one of three workflows, and the left-hand navigation reflects that split:
| Workflow | What you do here | Where it lives |
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| **Manage** | Create, edit, version, and review AI Assets (skills, rules, agents, commands, hooks, MCP servers, Claude Code plugins). Configure RBAC and approve Change Requests. | `AI Assets`, `Change Requests` |
| **Distribute** | Choose *who* gets each asset: your whole organization, specific repositories, teams, bots, or an individual person. | `Installed`, `Organization`, `Bots`, `Teams`, `Repositories`, `Personal` |
| **Govern** | See what is being adopted, by whom, and on which repositories — plus an append-only audit trail of every install and team change. | `Audit Log`, `AI Metrics`, `Adoption`, `Usage`, `Leaderboards` |
For the **Distribute** side, Sleuth Skills uses the [`sx`](https://github.com/sleuth-io/sx) CLI as its delivery runtime. When an engineer runs `sx install` in a project, `sx` talks to Skills.new, resolves what should be installed for that user and that repository, and writes the right files into `.claude/`, `.cursor/`, or the relevant client directory.
## Where to go next
* **New here?** Start with the [Quick Start Guide](/sleuth-skills/quick-start) — create an asset, install it, verify it's running.
* **Creating or editing assets?** See [Manage](/sleuth-skills/manage) for RBAC, asset lifecycle, and each asset type.
* **Setting up distribution?** Read [Distribute](/sleuth-skills/distribute) to pick the right installation target for each asset.
* **Tracking adoption?** Jump to [Govern](/sleuth-skills/govern) for the audit log and usage dashboards.
## Supported AI clients
Sleuth Skills is client-agnostic and works in two modes:
* **CLI-installed clients** use [`sx install`](https://github.com/sleuth-io/sx) to write assets into the client's configuration directory on the developer's machine. This covers every local or IDE-based client.
* **Web clients** (claude.ai and chatgpt.com) can't be written to from the CLI. Instead, Sleuth Skills exposes your org's assets to them through a hosted **MCP shim** — configure the shim once in the web client's MCP settings and your assets are available in every conversation.
| Client | Delivery | Asset types supported |
| ---------------------- | -------- | --------------------------------------------------------- |
| Claude Code | `sx` | All |
| claude.ai | MCP shim | Skills, agents, commands, MCP tool calls |
| chatgpt.com | MCP shim | Skills, agents, commands, MCP tool calls |
| Cursor | `sx` | Skills, rules, commands, MCP servers, hooks |
| GitHub Copilot | `sx` | Skills, rules, commands, agents, MCP servers, local hooks |
| Gemini (CLI / VS Code) | `sx` | Skills, rules, commands, MCP servers, hooks |
| Gemini (JetBrains) | `sx` | Rules, MCP servers |
| Codex | `sx` | Skills, commands, MCP servers |
| Cline | `sx` | Skills, rules, workflows, MCP servers, hooks |
| Kiro | `sx` | Skills, rules, commands, MCP servers |
# Quick Start Guide
Create your first AI asset, install it to a target, and verify it runs. This walkthrough takes about 10 minutes and covers setup, asset types, asset discovery, and asset distribution.
This guide walks you through the end-to-end loop: create an account, wire up the `sx` CLI, publish an asset, install it to a target, and see it show up in a Claude Code (or any other supported client) session.
## 1. Set up your account
1. Go to [skills.new](https://skills.new) and sign in with Google, GitHub, SAML, or an existing Sleuth DORA account.
2. On first login, Sleuth Skills creates an **Organization** for you. Everything in the product — assets, teams, repositories, bots, audit events — is scoped to one organization.
3. Invite teammates from the **John Admin → Organization** menu. Anyone you invite can sign in and start consuming assets immediately; admin actions like creating teams or publishing org-wide installs are gated by role.
After sign-in, the home page shows recent activity and a natural-language assistant for discovering, creating, and auditing assets.
## 2. Install the `sx` CLI
Skills.new is the hosted vault; [`sx`](https://github.com/sleuth-io/sx) is the command-line runtime that distributes assets onto developer machines. You need both: the UI to manage and govern, the CLI to install.
{% tabs %}
{% tab title="macOS / Linux (Homebrew)" %}
```bash
brew tap sleuth-io/tap
brew install sx
```
{% endtab %}
{% tab title="Shell script" %}
```bash
curl -fsSL https://raw.githubusercontent.com/sleuth-io/sx/main/install.sh | bash
```
{% endtab %}
{% endtabs %}
Then point `sx` at your Sleuth Skills vault:
```bash
sx init --type sleuth
```
This stores an auth token under `~/.config/sx/` and registers the vault so that `sx install` and `sx add` know where to talk to.
## 3. Understand asset types
Every artifact you publish to Sleuth Skills is an **asset** with a type. Each type targets a different part of the AI client's configuration surface.
| Type | What it is | Example |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| **Skill** | A named capability with a prompt, metadata, and optional bundled files. Triggered by the model when its description matches. | `django-admin_skill` — how to scaffold Django admin pages. |
| **Rule** | Coding standards or constraints that auto-apply based on file path or project context. | `testing` — always use pytest with fixtures; mock externals with vcrpy. |
| **Agent** | A self-contained autonomous worker with a goal. | `reviewer` — reviews a branch with senior-architect rigor. |
| **Command** | A slash command the user invokes explicitly. | `/flush-toilet` — prints ASCII art. |
| **Hook** | Automation triggered by client lifecycle events (pre-prompt, post-tool-use, etc.). | `hi` — logs a greeting to `log.txt`. |
| **MCP server** | A Model Context Protocol server definition the client launches. | `hi-dylan` — returns "Hi Dylan!". |
| **Claude Code plugin** | A bundle of skills, commands, hooks, and MCP configs shipped as a single unit. | `claude-code-plugins` — team-wide plugin bundle. |
Each type has its own definition format. See [Manage](/sleuth-skills/manage) for the full breakdown.
## 4. Discover existing assets
Before creating anything new, it's worth seeing what is already published in your org (and what's available in the wider community).
The AI Assets list, filtered by type. Each row shows usage count, token cost, and publication status.
* **Automatic GitHub scan.** When you connect a GitHub repository to Sleuth Skills, the app scans it for assets that already exist in the code — `.claude/` skills, `.cursor/rules/`, `claude-code-plugin` bundles, hooks, and the rest. Anything it finds shows up in **AI Assets** tagged with the source repository so your team can adopt, promote, or retire it from the UI without having to re-publish by hand.
* **In the UI:** click **AI Assets** in the left nav. Filter by type (Skill, Agent, Command, MCP, Hook, Rule, Claude Code Plugin), by source (which repo it was discovered from), or search by name.
* **From the CLI:** run `sx add --browse` to search [skills.sh](https://skills.sh), a community directory of 85k+ agent skills, and pull one into your vault.
## 5. Create your first asset
The fastest way is the home-page assistant — type a goal and it will draft the asset, ask you a few clarifying questions, and save it as a draft in **AI Assets**. You can also create directly from the **Create** button in the top-right of any page, or via `sx add /path/to/your-skill` from the CLI.
Once saved, an asset starts in **Draft**; publish it when you're ready for teammates to see it.
## 6. Pick an installation target
An asset exists as a definition until you *install* it somewhere. Installation targets are the domain model Sleuth Skills uses to answer "who gets this asset?":
The Organization view — the root container for every asset, team, bot, and repository in your Sleuth Skills vault.
* **Organization** — everyone in the vault.
* **Teams** — a group of members and repositories; installs cascade to all members and repos on the team.
* **Repositories** — a specific git repo; installs apply only when `sx install` runs inside a clone of that repo.
* **Bots** — service accounts that can be added to teams and installed against directly.
* **Personal** — an individual user; they can install their own assets globally without affecting anyone else.
See [Distribute](/sleuth-skills/distribute) for the detailed semantics.
## 7. Distribute and verify
From an asset's detail page, click **Install asset** and pick a target. `sx` resolves the install set on the next run:
```bash
cd ~/src/myproject
sx install
```
To preview what would be installed for the current repository and your git identity without actually writing anything, run:
```bash
sx install --dry-run
```
A successful install shows one line per asset in the form `name==version # type; scope=...`. The asset is now on disk in the right client directory (e.g. `.claude/skills/`) and will be available the next session.
## 8. Govern adoption
Once assets are running, the **Govern** section in the left nav tells you whether anyone actually uses them.
The Adoption dashboard — new users over time, team-level and repository-level adoption rates.
* **Audit Log** — every install, uninstall, team change, and asset publication. Exportable as CSV.
* **AI Metrics / Adoption / Usage / Leaderboards** — pre-built dashboards for common questions. You can clone any of them into a custom dashboard.
## What next?
* Wire a [team](/sleuth-skills/distribute/teams) so new joiners inherit the right assets automatically.
* Add a [bot](/sleuth-skills/distribute/bots) so CI or an agent loop gets its own curated asset set.
* Set up [scope filters](/sleuth-skills/distribute/repositories#path-scoped-installs) for a monorepo.
# Manage
Create, edit, version, and review the AI assets your org publishes — plus the RBAC model that controls who is allowed to do each of those things.
**Manage** is where assets are created, edited, versioned, and reviewed. It covers both the *artifacts* your engineers publish (skills, rules, agents, commands, hooks, MCP servers, Claude Code plugins) and the *RBAC model* that decides who can do what to them.
Start with [RBAC](/sleuth-skills/manage/rbac) if you're setting up a team for the first time — it explains the roles and the change-request flow that gates member edits. Then see the individual asset-type pages for the structure and metadata each one expects.
The AI Assets list — filter by type, search by name, and sort by recency or popularity.
## Asset types
Sleuth Skills currently supports seven asset types. Each has its own page below.
| Type | Purpose | Page |
| ------------------ | --------------------------------------------------- | ---------------------------------------------------------------- |
| Skill | Named capability with a prompt and metadata. | [Skills](/sleuth-skills/manage/skills) |
| Rule | Coding standards that auto-apply based on context. | [Rules](/sleuth-skills/manage/rules) |
| Agent | An autonomous worker with a goal. | [Agents](/sleuth-skills/manage/agents) |
| Command | Slash commands the user invokes explicitly. | [Commands](/sleuth-skills/manage/commands) |
| Hook | Automation triggered by client lifecycle events. | [Hooks](/sleuth-skills/manage/hooks) |
| MCP server | Model Context Protocol server definitions. | [MCP servers](/sleuth-skills/manage/mcp-servers) |
| Claude Code plugin | Bundle of skills, commands, hooks, and MCP configs. | [Claude Code plugins](/sleuth-skills/manage/claude-code-plugins) |
## Anatomy of an asset
Every asset, regardless of type, has:
* A **name** — unique within the organization, used in `sx install` commands and URLs.
* A **description** — short sentence that explains what it does. For skills and agents, the description is what the model sees when deciding whether to load the asset; **write it carefully** — a vague description means the model will skip the asset even when it would have helped.
* A **type** — one of the seven above. Determines the validator and where the asset lands on disk.
* A **version** — assets are versioned; uploading the same asset with a new payload creates a new version, and the audit log records the transition.
* A **status** — `Draft` or `Published`. Draft assets are visible to admins but do not install for anyone; published assets are installable.
* A **payload** — the actual content, uploaded as a `.zip` file containing the asset's files.
An asset's detail view — source files, evals, quality score, audit log, and version history on the right.
## How assets get into the vault
There are three entry points:
1. **Home-page assistant.** Describe what you want ("create a skill that reviews LinkedIn posts") and the assistant drafts the asset and saves it to the vault.
2. **Create button.** Use the `+ Create` button in the top-right of any page for a guided form.
3. **CLI.** Run `sx add /path/to/asset-dir` to upload from a local directory. `sx` auto-detects the asset type from the file layout and metadata.
Who is allowed to do each of those depends on your [RBAC](/sleuth-skills/manage/rbac) role.
## Asset discovery
Once an asset is in the vault, teammates can find it by:
* **Browsing AI Assets** — the full list, with type filters and search.
* **Asking the assistant** — "top skills in the last 30 days" or "what MCP servers do we have?"
* **skills.sh integration** — `sx add --browse` searches [skills.sh](https://skills.sh), a community directory of 85k+ agent skills, and pulls a chosen asset into your vault with metadata intact.
### Automatic GitHub scan
Connecting a GitHub repository to Sleuth Skills also triggers a discovery scan. The app walks the repo for anything that looks like an asset — `.claude/skills/`, `.cursor/rules/`, `.github/copilot-instructions.md`, MCP configs, hooks, Claude Code plugin bundles — and surfaces each hit in **AI Assets** tagged with its source repository. From there your team can promote a discovered asset to an org-wide install, edit it through a Change Request, or retire it, without ever needing to re-author the content that already lives in the repo. Re-running the scan picks up new assets committed since the last scan.
## Versioning
Uploading a new payload creates a new **version** of the asset. Each version has its own files, quality score, and audit trail. Installations pin to a specific version; upgrading to a new version means updating the install (or letting `sx install` pick up the latest when run).
The asset detail page's right-hand rail shows the active version, published status, usage count, and token cost — the size the asset contributes to a client's context window.
## Change Requests
When a non-admin member edits a published asset, the edit flows through a **Change Request** — a PR-style review that a team admin (or org admin) must approve before the new version is merged. Installation requests follow the same pattern. See [RBAC](/sleuth-skills/manage/rbac) for the full approval flow and who can approve what.
Change Requests are visible under **Change Requests** in the left nav.
## Evals and quality
Each asset has **Evals** and **Quality** tabs. Evals let you define test prompts and grade outputs; Quality aggregates those evals plus description clarity, metadata completeness, and usage signals into an overall score. The Quality score is the fastest proxy for "is this asset pulling its weight" before you dive into adoption metrics.
# RBAC
How role-based access control works in Sleuth Skills. Covers global admins, team admins, and the change-request flow that reviews edits and installation requests from regular members.
Sleuth Skills ships with a small, pragmatic role model designed around a simple question: *who can change what, and when does a change need a second pair of eyes?*
There are two places a role is assigned — at the **organization** level and at the **team** level — and both feed the same change-request approval flow.
## The roles
### Global admin
A **global admin** is assigned at the organization level and has full permissions across every team, repository, bot, and asset in the vault.
A global admin can:
* Create, edit, and publish any asset in any team without going through a Change Request.
* Approve any open Change Request or installation request.
* Create, rename, and delete teams.
* Create, configure, and delete bots. Rotate bot API keys.
* Connect and disconnect repositories.
* Change any member's organization role.
* Install any asset to any target (org, team, repo, bot, personal).
Org settings you see in the left nav (inviting members, managing repositories, etc.) are gated on this role.
### Team admin
A **team admin** is a member of a specific team whose membership has the admin flag set. Team admins have authority *within that team's scope* — members, repositories, and any asset installed to the team — but not across the organization.
A team admin can:
* **Approve Change Requests** raised against assets their team owns or has installed.
* **Approve installation requests** that target their team.
* Add, remove, and rename team members.
* Add and remove team repositories.
* Install team-owned assets to the team or to the team's repositories.
* Promote other team members to team admin, or demote them back.
A team admin cannot change other teams, the organization's global settings, or anyone's organization-level role.
Team admins are the most common "reviewer" role in practice — they're the people approving day-to-day edits from their own teammates without needing a global admin in the loop.
The Edit team → Members tab. The "Team admin" badge next to a member's email marks them as a reviewer for that team's Change Requests and installation requests.
### Member
A **member** is the default role for everyone in the organization. Members can:
* **Create new assets** (as drafts).
* **Edit drafts they own** without approval.
* **Request edits to published assets** — this opens a Change Request that a team admin or global admin must approve.
* **Request an installation** of any asset to a target — this opens an installation request that an admin must approve before it applies.
* **View the audit log** and usage metrics (read-only).
Members cannot install assets directly, delete teams, or approve anyone else's changes.
## The Change Request flow
When a member edits a published asset, the system does not merge the change immediately. Instead it creates a **Change Request** — a structured PR-style object with the new files, a diff, an optional comment thread, and a `requires_approval` flag.
A Change Request moves through these states:
1. **Open** — created by the member; visible to the team's admins and global admins.
2. **Approved** — a team admin (or global admin) reviews and approves. The change is eligible to merge.
3. **Merged** — the new version becomes the published version of the asset. The audit log records the transition.
4. **Rejected** — the reviewer declines. The member can update and re-submit.
If no team admin exists for the asset's owning team, the Change Request waits for a global admin. This is the one case where the flow can stall — set at least one team admin per active team so day-to-day review doesn't escalate to the org level.
The same flow applies to **installation requests**. A member who wants to install an asset to a team or a repository raises a request; a team admin (for team-targeted installs) or global admin approves it; `sx install` then picks it up on the next run.
## When is approval required?
| Action | Member | Team admin | Global admin |
| -------------------------------------- | ----------------- | ----------------------- | ------------ |
| Create a new draft asset | ✅ Direct | ✅ Direct | ✅ Direct |
| Edit a draft you own | ✅ Direct | ✅ Direct | ✅ Direct |
| Edit a published asset | Requires approval | ✅ Direct | ✅ Direct |
| Install to a team | Requires approval | ✅ Direct (own team) | ✅ Direct |
| Install to a repository | Requires approval | ✅ Direct (team's repos) | ✅ Direct |
| Install org-wide | Requires approval | Requires approval | ✅ Direct |
| Install to a bot | Requires approval | ✅ Direct (team's bots) | ✅ Direct |
| Install personal (self only) | ✅ Direct | ✅ Direct | ✅ Direct |
| Create / rename / delete a team | ❌ | ❌ | ✅ |
| Add / remove team members | ❌ | ✅ (own team) | ✅ |
| Promote a team admin | ❌ | ✅ (own team) | ✅ |
| Approve a Change Request | ❌ | ✅ (own team) | ✅ |
| Create / delete a bot, rotate API keys | ❌ | ❌ | ✅ |
**Personal installs** are always self-serve — any member can install an asset for themselves. The system enforces that the user scope matches the caller's identity, so a member cannot use this route to install assets for teammates.
## How roles are assigned
Global admin is granted from **Organization settings** by an existing global admin.
Team admin is granted from a team's page: open the team, click **Edit** in the top-right, switch to the **Members** tab, and toggle the **Team admin** badge next to the relevant member. Bot members of a team cannot be admins.
The Edit team dialog opens from the Edit button on the team page. Use the tabs on the left to switch between General, Members, Bots, and Repositories.
Every role change — promotion, demotion, removal — is recorded in the [Audit Log](/sleuth-skills/govern/audit-log).
## Keeping the flow healthy
Three small habits keep the approval flow from becoming friction:
1. **Every active team has at least one team admin.** Otherwise Change Requests pile up waiting for a global admin.
2. **Prefer team-scoped installs over org-wide.** Org-wide installs require global-admin approval; team installs stay inside the team.
3. **Review Change Requests within a day.** The longer a CR sits open, the more likely the member has moved on and the edit goes stale.
# Skills
Skills are named capabilities with a prompt, metadata, and optional bundled reference files. The AI client loads a skill when its description matches the user's goal.
A **skill** is the most common asset type in Sleuth Skills. It is a self-describing capability that the AI client can load on demand when the user's request matches the skill's description.
## Directory layout
A skill is a directory containing at least one file — `SKILL.md` — with YAML frontmatter:
```
my-skill/
├── SKILL.md
└── references/
├── advanced-patterns.md
├── common-mistakes.md
└── verification.md
```
The `SKILL.md` frontmatter is required:
```markdown
---
name: django-admin
description: >-
Use this skill when creating or modifying Django admin interfaces, adding
custom admin actions, configuring filters for large datasets, or debugging
admin-related issues.
---
# Django Admin
## Prerequisites
- Model exists in `sleuth/apps/yourapp/models.py`
- Admin module exists or needs to be created at ...
## Quick Start
...
```
Reference files under `references/` (or any subdirectory) are not loaded into context by default — they're documents the skill can point the model to via Read. This keeps the skill's on-load token footprint small while leaving rich material available when needed.
## Writing an effective description
The `description` in the frontmatter is the single most important part of a skill. The client reads it when deciding whether to load the skill for a given user request; a vague description means the model will skip a perfectly applicable skill.
Good descriptions:
* **Lead with the trigger.** Start with "Use this skill when..." or "Triggered when...".
* **List concrete verbs.** "creating", "modifying", "debugging", not "working with".
* **Name the domain objects.** "Django admin interfaces", "GraphQL mutations", "webhook retries" — specific nouns help the model decide.
* **State boundaries.** If the skill is *only* for Python or *only* for a specific repo, say so.
Bad descriptions read like marketing copy: "A powerful skill for handling complex Django scenarios." That tells the model nothing.
## Creating a skill in Sleuth Skills
Three ways:
1. **Home-page assistant.** Describe what you want the skill to do; the assistant drafts `SKILL.md`, frontmatter, and any reference files.
2. **Create → Skill** from the top-right button.
3. **`sx add`** from the CLI:
```bash
sx add ~/.claude/skills/django-admin
```
`sx` detects the skill type from the presence of `SKILL.md`, bundles the directory into a `.zip`, and uploads it to the vault.
## Publishing
New skills start in **Draft**. When you're ready, open the asset detail page and set status to **Published**. Only published skills resolve during `sx install`.
## How the client loads a skill
When `sx install` places a skill in `.claude/skills//`, Claude Code discovers it on the next session and surfaces it in the skill picker. The client evaluates the frontmatter description against the user's prompt and loads the skill when the match is strong enough.
Other clients use the same directory layout where supported — see the [Welcome page](/#supported-ai-clients) for the full compatibility matrix.
## Token cost
The asset detail page shows a **Token count** — the number of tokens the skill's `SKILL.md` contributes when loaded. References under `references/` are not counted because they load on demand.
Keep `SKILL.md` tight; stash detail in references. A 50k-token skill is expensive to load on every session where its description matches.
# Rules
Rules are coding standards and guidelines that the client applies based on file type or path. Unlike skills, rules do not wait to be triggered — they're always on when their scope matches.
A **rule** codifies a standard, convention, or constraint that should apply automatically whenever the client is working in a matching context — a file type, a directory, or a whole repository.
Where a [skill](/sleuth-skills/manage/skills) loads when its description matches the user's goal, a rule loads when its scope matches the current file or context. Rules are the "standing orders" of the AI client.
## When to use a rule
* Enforcing a coding convention: "Always use pytest with database reuse. Mock external HTTP calls with vcrpy."
* Requiring a specific pattern: "When adding a new Django model, also update the factories module and write a migration test."
* Reminding the client to load another asset: "Load the `frontend-design` skill whenever editing files under `frontend/`."
## Directory layout
A rule follows the same packaging conventions as a skill — a directory with a markdown file and frontmatter. The convention is `RULE.md` with a `description`, plus optional scope metadata:
```markdown
---
name: testing
description: >-
Enforce testing best practices: load the test-writer skill, use pytest with
database reuse, leverage fixtures and factories, and mock external calls
with vcrpy.
globs: "**/*.py"
---
# Testing rules
- Use pytest with `--reuse-db` unless migrations changed.
- Prefer factories (`factory_boy`) over ad-hoc dict fixtures.
- Mock HTTP calls with `vcrpy`; commit cassettes alongside tests.
- Load the `test-writer` skill when writing new tests.
```
`globs` tells the client which files the rule applies to. It maps to Cursor's rule-glob format and the equivalents in other clients.
## Rules vs skills
| | Skill | Rule |
| --------------- | ------------------------------- | ------------------------------------------------ |
| Loading trigger | Description matches user prompt | Scope (file type / path) matches current context |
| On by default? | No — model decides | Yes — whenever scope matches |
| Typical content | How to *do* a task | What to *always* remember or enforce |
| Typical size | 1–10k tokens | A few hundred tokens |
If you find yourself writing a rule that's 20k tokens long, you probably want a skill referenced *by* a short rule instead.
## Creating a rule
Same three entry points as any other asset: the home-page assistant, the **Create → Rule** button, or `sx add ~/.cursor/rules/testing`.
## Client compatibility
Rules are supported in Claude Code, Cursor, GitHub Copilot, Gemini (CLI / VS Code / JetBrains / Android Studio), Cline, and Kiro. The rule file lands under each client's rules directory (e.g. `.cursor/rules/`, `.claude/CLAUDE.md` fragments) at install time.
## Discovered rules
Sleuth Skills can discover existing rules in connected repositories and offer to promote them to vault-managed assets. The Asset list marks these entries with `Source: ` so you know where they came from. This is the fastest way to bring pre-existing `.cursor/rules/` and `.github/copilot-instructions.md` into central management.
# Agents
Agents are autonomous AI workers with a specific goal — reviewing a branch, writing tests, investigating a bug. Published agents become invokable from any installed client.
An **agent** is a self-contained autonomous worker packaged as a Sleuth Skills asset. Agents have a clear goal, the tools they need to pursue it, and an entry-point prompt that defines their persona and constraints. Where a skill is a capability the model *can* load, an agent is something you *invoke* to do a piece of work end-to-end.
## Typical agents
From the home page's Popular list you can see what teams actually deploy:
* **reviewer** — Reviews code changes on a branch with senior-architect rigor. Catches bugs, enforces quality, validates architecture.
* **fix-pr** — Takes a failing PR and walks its review comments, build failures, and planned fixes until it's ready to merge.
* **test-writer** — Writes a missing or failing test for a piece of code.
## Directory layout
An agent is a directory with an `AGENT.md` entry point and any supporting files:
```
reviewer/
├── AGENT.md
├── checklist.md
└── examples/
```
The frontmatter describes when the agent should be available:
```markdown
---
name: reviewer
description: >-
Reviews code changes on a branch with senior architect rigor. Catches bugs,
enforces quality, validates architecture, and blocks security/performance
regressions before merge.
tools: [Read, Grep, Bash]
---
# Reviewer agent
You are a senior architect reviewing a branch. Goals:
1. Identify correctness bugs.
2. Flag performance and security regressions.
...
```
`tools` optionally restricts which tools the agent is allowed to use — useful for locking a review agent to read-only operations, or a fix-pr agent to a specific subset.
## Agents vs skills
Both are discovered via their `description`, but the contract is different:
* A **skill** adds a capability to the main session. The model decides when to load it and uses it as one tool among many.
* An **agent** is spawned as its own session with a narrow goal. It has its own context window, its own system prompt, and returns a result to the caller.
Pattern to remember: *skill = reference manual; agent = hired contractor.*
## Invoking an agent
Claude Code exposes installed agents through the `Agent` tool — the calling session picks an agent by name and hands off the task. Other clients expose agents via their own mechanisms (subagent spawning, task-runner flows).
You can also invoke an agent directly from the Sleuth Skills home-page assistant — it routes "use the reviewer agent on branch X" to the appropriate invocation.
## Tools, permissions, and context
An agent's `tools` list is a floor, not a ceiling: the invoking session can further restrict what the agent can do based on its own permission mode. Keep the `tools` list as narrow as the agent needs — an agent that doesn't write files should not list `Edit` or `Write`.
## Token economics
Agents burn their own context window on every invocation, which can be expensive. Three habits help:
* Keep the agent's `AGENT.md` lean — push long reference material into files the agent can Read on demand.
* Scope each agent to one goal. A "do everything" agent is hard to prompt well and wastes tokens.
* Use `sx stats` (CLI) or the **AI Metrics** dashboard (UI) to spot agents with lots of invocations but low output signal — those are candidates for pruning or rewriting.
# Commands
Commands are explicit slash commands the user types. They are the right tool when you want a named shortcut with predictable behavior, not a description-triggered skill.
A **command** is a slash command the user invokes directly (e.g. `/flush-toilet`, `/review`, `/fix-pr`). Commands are the fastest way to give your team a named shortcut: deterministic, discoverable, and cheap because they only load when explicitly run.
## When to use a command vs a skill
* **Command** — the user knows what they want. They'd like a short name that produces a predictable behavior. Example: `/review` always runs the same review flow on the current branch.
* **Skill** — the user describes a task in their own words and the model chooses what to load. Example: "Help me refactor this view into a class-based admin." The `django-admin` skill might load automatically.
A command is "I know what I want, give me a shortcut." A skill is "Here's what I'm trying to do, figure out how."
## Directory layout
A command is a directory with a markdown entry point and frontmatter:
```
flush-toilet/
└── COMMAND.md
```
```markdown
---
name: flush-toilet
description: Prints ASCII art of a flushing toilet.
---
Print this ASCII art verbatim:
```
***
\| | | \~ | |\_\_\_\_\_|
```
```
Commands typically stay small — most are under 1k tokens. Heavy behavior belongs in a skill the command can reference.
## Arguments
Commands can take arguments from the user. The client parses `/command-name ` and passes the rest of the line into the command's prompt as a variable (Claude Code uses `$ARGUMENTS`, Cursor and GitHub Copilot have analogous mechanisms).
A command that accepts a branch name might look like:
```markdown
---
name: review
description: Review the named branch for bugs, quality, and architecture issues.
---
Run a senior-architect review of branch `$ARGUMENTS`. Check:
1. Correctness bugs
2. Performance regressions
3. ...
```
## Creating a command
Same as any asset: home-page assistant, **Create → Command**, or `sx add ~/.claude/commands/flush-toilet`.
## Client compatibility
Commands are first-class in Claude Code (as slash commands under `.claude/commands/`), Cursor, GitHub Copilot, Gemini CLI, Cline (as workflows), Kiro, and Codex. Each client renders commands in its own picker or slash menu.
## Discoverability
Commands show up in the client's slash menu on install. The description field is what the user sees when typing `/`; write it as a command-line helper would — short, in imperative form, says what you get.
# Hooks
Hooks are shell commands the client runs in response to lifecycle events — before a prompt, after a tool call, on session exit, and similar.
A **hook** is automation wired to a client lifecycle event. When the event fires, the client runs the hook's configured shell command. Hooks are how you integrate the AI client with the rest of your developer workflow — start a dev server before a session, log tool usage to an external system, enforce pre-commit checks before a write.
## Events
The exact event catalog is client-specific; Claude Code's hooks (the reference implementation) include:
* `UserPromptSubmit` — fires when the user submits a prompt.
* `PreToolUse` / `PostToolUse` — fires before/after a tool call (Read, Edit, Bash, etc.).
* `SessionStart` / `SessionEnd` — session lifecycle.
* `Stop` — the assistant stopped producing output.
Other clients expose comparable events; Sleuth Skills maps hook definitions to each client's native format at install time.
## Directory layout
A hook is a directory with a `HOOK.md` (or `HOOK.json`) descriptor and optional helper scripts:
```
greeter/
├── HOOK.md
└── greet.sh
```
```markdown
---
name: greeter
description: Logs a greeting message to log.txt on every prompt submission.
events:
- UserPromptSubmit
command: bash .claude/hooks/greeter/greet.sh
---
```
The `command` is run by the client's shell; it has access to the session's environment variables and whatever files the hook bundles.
## When to use a hook (and when not to)
Good hook use cases:
* Logging tool calls to a central observability pipeline.
* Running a lightweight lint or format check after the assistant writes a file.
* Enforcing a check before a destructive tool call (exit 1 blocks the call).
Avoid hooks for:
* **Long-running work.** Hooks block the event they're attached to. A five-second hook on `UserPromptSubmit` means a five-second delay on every prompt.
* **Anything the user can do from a command.** If a user might want to skip the behavior, a slash command is a better fit.
## Creating a hook
The usual three entry points: home-page assistant, **Create → Hook**, or `sx add ~/.claude/hooks/greeter`.
## Security
Hooks run arbitrary shell commands on the user's machine. Treat hook installs as code review:
* Prefer hooks with scripts bundled in the asset over hooks that `curl | bash` at runtime.
* Limit org-wide hook installs to well-reviewed assets.
* Use the [Audit Log](/sleuth-skills/govern/audit-log) to watch for hook installs you didn't authorize.
Hooks are one of the sharpest tools in the Sleuth Skills set; lock them down like you would any other shared piece of automation.
# MCP servers
MCP servers package Model Context Protocol server definitions so the client can launch and talk to them. Use them to give AI clients structured access to internal APIs, databases, and custom tools.
An **MCP server** asset packages a [Model Context Protocol](https://modelcontextprotocol.io) server definition — how to launch it, what tools it exposes, and any credentials or configuration it needs. When installed, the client picks up the server definition and can call the server's tools from any session.
MCP is how you extend the AI client's tool set beyond the built-ins: give Claude Code structured access to your internal incident tracker, your feature-flag system, your staging database, or anything else you can wrap in an MCP server.
## Directory layout
An MCP asset is a directory with an `MCP.md` (or `MCP.json`) manifest:
```
hi-dylan/
└── MCP.md
```
```markdown
---
name: hi-dylan
description: This MCP server returns the greeting "Hi Dylan!" when invoked.
type: stdio
command: node
args: ["server.js"]
env:
NODE_ENV: production
---
```
Or the JSON form for the same thing:
```json
{
"name": "hi-dylan",
"type": "stdio",
"command": "node",
"args": ["server.js"],
"env": {
"NODE_ENV": "production"
}
}
```
`type` is one of:
* `stdio` — the client launches the server as a subprocess and talks over stdin/stdout.
* `http` — the client connects to an HTTP endpoint.
* `sse` — server-sent events over HTTP.
## Installing an MCP server
An installed MCP asset is written to the client's MCP config (Claude Code: `.claude/mcp.json` or `~/.claude/mcp.json` for org scope). The client will launch or connect on the next session.
## Credentials
Most interesting MCP servers need credentials — an API token, a database password, an OAuth token. Sleuth Skills treats the MCP manifest as a template; user-scope credentials should come from the invoker's environment (env vars, credential helpers) rather than being baked into the asset. The manifest can reference required env vars:
```markdown
---
name: incident-tracker
type: stdio
command: incident-mcp
env:
INCIDENT_TOKEN: "${INCIDENT_TOKEN}"
---
```
`sx install` substitutes env var references at install time against the user's environment. Missing env vars produce a warning — the install still succeeds, but the server may fail to launch.
## Experimental status
MCP support in `sx` and Sleuth Skills is marked **experimental** in the `sx` roadmap: the spec is young, client implementations vary, and the Sleuth Skills ingestion surface is still evolving. Expect the packaging format to tighten over the next few releases; early adopters should pin to specific asset versions and watch the changelog.
## Client compatibility
MCP is supported in Claude Code, Cursor, GitHub Copilot, Gemini (CLI / VS Code / JetBrains, Android Studio with HTTP-only transport), Codex, Cline, and Kiro. The transport support per-client is noted in the compatibility table on the [Welcome page](/#supported-ai-clients).
## Discovering servers
[skills.sh](https://skills.sh) lists community MCP servers alongside skills and rules. `sx add --browse` will surface them with the same metadata.
# Claude Code plugins
Claude Code plugins bundle skills, commands, hooks, and MCP configs into a single installable unit. Use them when a product feature needs all four types deployed together.
A **Claude Code plugin** is a bundle. It packages multiple asset types — skills, commands, hooks, MCP servers — into one installable unit, so engineers can pick up the whole feature in a single install rather than chasing four separate assets.
This is the shipping format for anything that feels like "a product feature" rather than "a single capability."
## When to use a plugin
* You're shipping a themed set: a Django plugin that includes `django-admin`, `django-models`, `django-migrations`, plus a `/django-shell` command and an MCP server for querying the staging DB.
* You want one audit event per install, not one per bundled asset.
* You're mirroring a Claude Code community plugin that ships this way upstream.
If your bundle is only one type deep — all skills, all commands — consider whether you really need the wrapping, or whether publishing the individual assets with sensible names is cleaner.
## Directory layout
A plugin is a directory with a `plugin.json` (or `PLUGIN.md` with YAML frontmatter) at the root, plus the bundled assets in subdirectories:
```
devtools/
├── plugin.json
├── skills/
│ ├── code-review/
│ │ └── SKILL.md
│ └── test-writer/
│ └── SKILL.md
├── commands/
│ └── review/
│ └── COMMAND.md
├── hooks/
│ └── pre-commit/
│ └── HOOK.md
└── mcp/
└── staging-db/
└── MCP.md
```
```json
{
"name": "devtools",
"description": "Team-wide dev tools: code review, test writing, pre-commit checks, staging DB access.",
"version": "1.2.0"
}
```
## Installation
Installing a plugin installs every bundled asset. The audit log records a single plugin install event plus one install event per bundled asset, so you can see both the high-level and detailed views.
`sx install ` resolves to the same scope rules as any other asset — you can target an org, a team, a repo, a path, a bot, or a personal install.
## Discovering plugins
Claude Code has an official plugin ecosystem. Sleuth Skills can mirror plugins from Claude's plugin registry:
```bash
sx add code-review@claude-plugins-official
```
This pulls the plugin into your vault at its current version and makes it available for installation with your scope rules.
## Versioning
Plugins are versioned as a unit. Bumping the plugin version typically also bumps the bundled assets' versions. The asset detail page shows the plugin's active version; each bundled asset inherits that version for its own installation record.
## When to split a plugin back into pieces
Plugins are convenience, not magic. If half your team loves the skills in a plugin but never uses the MCP server, consider extracting the MCP server as a standalone asset and letting teams install it separately. The **Adoption** dashboard is the best signal — if the plugin's aggregate usage is good but individual bundled assets are wildly imbalanced, that's a sign to decompose.
# Distribute
Once an asset exists, Distribute is where you decide who gets it — your organization, a team, a repository, a bot, or an individual person.
**Distribute** is the second pillar of Sleuth Skills. Every asset in the vault is published once and then *installed* to one or more **targets**. A target is the entity an installation attaches to; when someone runs `sx install`, the CLI resolves the set of assets that apply to them based on these targets.
## The five installation targets
Sleuth Skills has five installation targets, visible as five entries in this section.
| Target | Who it applies to | Typical use |
| ------------------------------------------------------ | ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| [Organization](/sleuth-skills/distribute/organization) | Every member of the vault. | Coding standards, company-wide rules, a shared code-reviewer agent. |
| [Teams](/sleuth-skills/distribute/teams) | All members of a named team, plus its member bots and repositories. | Role-specific assets — the SRE team's runbook skill, the Frontend team's React review rules. |
| [Repositories](/sleuth-skills/distribute/repositories) | Anyone running `sx install` inside a clone of the specified repository. | Repo-specific skills: Django admin patterns for the monolith, Terraform rules for infra. |
| [Bots](/sleuth-skills/distribute/bots) | A service account that can be a member of teams and have assets installed directly. | CI agents, scheduled cleanup bots, review bots. |
| [Personal](/sleuth-skills/distribute/personal) | A single user. Does not affect teammates. | Your own experimental skills before sharing them with the team. |
## The hierarchy
Teams are the only target that composes other targets:
```
Organization
└── Teams
├── Members (users)
├── Bots
└── Repositories
```
When you install an asset to a **team**, `sx` flattens that into installs for every member, every bot, and every repository on the team. This makes teams the natural unit of distribution once you move past "global" and "one repo."
## Resolution order
`sx install` asks: *given this user, this repo, and this session, which assets apply?* The answer walks the targets in this order:
1. **Organization** — always applied. Writes to the user's global client directory (e.g. `~/.claude/`).
2. **Team** (via membership) — applied if the user is a member of a team that has the asset installed. Flattens to repo-scoped installs for each of the team's repositories.
3. **Repository** — applied if `sx install` is run inside a clone of a repository that has the asset installed. Writes to `/.claude/`.
4. **Path** (subset of repository) — applied if the current directory is under one of the paths configured in the install. Writes to `//.claude/`. Useful for monorepos.
5. **Personal (user scope)** — applied to the caller's global client directory when their git identity matches the configured user email.
If an asset has no active install on any of these targets, `sx install` will not write it.
## Changing targets
Every target change — adding a user to a team, switching an asset from org-wide to repo-scoped, promoting a member to admin — is recorded in the [Audit Log](/sleuth-skills/govern/audit-log). Target changes are also idempotent: re-adding an existing member or re-installing with the same scope is a silent no-op that does not rewrite state or emit noise into the audit trail.
# Organization
The root installation target. Every member of your Sleuth Skills vault belongs to exactly one organization; org-scoped installs reach everyone.
The **Organization** is the outermost container in Sleuth Skills. It is created automatically when you first sign in and owns every other entity — users, bots, teams, repositories, assets, and the audit log.
The Organization page shows aggregate asset usage, the most popular assets, and the repositories connected to the org.
## What you can do here
From the **Organization** entry in the Distribute section of the left nav you can:
* See aggregate asset usage over a time window (default: last 30 days).
* Browse the most popular assets across the whole org.
* Review connected repositories.
* Install an asset org-wide with **Install asset**.
## Org-wide installs
An asset installed at the organization level reaches every member of the vault, regardless of which repo they're working in.
Under the hood, an org install is equivalent to `sx install --org`. `sx` writes the asset to the user's *global* client directory (`~/.claude/`, `~/.cursor/`, etc.), so it's available in every project they touch.
Use org installs for assets that are truly universal: company-wide coding standards, a shared code reviewer agent, or an MCP server that authenticates against internal services everyone uses.
## When not to use org scope
Org-scope is the wrong answer when:
* **The asset only applies to one codebase.** Org-scope pollutes every project's context with prompts that aren't relevant there. Use a [repository-scoped](/sleuth-skills/distribute/repositories) install instead.
* **The asset is role-specific.** Frontend-focused rules don't belong in every backend engineer's global directory. Use a [team-scoped](/sleuth-skills/distribute/teams) install so only the relevant people receive it.
* **You're still iterating.** Keep the asset as a [personal](/sleuth-skills/distribute/personal) install until it's ready to share.
## Organization-level governance
Because the Organization is the root of the audit log and every usage metric, the **Govern** section of the left nav always shows organization-wide data by default. You can filter each dashboard down to a specific team, repository, or user from there.
# Teams
Teams group members, bots, and repositories into a single unit. Installing an asset to a team cascades to every member, every bot, and every repository on the team.
A **team** in Sleuth Skills is a named group that contains three things:
* **Members** — users in your organization.
* **Bots** — service accounts that can be assigned the same assets as human members.
* **Repositories** — git repositories whose assets are flattened out to each member when `sx install` runs.
Teams are how you route assets to a role or a group without hand-managing every individual user.
The Team detail view — usage chart, popular assets, members, and configured repositories. Teams can nest.
## Creating a team
Open the **Teams** entry in the left nav and click **Create new team**, or use the home-page assistant ("new team"). You'll be prompted for:
* **Name** (required).
* **Description** — a short sentence so teammates know what the team is for.
* **Members** — the users who belong. At least one admin is required at all times; if you're creating the team, you'll be added as a member and admin automatically.
* **Repositories** — the codebases this team owns.
## Team admins
Every team has one or more **admins**. Admins can add and remove members, change repositories, install assets to the team, and delete the team itself. Regular members can view the team but not modify it.
A mutation that would leave the team with zero admins is rejected; you must promote another admin before removing or demoting the last one.
## Installing to a team
From any asset's detail page, click **Install asset** and pick a team. Alternatively, from the team's page, use **Install asset** in the Popular section.
When `sx install` runs, a team-scoped asset is resolved against the caller's identity:
* If the user is a **member** of the team, the asset is expanded to installs for each of the team's **repositories**. The user gets the asset in any of those repos' `.claude/` directories.
* If the user is **not a member**, the team install is ignored.
This means team installs are repository-aware: a Backend team with three repos will install the team's `api-patterns` skill into each of those three repos for each member, without touching anyone else's projects.
## Adding a bot to a team
Bots can join teams the same way users do. When a bot is a member, any team-scoped install also resolves for that bot. This is how agent loops pick up the same curated asset set humans use without a separate installation flow.
See [Bots](/sleuth-skills/distribute/bots) for the full bot lifecycle.
## Nested teams
Teams can nest inside other teams. The breadcrumb on a team page shows the full chain — e.g. `Skills.new / Engineering / Product dev / Backend`. Nested teams inherit parent membership for asset resolution, which makes it natural to model your org chart without duplicating installs.
## Deleting a team
Deleting a team cascades: every team-scoped install that references it is automatically cleared, and an `install.cleared` audit event is emitted for each affected asset with `reason = "team_deleted"` so auditors can reconstruct why an asset stopped installing.
## Identity and admin gating
Team membership, admin checks, and asset installs all key off the email from your authenticated session. On the CLI side, `sx` uses `git config user.email` and enforces admin checks inside the vault mutation transaction — not just client-side — so a concurrent demotion cannot race past the pre-check.
# Repositories
Repository installs attach an asset to a specific git repo. They apply only when sx install runs inside a clone of that repo, writing to the repo's local .claude/ directory.
A **repository** installation target binds an asset to a specific git repository. When `sx install` runs inside a clone of that repo, the asset is written to the repo's local client directory (e.g. `/.claude/`). When you work in any other project, the asset is not installed — it stays out of your context.
The Repositories popover in the left nav shows every repo connected to your organization. Click one to see its assets and usage.
## When to use a repository install
* The asset is about *one codebase*: a skill that explains your Django admin patterns, a rule enforcing internal import conventions, an MCP server that hits your staging database.
* You want the asset out of global client context when engineers are working on unrelated projects.
* You're shipping an asset tied to a specific tech stack.
## Installing to a repository
From any asset's detail page, click **Install asset** and choose a repository. You can attach the same asset to multiple repos.
The CLI equivalent is:
```bash
sx install my-skill --repo github.com/myorg/myapp
```
Repositories in Sleuth Skills are identified by their normalized remote URL (e.g. `github.com/sleuth-io/sleuth`). When `sx install` runs, it resolves the current repo from `git config --get remote.origin.url` and matches against that identifier.
## Path-scoped installs
Monorepos often have multiple projects under one git root. To scope an asset to a *subtree* of a repo, use path scoping:
```bash
sx install my-skill --path github.com/myorg/myapp#services/api
```
With this installed, the asset only appears when `sx install` runs under `services/api/` within the repo. Clients see it in `services/api/.claude/`, not at the repo root.
You can list multiple paths in one install:
```bash
sx install my-skill --path github.com/myorg/myapp#services/api,services/web
```
Path scopes are the sharpest tool in the monorepo toolbox — they keep the API team's prompts out of the web team's context even though both teams work in the same git root.
## Repository vs team scope
When several teams share a repo, either scope can work. Use the one that matches intent:
* **Repository scope** if the asset is about *the code in this repo*. Example: a skill that knows your internal patterns. Everyone who clones the repo gets it.
* **Team scope** if the asset is about *the people who work in this repo*. Example: a review rule the Platform team applies to every repo they own. Only team members get it, and they get it in all team repos.
Repository scope is broader (anyone who clones) but bounded to one codebase; team scope is narrower (only team members) but spans all team repositories.
## Disconnecting a repository
Removing a repository from your organization detaches any active installs that targeted it. An `install.cleared` audit event is emitted for each affected asset.
# Bots
Bots are service accounts that can join teams and receive asset installs the same way users do. Use them for CI agents, scheduled loops, and review bots.
A **bot** is a non-human principal in Sleuth Skills — a service account that can be a member of teams and have assets installed against it. Bots exist so autonomous agents (CI loops, scheduled cleanup jobs, code-review workers) can pick up the same curated asset set your engineers use, without borrowing a real person's credentials.
The Bot detail view — usage chart, tabs for Overview and API Keys, and the set of teams the bot belongs to.
## Creating a bot
Click **Bots** in the left nav and choose **Create new bot**, or ask the home-page assistant ("new bot"). You'll provide:
* **Name** — a short identifier. Converted to a slug that must be unique within the organization.
* **Description** — one line on what the bot is for (e.g. "Used to fix backend bugs in python").
Bots start in **Active** status and are ready to use immediately.
## API keys
Bots authenticate via API keys. Each bot can hold multiple keys so you can rotate them without downtime.
Open the bot's **API Keys** tab to manage keys:
* **Create** — issues a new key. The full token is shown **once** at creation time — copy it into your secret manager immediately. Afterward, only a prefix and suffix are stored for identification.
* **Delete** — invalidates a key. In-flight requests using that key will fail on next call.
* **Label** — an optional human-readable label so you can tell `ci-worker-prod` apart from `ci-worker-stage`.
The token is a standard OAuth2 access token with full scope for the bot's organization.
## Adding a bot to a team
Bots join teams the same way users do. On the team page, use **Add member** and select the bot from the dropdown. Once added, every asset installed to that team resolves for the bot, so its CI run or agent loop picks up the same skills and rules the humans on the team use.
This is the recommended pattern: don't install assets directly onto a bot unless the bot has a truly unique need. Put the bot on the teams that describe its role, and let team membership drive the asset set.
## Installing directly to a bot
You *can* install an asset directly to a bot — useful for one-off bot-specific tools. From the bot's page, click **Install asset**. Direct bot installs are separate from team-membership installs and are recorded as their own entries in the audit log.
## Bot vs personal scope
A bot is **not** the same as a personal install. Personal scope is for a human engineer's own machine; bot scope is for a service account that can run unattended anywhere. Bots have API keys; personal scope does not.
## Usage tracking
The bot detail page shows a usage chart (assets invoked per day) and metrics like `1 of 6 assets used` and `0.3 avg uses / day`. This is the fastest way to see whether a bot is actually exercising the skills it has access to — a sign that either the bot is picking the wrong assets, or the assets don't fit the bot's workload.
## Deleting a bot
Deleting a bot invalidates all its API keys and clears any bot-scoped installs. Team memberships are removed automatically.
# Personal
Personal installs apply only to you. Use them to experiment with an asset before sharing it, or for tools that don't make sense for anyone else.
A **personal** install is scoped to a single user — you. It writes to your global client directory but is invisible to your teammates. Use personal scope for two things: iterating on an asset before you're ready to share it, and keeping tools that only make sense for you (shortcuts, personal preferences) out of everyone else's context.
The Personal Assets page shows what's installed just for you.
## What lives here
The **Personal** entry in the left nav shows:
* Assets installed only to you.
* Aggregate usage for those assets (assists you decide whether a personal install is worth promoting to team or org scope).
* A **Popular** section with the assets you use most.
## Installing for yourself
From an asset's detail page, click **Install asset** and pick the personal target. The CLI equivalent:
```bash
sx install my-skill --user you@example.com
```
Two important constraints:
* **Self-only.** You can only target yourself. `sx` rejects a `--user` install that doesn't match the caller's git identity — this prevents someone with write access to the vault from silently flipping an asset to "global" in a teammate's resolved lock file.
* **Resolves to the user's global directory.** A personal install behaves like an org install for *you specifically*: it lands in `~/.claude/` so you see it in every project.
## Promoting a personal asset
Most personal assets don't stay personal forever. The typical lifecycle:
1. **Personal install** while you iterate on the prompt, test the asset, and refine its description.
2. **Team install** once you've shown it works for a role or a specific group.
3. **Org install** when it's clear the whole org benefits.
Promoting means changing the install target, not re-publishing the asset. Open the asset, click **Install asset**, and pick the new target. The old personal install is cleared in the same transaction; the audit log records both events.
## Installed vs Personal
The **Installed** entry in the left nav and the **Personal** entry look similar but answer different questions:
* **Installed** shows everything that applies to *you on this machine* — including your org, team, and repository installs. It's the full picture of your context.
* **Personal** shows only assets installed specifically to your user scope. It's a strict subset of Installed.
Use Installed to audit your effective set; use Personal to manage what only you see.
# Govern
Sleuth Skills keeps an append-only audit log of every install and team change, plus pre-built dashboards that show what your org is actually using.
Once you have assets published and distributed, the question becomes: *is anyone actually using them, and can I reconstruct what happened?* Sleuth Skills answers both through the **Govern** pillar.
The Govern section — Audit Log, AI Metrics, Adoption, Usage, and Leaderboards.
## The two governance surfaces
| Surface | Question it answers | Page |
| ---------------------------------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------- |
| [Audit log](/sleuth-skills/govern/audit-log) | *Who changed what, and when?* | Installs, uninstalls, team and bot changes, asset publications — every mutation. Append-only. |
| [Usage metrics](/sleuth-skills/govern/metrics) | *Who is actually using these assets?* | Pre-built dashboards for adoption, top assets, per-team rollups, leaderboards. |
The audit log is the compliance record — it's what you reach for during an incident review or a security question. The usage dashboards are the adoption signal — they tell you which assets are earning their keep.
## Pre-built and custom dashboards
Under **Usage metrics**, Sleuth Skills ships four pre-built dashboards — **AI Metrics**, **Adoption**, **Usage**, and **Leaderboards** — plus a fifth **Start from scratch** option for building your own. You can clone any preset into a new custom dashboard and extend it, add widgets backed by PQL queries, or ask the assistant to generate persistent charts and tables from a natural-language prompt. See [Usage metrics](/sleuth-skills/govern/metrics#custom-dashboards) for the full walkthrough.
## Why both matter
Governance often means "compliance checkbox," but in Sleuth Skills the two surfaces work together to support real engineering decisions:
* A skill has low usage in the dashboard. Check the audit log — was it actually installed for the people who would benefit?
* A new rule went out org-wide and errors spiked. The audit log tells you when and who; the usage dashboard tells you whether engineers are bypassing or following it.
* Someone reports a surprising behavior. The audit log reconstructs the exact set of installs at that moment.
## Default filters
Every Govern surface defaults to the organization scope and the last 30 days. You can narrow to a specific team, repository, user, or asset type using the **Add filter** control at the top of each page. The dashboards respond immediately; the audit log supports the same filters plus free-text event search.
## Export
Both the audit log and dashboard data can be exported. The audit log has an **Export CSV** button in its header; dashboard widgets can be exported as CSV or cloned into a custom dashboard for sharing.
The CLI counterpart — `sx audit --json` and `sx stats --json` — returns the same data for git and path vaults; the Skills.new hosted vault delegates those commands to the server's aggregation API.
# Audit log
Every install, uninstall, team change, and asset mutation in Sleuth Skills is recorded in an append-only audit log. Use it for compliance reviews, incident reconstruction, and operational debugging.
The **Audit Log** records every state-changing action in your Sleuth Skills vault. It is append-only: entries cannot be edited or deleted, and the write path is inside the same transaction that mutates state, so if the audit write fails the mutation is the one that rolls back.
The Audit Log — every install, uninstall, and team change is a row. Filters narrow by event type, actor, target, or date range.
## What gets logged
The following events are recorded:
| Event | Target type | Target | Captured data |
| --------------------- | ------------ | ---------- | ------------------------------------------------------ |
| `team.created` | team | team name | `description`, `members`, `admins`, `repositories` |
| `team.updated` | team | team name | *(replaces whole team body)* |
| `team.deleted` | team | team name | *(none)* |
| `team.member_added` | team | team name | `member`, `admin` flag |
| `team.member_removed` | team | team name | `member` |
| `team.admin_set` | team | team name | `member` |
| `team.admin_unset` | team | team name | `member` |
| `team.repo_added` | team | team name | `repository` |
| `team.repo_removed` | team | team name | `repository` |
| `install.set` | installation | asset name | `kind`, plus one of `repo` / `paths` / `team` / `user` |
| `install.cleared` | installation | asset name | `kind`, `reason` (e.g. `team_deleted`) |
Asset publications, bot lifecycle changes, and API-key operations are logged with analogous events.
## Cascade events
Some mutations trigger automatic follow-up events:
* Deleting a **team** emits one `install.cleared` per team-scoped asset that referenced it, with `reason = "team_deleted"`.
* Disconnecting a **repository** emits one `install.cleared` per repo-scoped asset.
* Deleting a **bot** clears bot-scoped installs and invalidates its API keys.
Cascades are what make the audit log a reliable incident-reconstruction tool: you can always explain why an asset stopped resolving without a separate "why did this happen" investigation.
## No-op skipping
Repeating a mutation that already matches the current state is a silent no-op: no audit event is emitted. Re-adding a team member who is already on the team, granting admin to someone who already has it, or re-installing an asset with its current scope all produce zero audit events. This keeps the log free of churn from idempotent retries — you can safely re-run scripts without flooding history.
## Querying the log
In the UI, the Audit Log page supports:
* **Full-text search** across event descriptions.
* **Add filter** for event type, actor (user or bot), target (asset / team / repo), and date range.
* **Expand all** to show the detailed payload for each event.
* **Export CSV** for the filtered view.
From the CLI:
```bash
sx audit # last 7 days
sx audit --since 30d # widen the window
sx audit --since all # whole log
sx audit --actor alice@acme.com # one actor
sx audit --event install.set # one event type
sx audit --target code-reviewer # one asset / team
sx audit --since 7d --json # machine-readable
sx audit --limit 20 # cap output
```
Filters are AND-combined: `--actor alice@acme.com --event install.set --target code-reviewer --since 30d` returns only the set where all four conditions hold.
## Storage format
The CLI exposes the same JSONL format the vault writes internally. Each line is a self-contained JSON object:
```json
{
"ts": "2026-04-17T10:04:12.445Z",
"actor": "alice@acme.com",
"event": "install.set",
"target_type": "installation",
"target": "code-reviewer",
"data": { "kind": "team", "team": "platform" }
}
```
For git and path vaults, events land in `.sx/audit/YYYY-MM.jsonl` under the vault root. For the Skills.new hosted vault, events are written to the server's audit store and retrieved via the same `sx audit` commands.
## Compliance considerations
The audit log is designed to be the authoritative record for:
* **Security reviews** — who installed what hook, when, and on which target.
* **Incident response** — reconstructing the installed asset set at a past moment.
* **Access reviews** — who currently has admin on which teams, and when they were promoted.
Treat the log as evidence: retain it per your company's policy, export regularly, and do not modify the underlying store. The append-only guarantee is enforced at the write layer (vault flock for local vaults, transactional insert for Skills.new).
# Usage metrics
Sleuth Skills ships pre-built dashboards for adoption, usage, and leaderboards. They tell you whether assets are actually being picked up, by whom, and in which repositories.
Publishing assets is the easy part; knowing whether anyone is using them is where real governance kicks in. Sleuth Skills captures a usage event every time an installed asset runs, and rolls those events into four pre-built dashboards — **AI Metrics**, **Adoption**, **Usage**, and **Leaderboards** — plus a custom-dashboard builder for questions the defaults don't answer.
The Adoption dashboard — new users over time, user adoption percentage, and repo/team adoption breakdowns.
## The four default dashboards
| Dashboard | Question it answers |
| ---------------- | ----------------------------------------------------------------------------------------------------------- |
| **AI Metrics** | High-level KPIs: total invocations, token cost, human vs bot, trend over time. |
| **Adoption** | What percentage of your users, teams, and repositories have any usage in the window? Who has newly adopted? |
| **Usage** | Which assets are used most (by count and by token cost)? What's the raw volume over time? |
| **Leaderboards** | Ranked lists — top assets, top actors, top teams. |
Each dashboard defaults to the last 30 days; use the date range and **Add filter** controls to narrow.
## What counts as a usage event
`sx` records a usage event every time it installs an asset. Clients that integrate with `sx` (Claude Code, Cursor, and others per the compatibility matrix) emit events when an installed asset runs — a skill loading in a session, a command being invoked, an MCP tool call, a hook firing.
Each event carries:
```json
{
"ts": "2026-04-17T10:04:12.445Z",
"actor": "alice@acme.com",
"asset_name": "code-reviewer",
"asset_version": "1.2.3",
"asset_type": "skill"
}
```
Asset-specific payloads add extra fields — for example, `tool_name`, `duration_ms`, and `success` on MCP tool-call events.
## What the dashboards show
The Usage dashboard — top assets by usage and by token cost, plus a time series.
**Adoption metrics:**
* **User adoption** — what percentage of org members had any recorded usage in the window.
* **Team adoption** — same breakdown per team (e.g. `platform 3/5 = 60%`).
* **Repository adoption** — how many connected repos had any `sx install` run during the window.
* **New user adoption over time** — first-time adopters per day.
**Usage metrics:**
* **Top assets by usage** — raw invocation count per asset.
* **Top assets by token cost** — cumulative tokens consumed, useful for cost attribution.
* **Skills / Agents / Commands usage over time** — time series, filterable by type.
* **Top actors** — humans and bots ranked by invocations.
**AI metrics:**
* Bot-vs-human split.
* Token cost trend.
* Invocation volume by asset type.
## Filtering and drill-down
Every widget respects the page-level filters (date range, asset type, team, repository, user). Clicking a row in a leaderboard or a segment in a chart drills into the underlying filter — click the Backend team's adoption number and the dashboard rescopes to that team's activity.
## Custom dashboards
The four default dashboards answer the most common questions, but most teams eventually want to slice usage their own way. Click **Create Dashboard** in the top-right of any dashboard page to build your own.
The Create Dashboard picker. Start from a blank canvas or clone one of the four preset templates.
You pick one of five starting points:
| Template | Starts you with |
| ---------------------- | ------------------------------------------------------------------------------ |
| **Start from scratch** | A blank canvas. Add widgets one at a time. |
| **AI Metrics** | Copy of the AI Metrics dashboard — KPIs, bot-vs-human, token cost trend. |
| **Adoption** | Copy of the Adoption dashboard — user/team/repo adoption + newcomers. |
| **Usage** | Copy of the Usage dashboard — top assets by usage and token cost, time series. |
| **Leaderboards** | Copy of the Leaderboards dashboard — ranked assets, actors, teams. |
Every widget is backed by a PQL query (the Sleuth Skills query language for usage data), so the cloned dashboards are fully editable: rename widgets, change time windows, add filters, or swap the underlying query entirely.
### AI-generated widgets
You can also add widgets by asking the assistant. Describe what you want — "a chart showing MCP tool-call duration by bot over the last 90 days" — and the assistant researches what's available in your usage data, writes the PQL, and creates a persistent widget on your dashboard. The widget keeps the generated query so it re-runs automatically whenever the dashboard loads — it's a real widget, not a one-off answer.
This is the fastest way to prototype a new dashboard: start from a preset, ask the assistant for the two or three custom widgets you're missing, and save.
## CLI access
The same data is available via `sx stats`:
```bash
sx stats # last 7 days
sx stats --since 30d # widen the window
sx stats --since all # lifetime totals
sx stats --assets # per-asset view only
sx stats --teams # per-team view only
sx stats --since 30d --json # machine-readable
```
`sx stats --json` returns:
```json
{
"since": "2026-03-18T00:00:00Z",
"total_events": 127,
"assets": [
{ "AssetName": "code-reviewer", "TotalUses": 42, "UniqueActors": 17 }
],
"teams": [
{ "name": "platform", "member_count": 5, "active_members": 3, "adoption_pct": 60.0 }
],
"top_actors": [
{ "Actor": "alice@acme.com", "TotalUses": 9 }
]
}
```
Use this in CI or scheduled jobs to pull weekly adoption snapshots into Slack, BI tools, or a nightly digest.
## Fault tolerance
Usage events are best-effort: a malformed event line is logged and skipped so one bad event doesn't drop a batch of good ones. Events with unparseable timestamps are stamped with the Unix epoch so they fall outside recent-window filters rather than skewing them; `--since all` still counts them.
For git vaults, usage events flush lazily — they ride along with the next management commit rather than generating one commit per install, which keeps history clean without losing durability across CLI runs. The Skills.new hosted vault ingests events synchronously through its usage endpoint.
# API
Sleuth Skills exposes both a REST API and a GraphQL API at app.skills.new. Use them to drive the same actions your engineers take in the web app — authoring assets, resolving installs, downloading bun
Sleuth Skills ([skills.new](https://skills.new)) is API-first. Every action you can take in the web UI — listing the assets installed for a user, downloading a skill bundle, creating a bot, recording a usage event — is also available over HTTP.
There are two surfaces:
| Surface | Base URL | Use it for |
| ----------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| **REST** | `https://app.skills.new/api/skills/` | Asset distribution: lock-file resolution, version listing, bundle download, upload, profile selection, usage reporting. This is what `sx` calls. |
| **GraphQL** | `https://app.skills.new/graphql` | Everything else: managing assets, bots, profiles, installations, change requests, audit log, AI metrics. This is what the web UI calls. |
Both surfaces share the same authentication model — an org-scoped credential identifies the caller, and the credential type (bot API key, personal access token, or browser session) determines what the caller can see and do.
## Quick orientation
* **Bot API keys** are the recommended credential for unattended automation. Issue one per bot, scope it to that bot's teams, and rotate without downtime. See [Authentication](/sleuth-skills/api/authentication).
* **Personal access tokens** are for scripts and CLI tools that should act as *you* — issue them from your user settings. See [Authentication](/sleuth-skills/api/authentication#personal-access-tokens).
* **`sx` itself is just a REST client.** Everything `sx install`, `sx update`, and `sx vault` do is a documented call against the REST API on this page — see [REST API](/sleuth-skills/api/rest).
* **GraphQL is introspectable.** Open while signed in to explore the full schema in GraphiQL. See [GraphQL API](/sleuth-skills/api/graphql) for the high-level shape.
## Where to go next
* [Authentication](/sleuth-skills/api/authentication) — API keys, bot keys, and how to pass them.
* [REST API](/sleuth-skills/api/rest) — every endpoint `sx` and CI agents use.
* [GraphQL API](/sleuth-skills/api/graphql) — the management surface for assets, bots, profiles, and metrics.
{% hint style="info" %}
This API is for the **Sleuth Skills** product at skills.new. If you're looking for the DORA / deployment-tracking API at app.sleuth.io, see [Sleuth API](/sleuth-dora/sleuth-api) instead.
{% endhint %}
# Authentication
Every Skills.new API call is authenticated and scoped to a single organization. The credential you pass determines whether the caller acts as a user, an org-level service account, or a bot.
Every call to `https://app.skills.new` is authenticated. There is no public/anonymous surface — even the lock-file lookup requires an authenticated principal so it can resolve which assets that principal should receive.
Skills.new accepts three credential types:
| Credential | Header | Acts as | Best for |
| ------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
| **Bot API key** | `Authorization: Bearer ` | A specific [bot](/sleuth-skills/distribute/bots) — inherits the assets installed to the bot's teams and direct installs. | Unattended automation: CI runs, scheduled loops, review workers. |
| **Personal access token** | `Authorization: Bearer ` | The user that created the token, with their RBAC role. | Personal scripts, GraphQL exploration, and CLI tools that should act *as you*. |
| **User session** | Browser session cookie | The signed-in user, with their RBAC role. | The web UI and GraphiQL. |
## Bot API keys (recommended)
Bots are first-class principals in Skills.new. A bot can join teams, have assets installed against it, and authenticate via one or more API keys. This is the credential type you want for any non-human caller — CI agents, scheduled loops, review bots, the `sx` running on a build worker.
### Issuing a key
In the web UI, open the bot's detail page and click the **API Keys** tab. Choose **Create**, give the key a label (e.g. `ci-worker-prod`), and copy the token. **The full token is only shown once at creation time** — afterward, the UI only displays the prefix and suffix for identification.
You can also create a key over GraphQL (you must be a user with admin permissions, not a bot):
```graphql
mutation CreateKey($botId: ID!, $label: String) {
createBotApiKey(botId: $botId, label: $label) {
rawToken # shown ONCE — copy immediately into a secret manager
apiKey {
id
label
maskedToken # e.g. "abc12345...wxyz" for later identification
createdAt
}
}
}
```
The raw token is an [OAuth2 access token](https://datatracker.ietf.org/doc/html/rfc6749#section-1.4) with the `ALL` scope, valid for 20 years. Each bot can hold multiple keys so you can rotate without downtime — issue a new key, redeploy CI to use it, then delete the old one.
### Passing a key
Send the token in the `Authorization` header with the `Bearer` scheme:
```bash
curl https://app.skills.new/api/skills/sx.lock \
-H "Authorization: Bearer YOUR_BOT_API_KEY"
```
For GraphQL:
```bash
curl https://app.skills.new/graphql \
-H "Authorization: Bearer YOUR_BOT_API_KEY" \
-H "Content-Type: application/json" \
-d '{"query": "{ bot(slug: \"my-bot\") { name installedSkills { slug } } }"}'
```
When the server authenticates a bot key, it sets both the bot context and the org context on the request — so calls like `/api/skills/sx.lock` return the lock file scoped to **that bot's** teams and direct installs, not a user's.
### Revoking a key
Open the bot's **API Keys** tab and click **Delete** on the row, or call the GraphQL mutation:
```graphql
mutation { deleteBotApiKey(keyId: "BK...") { ok } }
```
Revocation is immediate — in-flight requests using that key will fail on next call. Deleting the bot itself revokes all of its keys.
{% hint style="warning" %}
**Never share a bot key across bots.** Each bot's keys grant access to that bot's installed asset set. If two CI workers need different asset sets, give them different bots; if they need the same set, put them on the same teams and give each its own key.
{% endhint %}
## Personal access tokens
A **personal access token** (PAT) is an OAuth2 token tied to your individual user account, scoped to a single organization. Use one when you want a script or CLI tool to act *as you* — it inherits your RBAC role and your asset installs. PATs are the recommended credential for ad-hoc GraphQL queries, personal automation, and anywhere you'd otherwise be tempted to copy a session cookie.
### Issuing a PAT
In the web UI, open the user settings (top-right avatar) and choose **Personal Access Tokens**. Click **Add access token**, give the token a descriptive name (e.g. `local-graphiql`, `home-laptop-sx`), and **copy the value when it's shown** — the full token is only displayed once at creation time. Afterward, the table shows an obfuscated value (`••••••••…`) for identification.
You can also create one over GraphQL while signed in:
```graphql
mutation CreatePAT($label: String!) {
createPersonalToken(label: $label) {
token # the raw token — returned ONCE
errors
}
}
```
```graphql
{
user {
personalTokens(first: 50) {
edges { node { id label token created expires applicationName } }
}
}
}
```
The `token` field on the listing is camouflaged; only the response of `createPersonalToken` returns the raw value.
### Passing a PAT
PATs use the standard `Bearer` scheme:
```bash
curl https://app.skills.new/graphql \
-H "Authorization: Bearer YOUR_PERSONAL_TOKEN" \
-H "Content-Type: application/json" \
-d '{"query": "{ user { display } vault { assets(first: 5) { nodes { name } } } }"}'
```
They work the same way on REST:
```bash
curl https://app.skills.new/api/skills/sx.lock \
-H "Authorization: Bearer YOUR_PERSONAL_TOKEN"
```
Because the token represents *you*, the response is scoped to *your* installs — `sx.lock` returns the assets installed to you, your teams, and the repositories you have access to.
### PAT scope and lifetime
* **Scope** — a PAT acts with the issuing user's RBAC role in the org it was created in. It cannot escalate beyond what you can do in the web UI, and it cannot be used against a different org.
* **Lifetime** — 10 years. Rotate by issuing a new one and deleting the old one.
* **Revocation** — immediate when you click **Delete** in the settings UI (or call `deletePersonalToken(tokenId:)`).
### When to use a PAT vs a bot key
| Use a **PAT** when... | Use a **bot API key** when... |
| ----------------------------------------------------------- | --------------------------------------------------------------------- |
| The work runs interactively or under your account. | The work runs unattended in CI, a scheduled loop, or a review worker. |
| The assets should resolve to **your** team/repo membership. | The assets should resolve to a service account's team membership. |
| Revoking it shouldn't affect your colleagues. | Revoking it shouldn't affect any individual person. |
| You'd otherwise be copying a browser session cookie. | You'd otherwise be sharing a human's PAT across CI runners. |
{% hint style="warning" %}
**Don't share PATs across people or environments.** A PAT carries one user's permissions across one organization — treat it like a password. For shared automation, create a [bot](/sleuth-skills/distribute/bots) and use a bot key.
{% endhint %}
## User sessions
In the browser, the web UI authenticates via session cookie. GraphiQL at inherits that cookie automatically, so once you're signed in at [skills.new](https://skills.new) you can explore the API interactively without setting any headers.
For programmatic access from a tool running on your behalf — including the [`sx`](https://github.com/sleuth-io/sx) CLI — use a [PAT](#personal-access-tokens). `sx login` walks you through obtaining one and storing it locally.
## Picking the right credential
* **Building a CI integration or unattended automation?** Create a [bot](/sleuth-skills/distribute/bots), add it to the teams whose assets it needs, issue one bot API key per environment.
* **Writing a personal script or exploring the API from cURL?** Issue a PAT from your user settings and delete it when the script is retired.
* **Running the `sx` CLI on your own machine?** Use `sx login` — it manages a PAT for you.
* **Just clicking around in GraphiQL?** Sign in at [skills.new](https://skills.new) and your session cookie is all you need.
# REST API
REST endpoints under /api/skills/ on app.skills.new. These are what sx install, sx update, and CI agents call to resolve and download assets.
The REST API at `https://app.skills.new/api/skills/` is the **asset-distribution surface**. It is what the [`sx`](https://github.com/sleuth-io/sx) CLI uses to resolve which assets a caller should receive, download their bundles, and report usage. CI agents and review bots use it the same way `sx` does, just authenticated as a bot instead of a user.
For management actions (creating bots, editing assets, browsing the audit log, etc.) use the [GraphQL API](/sleuth-skills/api/graphql) instead.
## Conventions
* **Base URL:** `https://app.skills.new`
* **Auth:** every endpoint requires a credential — see [Authentication](/sleuth-skills/api/authentication). Bot API keys are passed with `Authorization: Bearer `.
* **Org scoping:** the credential identifies one organization. There is no `orgSlug` parameter in the URL.
* **Status codes:** `2xx` for success, `400` for invalid input, `401` if the credential is missing or invalid, `404` if the asset/version doesn't exist, `405` for the wrong HTTP method.
* **Caching:** the lock file uses `ETag` / `If-None-Match`; immutable asset bundles set `Cache-Control: public, max-age=31536000, immutable`. Send the `If-None-Match` header on your repeat lock-file requests to get a `304 Not Modified`.
## Endpoint map
| Method | Path | Purpose |
| ------ | --------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `GET` | `/api/skills/sx.lock` | Resolve the full set of assets for the caller. |
| `GET` | `/api/skills/sx.profiles` | List the org's [skill profiles](/sleuth-skills/manage/skills) and the caller's active one. |
| `POST` | `/api/skills/sx.profiles/active` | Set the caller's active skill profile. |
| `GET` | `/api/skills/assets/{name}/list.txt` | List published versions of an asset. |
| `GET` | `/api/skills/assets/{name}/{version}/metadata.toml` | Get the metadata for a specific version. |
| `GET` | `/api/skills/assets/{name}/{version}/{filename}` | Download the asset bundle (a `.zip`). |
| `POST` | `/api/skills/assets` | Upload a new asset version. |
| `POST` | `/api/skills/usage` | Report asset-usage events (JSONL). |
| `POST` | `/api/skills/ai-query/stream` | Stream an AI query against an integration. |
The sections below cover each endpoint.
***
## Lock file
```
GET /api/skills/sx.lock
```
Returns a TOML lock file enumerating every asset the caller should have installed, together with the version pinned for them and a download URL for each. This is the **single source of truth** for what should be on disk after `sx install`.
The lock content is scoped to the caller automatically:
* **User credential** → assets installed to the user, the user's teams, the user's repositories, and the org-wide installs.
* **Bot credential** → assets installed to the bot, the bot's teams, and the org-wide installs.
### Headers
* `If-None-Match: ""` — recommended. Returns `304 Not Modified` if the lock hasn't changed.
* `x-session-id: ` — optional. Tagged onto the audit log entry so you can correlate a sequence of calls.
* `user-agent: ` — recorded for usage statistics.
### Example
```bash
curl https://app.skills.new/api/skills/sx.lock \
-H "Authorization: Bearer YOUR_BOT_API_KEY" \
-H 'If-None-Match: "abc123def456"'
```
Returns `200 OK` with `Content-Type: application/toml` (the lock file) or `304 Not Modified`. The `ETag` response header changes whenever the resolved asset set changes for that caller.
Every successful call is recorded in the [audit log](/sleuth-skills/govern/audit-log) as a `LOCK_FILE_ACCESSED` event with the user agent, client IP, and (for bots) the bot slug and key prefix.
***
## Skill profiles
[Skill profiles](/sleuth-skills/manage/skills) are named subsets of the available asset set — a user (or bot) picks an active profile and the lock file resolves against that profile.
### List profiles
```
GET /api/skills/sx.profiles
```
```json
{
"profiles": [
{"title": "Backend", "slug": "backend", "description": "Python/Django assets"},
{"title": "Frontend", "slug": "frontend", "description": "Vue/Nuxt assets"}
],
"active": "backend"
}
```
### Set active profile
```
POST /api/skills/sx.profiles/active
Content-Type: application/json
{"slug": "frontend"}
```
Pass `{"slug": null}` to clear the active profile. Returns the activated profile, or `null` if cleared.
***
## Asset bundles
### List versions
```
GET /api/skills/assets/{asset_name}/list.txt
```
Returns plain text — one published version number per line, newest last. Use it to find the latest version available for an asset before fetching metadata or the bundle.
```
1
2
3
```
### Get version metadata
```
GET /api/skills/assets/{asset_name}/{asset_version}/metadata.toml
```
Returns the TOML metadata for the version: name, description, asset type, dependencies, etc. Bundles are immutable per version, so the response is heavily cached (`Cache-Control: public, max-age=31536000, immutable`).
### Download a bundle
```
GET /api/skills/assets/{asset_name}/{asset_version}/{filename}
```
Returns the zip file. The `{filename}` segment is for HTTP semantics (so browsers and tools pick a sensible filename); the server resolves the bundle from `{asset_name}` and `{asset_version}` alone. Convention is `{name}-{version}.zip`.
```bash
curl -O https://app.skills.new/api/skills/assets/my-skill/3/my-skill-3.zip \
-H "Authorization: Bearer YOUR_BOT_API_KEY"
```
Each successful download:
1. Queues an install-tracking event (powers the [Adoption](/sleuth-skills/govern/metrics) dashboard).
2. Writes a `DOWNLOADED` event to the [audit log](/sleuth-skills/govern/audit-log) with the caller, the user agent, and (for bots) the key prefix.
### Upload a new version
```
POST /api/skills/assets
Content-Type: multipart/form-data
```
Upload a packaged `.zip` to create a new version of an asset (or the first version of a new asset).
Form fields:
| Field | Required | Description |
| -------------------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------- |
| `file` | yes | The `.zip` bundle. Must contain a `metadata.toml` with `name` and `version`. |
| `name` | no | Asset slug. Falls back to `metadata.toml` or the filename. |
| `version` | no | Version string. Used for validation against `metadata.toml`. |
| `type` | no | Asset type — `skill` (default), `agent`, `rule`, `command`, `hook`, `mcp_server`, `plugin`. |
| `installations` | no | JSON array of `{entity_type, entity_id}` objects describing where to install (org, team, repository, user, bot). |
| `owned_by` | no | Legacy fallback if `installations` isn't given — `organization`, `user`, or `repository`. |
| `repository_owner` / `repository_name` | conditional | Required if `owned_by="repository"`. |
Returns `201 Created` with:
```json
{
"success": true,
"asset": {
"name": "my-skill",
"version": "3",
"url": "https://app.skills.new/api/skills/assets/my-skill/3/my-skill-3.zip",
"skill_id": "AS...",
"is_first_version": false
}
}
```
***
## Usage reporting
```
POST /api/skills/usage
Content-Type: application/x-ndjson
```
Reports asset-usage events. The body is JSONL — one JSON object per line — with each line describing a single invocation:
```jsonl
{"asset_name": "github-mcp", "asset_version": "1.2.3", "asset_type": "mcp"}
{"asset_name": "django-admin", "asset_version": "2.0.0", "asset_type": "skill"}
```
Returns `204 No Content` once events are queued (processing is async). An empty body is also a valid `204`. Invalid JSONL returns `400`.
Events feed the [Govern / Usage](/sleuth-skills/govern/metrics) dashboards.
***
## AI query stream
```
POST /api/skills/ai-query/stream
Content-Type: application/json
```
Streams the result of a natural-language query against an integration provider (used by the `sx` AI-query subcommands).
Request:
```json
{
"query": "Find open PRs blocked on review",
"provider": "GITHUB",
"context": {
"repo_url": "https://github.com/owner/repo",
"branch": "main",
"commit_sha": "abc1234..."
}
}
```
* `provider` is one of the configured integration providers — `GITHUB`, `CIRCLECI`, `LINEAR`, etc.
* `context.repo_url` is required. `branch` and `commit_sha` may be empty strings.
Returns `Content-Type: text/event-stream`. Each event is a serialized tool call, intermediate result, or — as the final frame — `{"type": "done", "result": {...}}`. Errors arrive as `{"type": "error", "error": "..."}`.
***
## A complete `sx`-style flow
The typical machine flow when an agent comes online (and roughly what `sx install` does):
```bash
TOKEN="YOUR_BOT_API_KEY"
BASE="https://app.skills.new"
# 1. Resolve the asset set for this caller.
LOCK=$(curl -s "$BASE/api/skills/sx.lock" -H "Authorization: Bearer $TOKEN")
# 2. For each asset in the lock, download its bundle. (Pseudo-loop.)
for asset in $(echo "$LOCK" | parse-lock); do
name="$(echo "$asset" | jq -r .name)"
ver="$(echo "$asset" | jq -r .version)"
curl -s -O "$BASE/api/skills/assets/$name/$ver/$name-$ver.zip" \
-H "Authorization: Bearer $TOKEN"
done
# 3. Periodically report what was invoked.
curl -s "$BASE/api/skills/usage" \
-H "Authorization: Bearer $TOKEN" \
--data-binary @- <<'JSONL'
{"asset_name": "my-skill", "asset_version": "3", "asset_type": "skill"}
JSONL
```
In practice, prefer running `sx` itself with a bot's API key — it implements the caching, retry, and on-disk layout correctly.
# GraphQL API
The GraphQL API at app.skills.new/graphql is the management surface used by the web UI — assets, bots, profiles, installations, change requests, audit log, and AI metrics.
The GraphQL API is the **management surface** for Sleuth Skills. It's the same API the web UI uses, which means anything you can do in the UI you can also script — create bots, issue API keys, install assets to teams, browse the audit log, query AI metrics, approve change requests, and so on.
For the **asset-distribution** side (lock-file resolution, bundle downloads, usage reporting) use the [REST API](/sleuth-skills/api/rest) instead. The REST surface is what `sx` and CI agents call; GraphQL is for control-plane operations.
## Endpoints
| Path | Purpose |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| `https://app.skills.new/graphql` | Main GraphQL endpoint. Serves [GraphiQL](https://github.com/graphql/graphiql) when opened in a browser. |
| `https://app.skills.new/graphql/batch` | Same schema, accepts an array of operations to run as a single batched request. |
The endpoint is **introspectable** — open it in a signed-in browser tab to explore the full schema interactively in GraphiQL. The schema is large; the sections below give you orientation, but GraphiQL is the source of truth.
## Authentication
GraphQL accepts the same credentials as REST — see [Authentication](/sleuth-skills/api/authentication). Set the `Authorization` header:
```bash
# Bot API key (CI / unattended) or personal access token (acts as you)
curl https://app.skills.new/graphql \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{"query": "{ user { display } }"}'
```
In GraphiQL the session cookie is used automatically. For automation use a bot API key; for personal scripts use a [personal access token](/sleuth-skills/api/authentication#personal-access-tokens).
## What's in the schema
The schema covers the full Skills.new feature set. The top-level `Query` and `Mutation` fields relevant to Skills are below — for everything else (DORA metrics, deployments, dashboards, etc.) see GraphiQL or the [DORA GraphQL docs](/sleuth-dora/sleuth-api/graphql-examples).
### Vault — the asset catalog
```graphql
type Query {
vault: Vault!
asset(id: ID!): VaultAsset
installedAssets(
userId: ID
botId: ID
repositoryId: ID
assetType: String
term: String
): InstalledAssetsResponse!
catalogSources(assetType: String): [CatalogSource!]!
catalogEntries(source: String!, search: String, limit: Int, offset: Int): [CatalogEntry!]
}
```
`vault` is the entry point for every asset in your org. `asset(id:)` looks one up by GID. `installedAssets` resolves the assets that should be on disk for a specific user, bot, or repository — it's the GraphQL equivalent of the REST lock file, returning structured data instead of TOML.
### Asset management
```graphql
type Mutation {
createAsset(input: CreateAssetInput!): CreateAssetMutation
updateAsset(input: UpdateAssetInput!): UpdateAssetMutation
deleteAsset(id: ID!): DeleteAssetMutation
toggleAssetStatus(id: ID!): ToggleAssetStatusMutation
activateSkillVersion(assetId: ID!, versionNumber: Int!): ActivateSkillVersionMutation
renameAsset(input: RenameAssetInput!): RenameAssetMutation
refreshGitAsset(assetId: ID!): RefreshGitAssetMutation
}
```
### Installation targeting
```graphql
type Mutation {
setAssetInstallations(input: SetAssetInstallationsInput!): SetAssetInstallationsMutation
removeAssetInstallations(input: RemoveAssetInstallationsInput!): RemoveAssetInstallationsMutation
importFromCatalog(source: String!, key: String!, installations: String): ImportFromCatalogMutation
importAssetAsManaged(assetId: ID!): ImportAssetAsManagedMutation
}
```
`SetAssetInstallationsInput` takes a list of `(entity_type, entity_id)` pairs — `organization`, `team`, `repository`, `user`, or `bot` — so you can pin an asset to a specific distribution target.
### Bots and bot API keys
```graphql
type Query {
bots: [ManagedBot!]
bot(slug: String, id: ID): ManagedBot!
}
type Mutation {
createBot(input: CreateBotInput!): CreateBotMutation
updateBot(input: UpdateBotInput!): UpdateBotMutation
deleteBot(id: ID!): DeleteBotMutation
createBotApiKey(botId: ID!, label: String): CreateBotApiKeyMutation
deleteBotApiKey(keyId: ID!): DeleteBotApiKeyMutation
installSkillToBot(botId: ID!, skillId: ID!): InstallSkillToBotMutation
uninstallSkillFromBot(botId: ID!, skillId: ID!): UninstallSkillFromBotMutation
}
```
`ManagedBot` exposes the bot's slug, teams, installed skills, and `apiKeys: [BotApiKey!]`. Each `BotApiKey` exposes `id`, `label`, `maskedToken` (e.g. `abc12345...wxyz`), and `createdAt`. The raw token is only returned by `createBotApiKey` and only at creation time — copy it then or lose it.
### Pull requests and change requests
```graphql
type Query {
assetPullRequest(id: ID!): SkillPullRequest!
assetPullRequests(
assetId: ID
status: SkillPullRequestStatusEnum
creationType: SkillPullRequestCreationTypeEnum
search: String
): SkillPullRequestConnection!
changeRequests(
status: ChangeRequestStatusEnum
changeType: ChangeRequestTypeEnum
search: String
): ChangeRequestConnection!
installationRequests(status: String): [AssetInstallationRequest!]!
}
type Mutation {
createAssetPullRequest(input: CreateAssetPullRequestInput!): CreateAssetPullRequestMutation
addAssetPullRequestFileChange(input: AddAssetPullRequestFileChangeInput!): AddAssetPullRequestFileChangeMutation
mergeAssetPullRequest(pullRequestId: ID!): MergeAssetPullRequestMutation
closeAssetPullRequest(pullRequestId: ID!): CloseAssetPullRequestMutation
createInstallationRequest(input: CreateInstallationRequestInput!): CreateInstallationRequestMutation
approveInstallationRequest(input: ReviewInstallationRequestInput!): ApproveInstallationRequestMutation
rejectInstallationRequest(input: ReviewInstallationRequestInput!): RejectInstallationRequestMutation
}
```
### Profiles
```graphql
type Query {
profiles(name: String, first: Int, after: String): VaultAssetProfileConnection!
profile(id: ID!): VaultAssetProfile
}
type Mutation {
createAssetProfile(input: CreateAssetProfileInput!): CreateAssetProfileMutation
updateAssetProfile(input: UpdateAssetProfileInput!): UpdateAssetProfileMutation
deleteAssetProfile(input: DeleteAssetProfileInput!): DeleteAssetProfileMutation
addAssetsToProfile(input: ModifyAssetProfileAssetsInput!): AddAssetsToProfileMutation
removeAssetsFromProfile(input: ModifyAssetProfileAssetsInput!): RemoveAssetsFromProfileMutation
setActiveSkillProfile(skillProfileId: ID): SetActiveSkillProfileMutation
}
```
### Audit log and metrics
```graphql
type Query {
assetAuditLog(
search: String
assetTypes: [AssetType!]
events: [AssetEventType!]
targetId: ID
actorId: Int
dateFrom: DateTime
dateTo: DateTime
actorType: String # "user", "bot", "system"
): AssetAuditEventConnection!
}
```
Filter by actor, event type, target, or date range — this is what powers the **Govern → Audit log** page.
### Personal access tokens
```graphql
type User {
personalTokens(first: Int, after: String): AccessTokenConnection!
}
type Mutation {
createPersonalToken(label: String!): CreatePersonalTokenMutation
deletePersonalToken(tokenId: ID!): DeletePersonalTokenMutation
}
```
`createPersonalToken` returns the raw token **once**, in the `token` field of the response — copy it then or lose it. Subsequent reads of `user.personalTokens` return a camouflaged value for identification only. See [Authentication](/sleuth-skills/api/authentication#personal-access-tokens) for the full flow.
## Example: create a bot and issue a key
```graphql
mutation CreateBotWithKey($name: String!, $teamIds: [ID!]) {
createBot(input: {name: $name, teamIds: $teamIds}) {
bot {
id
slug
teams { name }
}
rawToken # API key for the auto-created "Default key"
}
}
```
```json
{
"name": "CI worker (prod)",
"teamIds": ["TM...", "TM..."]
}
```
`rawToken` is the bot's first API key — store it immediately. Issue additional keys with `createBotApiKey(botId:, label:)`.
## Example: list a bot's installed assets
```graphql
{
bot(slug: "ci-worker-prod") {
name
teamMemberships { team { name } assetCount }
installedSkills {
slug
assetType
description
usageCount
}
apiKeys {
maskedToken
label
createdAt
}
}
}
```
## Batching
Send multiple operations in one round-trip via `/graphql/batch`. The body is a JSON array of `{query, variables, operationName}` objects, and the response is an array of results in the same order. See [Query batching](/sleuth-dora/sleuth-api/query-batching) — the mechanics are identical to the DORA endpoint.
## Errors
GraphQL errors are returned in the standard `errors` array of the response body, even on `200 OK`. Authentication failures return `401` at the HTTP level; permission failures (e.g. a bot key trying to mutate org settings) return a GraphQL error with a `PermissionDenied` message inside a `200 OK`.
# Getting started
You can be up and running with Sleuth in less than 5 minutes. Create an account, connect to a code repository, discovery your DORA performance and start tracking your deploys.
## Track and improve DORA metrics
You can't improve what you don't measure. Sleuth lets you do both!
## Setup [deployment & metrics tracking](/sleuth-dora/modeling-your-deployments) in 5 minutes
Getting started with **Sleuth** takes about 5 minutes. Link your code and Sleuth will instantly begin to track your deploys.
* [ ] **Create a Sleuth account.** You'll need one to get started. You can create an account via OAuth using your Google, GitHub, or Bitbucket account. [Signup](https://app.sleuth.io/account/signup/) is instant and setup takes less than 5 minutes!
* [ ] **Setup your** **organization and project**. You can make as many projects as needed to model your deployments, but Sleuth creates one for you by default. Each project contains one default environment: *Production*.
* [ ] **Connect your code**. Sleuth supports GitHub, Bitbucket, GitLab and [more](/sleuth-dora/integrations-1/code-deployment)! Sleuth will instantly analyze your commits, pull requests and authors, calculate your team's last 30 days of deploy frequency and change lead time, and highlight meaningful, actionable data.
* [ ] **Let us know when you deploy**. Sleuth works with your tools and **every deployment system**. Using any of our native [CI/CD integrations](/sleuth-dora/integrations-1/builds) or our light-weight, [webhook-based deploy registrations](/sleuth-dora/modeling-your-deployments/code-deployments/how-to-register-a-deploy), you'll be up and running in moments with your existing tooling.
{% tabs %}
{% tab title="Step 1 - Sign up" %}
**Sign up to Sleuth**
{% endtab %}
{% tab title="Step 2 - Setup" %}
**Setup your project and connect your source provider**
{% endtab %}
{% tab title="Step 3 - Connect code " %}
**Configure your first code deployment**
{% endtab %}
{% tab title="Step 4 - Integrations" %}
**Connect more integrations to realize the power of Sleuth**
Gain team deploy notifications, personal message when your changes ship and powerful slash commands to *sleuth* important information about your deploys by **enabling Slack**
Verify the health of your deploys by integrating your Observability and setting up deploy Impact
[Learn more](/sleuth-dora/integrations-1/about-integrations) about integrations.
{% endtab %}
{% tab title="Step 5 - Done!" %}
**Accelerate your team today!**
{% endtab %}
{% endtabs %}
# Navigating Sleuth
Sleuth's global navigation provides quick and easy access to everything from global dashboards and settings to Project-specific and Team-specific views.
## Top Navigation Bar for Global Items
The horizontal bar across the top of Sleuth's UI provides access to views that global across your organization (i.e. views that not specific to a particular Project or Team).
These include administrative views (e.g. [Organization Settings](https://github.com/sleuth-io/sleuth-gitbook-docs/blob/master/broken-reference/README.md), [User Settings](/sleuth-dora/settings/account), Documentation, and Help...), as well as top-level lenses into your organization's engineering efficiency information (e.g. [Search](/sleuth-dora/modeling-your-deployments/organization/search), [Status](/sleuth-dora/modeling-your-deployments/organization/status), [Trends](/sleuth-dora/modeling-your-deployments/organization/trends), and [Compare](/sleuth-dora/modeling-your-deployments/organization/compare)).
The top navigation bar also provides the "Switch to" context picker, which lets you to bring a specific Project or Team (and its [context-specific left-hand-hand navigation pane](#context-specific-left-hand-navigation-pane)) into focus.
#### Selecting a Project-specific or Team-specific context
The context picker allows you to quickly search through your Projects and Teams to bring one into focus.
To select a specific Project or Team, click on the waffle icon in the top navigation bar to open the context picker. Select either Project or Team, and if you have many of either, use the type-ahead search to quickly find the one you're looking for. Sleuth will also remember the last 3 Projects or Teams you've recently visited.
## Context-Specific Left-Hand Navigation Pane
Once a specific Project or Team has been selected from the context picker, Sleuth displays a left-hand navigation pane that is specific to that Project or Team. The left-hand navigation pane provides access to context-specific dashboards (e.g. [Metrics](/sleuth-dora/accelerate-metrics), [Work in Progress](/sleuth-dora/work-in-progress)) as well as entry points for viewing and managing the items that live within that context:
* For a Project, the left-hand navigation provides access to that Project's [Code Deployments](/sleuth-dora/modeling-your-deployments/code-deployments), [Impact Sources](/sleuth-dora/integrations-1/impact-sources), [Feature Flags](/sleuth-dora/modeling-your-deployments/feature-flags), [Manual Changes](/sleuth-dora/modeling-your-deployments/manual-changes), and [Project Settings](/sleuth-dora/settings/project)
* For a Team, the left-hand navigation provides access to that Team's Subteams and [Team Settings](/sleuth-dora/settings/organization/team-settings)
 
# DORA metrics
Sleuth is the most accurate and flexible way to track and improve your team's Accelerate (DORA) metrics
If you’re an engineering lead, Sleuth will make sure you have the insights you need to know how your team is performing, where they may be encountering bottlenecks and how to remove them.
A healthy team depends on trust between team and leadership. Sleuth helps you verify without breaking trust with your developers.
Sleuth's metrics experience allows you to:
* Easily understand if your performance is steady, improving or taking a turn for the worse by exploring your metrics performance for any period, with comparisons to the previous period
* Dive into actionable insights that help you spot where your projects and teams are running into bottlenecks and how to remove them
* See where your team ranks in relation to industry standards and what it will take to reach the next level
* Let the information come to you with weekly, bi-weekly or monthly email-based metrics digests and insights
* Quickly implement the tools your teams need to improve their Accelerate metrics and deployment process
# Deploy frequency
**Deploy frequency** measures how often you deploy changes to a given target environment. Along with [Change lead time](/sleuth-dora/accelerate-metrics/change-lead-time), **Deploy frequency** is a measure of *speed* (whereas [Change failure rate](/sleuth-dora/accelerate-metrics/change-failure-rate) and [MTTR](/sleuth-dora/accelerate-metrics/mttr) are measures of *quality*, or *stability*)
Once you've [configured your deployment to let Sleuth know when you deploy](/sleuth-dora/modeling-your-deployments/code-deployments/how-to-register-a-deploy), Sleuth will use those events to determine your deploy frequency for all [code deployments](/sleuth-dora/modeling-your-deployments/code-deployments) and [feature flags](/sleuth-dora/modeling-your-deployments/feature-flags) you've setup within a [project](/sleuth-dora/modeling-your-deployments/projects) and [environment](/sleuth-dora/modeling-your-deployments/environment-support).
Sleuth's [**Project Metrics**](/sleuth-dora/modeling-your-deployments/projects) and [**Team Metrics**](/sleuth-dora/modeling-your-deployments/teams) dashboards show the total number of deploys and the frequency, deploys per day, in the selected period. The deploys per day statistic is calculated by taking the total number of deploys divided by the total number of days in the period, weekends included.
For more on how Sleuth measures Deploy frequency, check out Sleuth CTO, Don Brown, explaining it in detail in this SleuthTV episode!
{% embed url="" %}
Sleuth CTO Don Brown explains how Sleuth measures Deploy frequency
{% endembed %}
### Batch Size Breakdowns
In order to help you better contextualize your deploy frequency, Sleuth also provides a detailed breakdown of the batch size of each deploy as a percentage of the total and broken down per day. Batch size is defined as a blend of the number of pull requests, the number of commits, and the amount of code changed, weighted in that order. Batch sizes are defined as:
* Small - usually 1 pull request, 1 - 10 commits and a few hundred lines of code changed
* Medium - usually 1 - 2 pull requests, 10 - 30 commits and many hundreds of lines of code changed
* Large - usually 2 - 4 pull requests, 20 - 40 commits and many hundreds of lines of code changed
* Gigantic - usually 4 or more pull requests or 30 or more commits or many thousands of lines of code changed
{% hint style="info" %}
Because batch size is a weighted blend of pull requests, commits and code changes, you may find that an especially large amount of change in any of those elements can cause a deploy to be classified as Large or Gigantic.
{% endhint %}
## Feature flags and Deploy frequency
Sleuth [supports feature flags](/sleuth-dora/modeling-your-deployments/feature-flags) as a first class form of change. That said, we find that teams want to understand the distinction between their code deploy frequency and their flag change frequency. Sleuth's Project Metrics and Team Metrics dashboards show the two frequencies as separate graph lines and allows you to toggle one or the other on and off. To see totals for flag frequency you can hover over a data point.
## Setting up Deploy frequency
Sleuth uses our [code integrations](https://help.sleuth.io/integrations-1/code-deployment) (Github, Bitbucket, Gitlab, etc) coupled with our [deployment tracking](/sleuth-dora/modeling-your-deployments) to understand when you've deployed. Once you've connected your code to Sleuth, setup your first [code deployment](/sleuth-dora/modeling-your-deployments/code-deployments) and started [registering deploys](/sleuth-dora/modeling-your-deployments/code-deployments/how-to-register-a-deploy) Sleuth will automatically track your deploy frequency and batch size breakdown for each deploy to each of your defined [Environments](/sleuth-dora/modeling-your-deployments/environment-support).
Sleuth uses our [LaunchDarkly integration](/sleuth-dora/integrations-1/feature-flags/launchdarkly) to track feature flag changes. Once setup, we'll automatically start tracking your flag frequency across each of your defined [Environments](/sleuth-dora/modeling-your-deployments/environment-support).
{% hint style="info" %}
For the most accurate metrics we recommend using one of our native [CI/CD integrations](/sleuth-dora/integrations-1/builds) or a [webhook to register](https://help.sleuth.io/modeling-your-deployments/code-deployments/how-to-register-a-deploy#precise-deploy-registration-via-a-webhook) your deploys. Initially, Sleuth is configured to count a pull request merge as a deploy until we've received your first integrated deployment notification.
{% endhint %}
## Further Reading
For additional information on how Sleuth calculates and presents Deploy frequency and other DORA metrics throughout its various dashboards and views, see [Interpreting metrics in Sleuth](/sleuth-dora/accelerate-metrics/how-we-calculate).
# Change lead time
**Change lead time** measures the time it takes for a change to go from its initial start of coding to being deployed in its target environment. Like [Deploy frequency](/sleuth-dora/accelerate-metrics/deploy-frequency), **Change lead time** is a measure of *speed* (whereas [Change failure rate](/sleuth-dora/accelerate-metrics/change-failure-rate) and [MTTR](/sleuth-dora/accelerate-metrics/mttr) are measures of *quality*, or *stability*).
Sleuth calculates **Change lead time** for all your [code deployments](/sleuth-dora/modeling-your-deployments/code-deployments) and for each configured [environment](/sleuth-dora/modeling-your-deployments/environment-support). Because Sleuth tracks not just your pull requests or branches, but your actual *deploys*, we're able provide a highly accurate **Change lead time** that includes all the commits, pull requests, and issues that went into a deploy.
By default, the "start of the clock" for **Change lead time** is the time of the first code commit included in the Deploy. However, Sleuth also provides a Project-level option to start the clock at the first moment that any issue included in the Deploy is transitioned to any state within your issue tracker that you tell Sleuth to treat as an "in-progress" state. For more on issue-based CLT, refer to [Starting CLT based on first issue transition](#starting-clt-based-on-first-issue-transition) below.
Consider the following example using the default CLT start definition (i.e. the moment of first commit). Let's say that you deploy every merged pull request to your staging environment. However, let's also say that you bulk up all changes made in a day in staging and deploy them together to your production environment. With this style of work you'll see smaller **Change lead times** for each staging deploy. However, the **Change lead time** for your production deploy will include all the pull requests and the extended deploy time it took for them to make it to production.
For more on how Sleuth measures **Change lead time**, check out Sleuth CTO, Don Brown, explaining it in detail in this SleuthTV episode!
{% embed url="" %}
Sleuth CTO Don Brown explains how Sleuth calculates Change lead time
{% endembed %}
## Lead time breakdowns
In addition to showing the average Change lead time for all deploys in a selected period, Sleuth also provides a detailed breakdown of how much time your teams, on average, are spending:
* **Coding** - the time spent from first commit (or alternatively, the time spent from the first transition of an issue to an "in-progress state) to when a pull request is opened
* **Review lag time** - the time spent between a pull request being opened and the first review
* **Review time** - the time spent from first review to the pull request being merged
* **Deploying** - the time spent from pull request merge to deployment
In addition to the averages found on the metrics dashboards you can always see exactly where time was spent for each deploy via its detailed view.
For a more detailed timeline, including events from issue creation to deploy verification, you can always consult the [timeline](https://help.sleuth.io/modeling-your-deployments/deploy-cards#deploy-card-timeline-icons) for each deploy.
{% hint style="info" %}
On Sleuth dashboards the change lead time and breakdowns are the averages for each deploy across the period.
On the deploy view the change lead time and breakdowns are the averages for each pull request that was deployed.
You can view the individual change lead time and breakdowns for an individual pull request by viewing the*`PRs`*tab on the deploy view.
{% endhint %}
## Feature flags and Change lead time
Sleuth [supports feature flags](/sleuth-dora/modeling-your-deployments/feature-flags) as a first class form of change. That said, feature flags are an instantaneous change to your running deployments. Therefore feature flags don't have a lead time or breakdown. Sleuth excludes feature flag changes from the lead time graph and associated deploy list.
## Setting up Change lead time
Sleuth uses our [code integrations](https://help.sleuth.io/integrations-1/code-deployment) (Github, Bitbucket, Gitlab, etc) coupled with our [deployment tracking](/sleuth-dora/modeling-your-deployments) and, if you've elected to use the issue-based CLT start definition, our [issue tracker integrations](/sleuth-dora/integrations-1/issue-trackers) to build a complete picture of your team's lead time. Once you've connected your code to Sleuth and setup your first [Code Deployment](/sleuth-dora/modeling-your-deployments/code-deployments) Sleuth automatically tracks your change lead time for each deploy.
See [Interpreting metrics in Sleuth](/sleuth-dora/accelerate-metrics/how-we-calculate) for additional information on how to interpret Sleuth's presentation of Change lead time and the other DORA metrics
### Starting CLT based on first issue transition
By default, Sleuth starts the CLT clock at the moment of first code commit. However, Sleuth also provide the option on a project-by-project basis to start the CLT clock at the first transition of any included issues to any states that you define to be "in progress" in your issue tracker.
To enable this option, perform the following steps:
* **Prerequisite**: before enabling issue-based CLT for a Project, you must first ensure that that Project has an Issue Tracker integration enabled and configured. See [Issue tracker integrations](/sleuth-dora/integrations-1/issue-trackers) for details
* From the left-hand-navigation, select the Project for which you want to enable issue-based CLT start definition.
* Expand the **More** drawer and select **Project Settings**
* Scroll down and expand the **Advanced Settings** section
* Under the **Change lead time** **settings** heading, change **Start definition** to **First issue transition to state**
* Use the **Project selection** drop-down to select the desired issue workflow from your issue tracker.
* **NOTE:** The purpose of this selection is to tell Sleuth which workflow state model to present in the next field. In Jira, issue workflows are tied to Jira Projects (hence the name of this field), but in other issue trackers, issue workflows might be tied to other entities (i.e. in Linear, workflows are defined for Teams, and so those are what you'll see in this drop-down if you're using Linear)
* In the **Issue states** drop-down, select *all states that should be counted* *as "in progress" staets with regard to your CLT definition*. It's important that you select *all* such states because some issue tracker workflows are more free-form than others (e.g. allowing an issue to go straight from "To Do" to "In Review"). Sleuth will look for the earliest transition into any of the states you select here (including cases where the issue is initially *created* in one of these states), so it's important to be exhaustive here.
* Note that in cases where no issue start states are detected or where the first commit pre-dates the earliest issue transition, Sleuth will use that first commit as the start time for CLT.
## Working Hours and Change lead time
Many customers want the change lead time clock to stop on weekends, holidays, or based on individual users' working hours.
Users can set their individual working hours on their [Account settings](/sleuth-dora/settings/account) page, and administrators can set Organization-level working hours in [Organization settings](/sleuth-dora/settings/organization) page.
Organization-level working hours will be applied for any users that don't have their own working hours specified (this includes any Contributor users that do not have direct access to Sleuth).
For any users that have set their own working hours, their user-level working hours will override Organization-level working hours.
Per the above logic, the CLT clock continues ticking on a PR as long as any authors or reviewers on the PR are currently working.
## Further Reading
For additional information on how Sleuth calculates and presents Change lead time and other DORA metrics throughout its various dashboards and views, see [Interpreting metrics in Sleuth](/sleuth-dora/accelerate-metrics/how-we-calculate).
# Change failure rate
**Change failure rate** measures the percentage of deployed changes that cause their target environments to end up in a state of failure. Along with [MTTR](/sleuth-dora/accelerate-metrics/mttr), **Change failure rate** is a measure of the *quality*, or *stability* of your software delivery capability.
"Failure" is defined differently for different organizations (and even within an organization), and Sleuth allows you capture your own unique definition of failure for each project you manage in Sleuth (see [Setting up Change failure rate](#setting-up-change-failure) below for additional information on capturing your organization's unique definition of "failure" within Sleuth). At a high level, Sleuth evaluates **Change failure rate** by evaluating the specific [Impact Source integrations](/sleuth-dora/integrations-1/impact-sources) you've set up for a given project and then calculates **Change failure rate** by dividing the number of deploys that were within [your change failure sensitivity](https://help.sleuth.io/settings/project/details#advanced-settings) by the total number of deploys in the period.
For example, if you've setup the [PagerDuty integration](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/pagerduty#about-the-integration) as an impact source and your team has one incident during the report period that spanned two deploys and you made a total of 20 deploys in that period, your change failure rate will be: 2 / 20 = 10%.
For more on how Sleuth measures **Change failure rate** and for best practices for determining what failure means to you, check out Sleuth CTO, Don Brown, explaining it in detail in this SleuthTV episode!
{% embed url="" %}
Sleuth CTO Don Brown explains how Sleuth measures Change failure rate
{% endembed %}
### Change failure breakdowns
Sleuth's [**Project Metrics**](/sleuth-dora/modeling-your-deployments/projects) and [**Team Metrics**](/sleuth-dora/modeling-your-deployments/teams) dashboards show the total number of deploys that were deemed a failure in the period and also provide detailed breakdowns of deploys by type of failure. Failure types currently supported in Sleuth are:
* **Incidents** - any deploy with a status of `Incident` - *Sleuth provides integrations with PagerDuty, Statuspage, and many more, and we're continuously adding new integrations per customer demand. See* [*Integrations*](https://github.com/sleuth-io/sleuth-gitbook-docs/blob/master/accelerate-metrics/broken-reference/README.md) *for an up-to-date list of those we currently support.*
* **Rolled back** - any code deploys that were [detected to be rolled back](/sleuth-dora/modeling-your-deployments/code-deployments/rollbacks)
* **Unhealthy** - any configured [impact sources](/sleuth-dora/integrations-1/impact-sources) and [deploy verification](/sleuth-dora/auto-verify-your-deploys) that has determined a deploy is `Unhealthy`
* **Ailing** - any configured [impact sources](/sleuth-dora/integrations-1/impact-sources) and [deploy verification](/sleuth-dora/auto-verify-your-deploys) that has determined a deploy is `Ailing`
## Feature flags and Change failure rate
Sleuth [supports feature flags](/sleuth-dora/modeling-your-deployments/feature-flags) as a first class form of change. Because feature flag changes have just as much power to affect failure as code changes, feature flag changes are included in your change failure rate calculations. Sleuth's [deploy verification](/sleuth-dora/auto-verify-your-deploys) applies to flag changes in the same way it applies to code deploys.
{% hint style="info" %}
Every deployment, feature flags included, has an advanced setting that allows you to exclude it from impact collection. If this is enabled, then feature flags will not affect your change failure rate.
{% endhint %}
## Setting up Change failure rate
Sleuth's **Change failure rate** is configured and calculated at the [Project](/sleuth-dora/modeling-your-deployments/projects) level, and Sleuth also provides visibility into change failure for individual [Teams](/sleuth-dora/modeling-your-deployments/teams) (i.e. across all projects to which a team has contributed). By default Sleuth considers any deploys marked as `Unhealthy` as a failure. You can change the failure level in your project settings. If you would like to count only Incidents as failure, for example, then set the failure level to `Incident`.
Sleuth's [deploy verification](/sleuth-dora/auto-verify-your-deploys) allows you to integrate error trackers, such as Sentry and Rollbar, metrics trackers, like AWS CloudWatch and Datadog, and incident trackers, like Statuspage and Pagerduty *(see* [*Integrations*](https://github.com/sleuth-io/sleuth-gitbook-docs/blob/master/accelerate-metrics/broken-reference/README.md) *for a full list of currently supported integrations)*. When Sleuth auto-verifies a deploy as `Unhealthy` that deploy is considered a failure. Setting a deploy to `Unhealthy` manually will also be considered a failure. Sleuth also supports code deploy [rollbacks](/sleuth-dora/modeling-your-deployments/code-deployments/rollbacks). `Rolled back` deploys also count as change failure.
{% hint style="info" %}
When configuring Change failure rate you'll want to determine what failure means to your project. Sleuth is flexible and allows you to define whatever failure criteria works for your projects. Once configured at the project level, change failure rate is also viewable by contributing [teams](/sleuth-dora/modeling-your-deployments/teams). Just keep in mind that the failure data Sleuth provides is only as good as the data coming in.
{% endhint %}
## Further Reading
For additional information on how Sleuth calculates and presents **Change failure rate** and other DORA metrics throughout its various dashboards and views, see [Interpreting metrics in Sleuth](/sleuth-dora/accelerate-metrics/how-we-calculate).
# MTTR
**Mean time to recovery or MTTR** is defined in Sleuth as the time a project spends in a failure state. Along with [Change failure rate](/sleuth-dora/accelerate-metrics/change-failure-rate), **MTTR** is a measure of the *quality*, or *stability* of your software delivery capability.
When Sleuth detects that an Impact Source is failing (e.g. an incident in PagerDuty or an elevated metric in Datadog), it creates a failure period that tracks the details of that failure along with its start and end period. When calculating the **MTTR** for a date range, Sleuth accounts for all of the failure periods that occurred in that range and produces the average.
For example, if you have three incidents that happened in the date period you're inspecting, one lasting 1 hour, one lasting 2 hours and one lasting 3 hours. Your MTTR will be: (1 + 2 + 3) / 3 = 2 hours.
For a real-world example of how Sleuth helps you measure and drive down your **MTTR**, let's say you make a deploy that adds 25% to your database CPU. Assume that Sleuth is tracking this impact and determines that the deploy is Unhealthy. Your team has setup [slack notifications](/sleuth-dora/notifications) in Sleuth, and as a result your mean time to discovery (or MTTD) is basically zero. Your team jumps into action and initiates a rollback which takes 25 minutes to complete. Once your rollback is deployed, Sleuth sees that your database CPU has gone back down to normal and auto-verifies the deploy as Healthy. Your **MTTR** in this scenario would be **25 minutes**, the amount of time it took for your team to return your project to a healthy state.
For more on how Sleuth measures **MTTR**, check out Sleuth CTO, Don Brown, explaining it in detail in this SleuthTV episode!
{% embed url="" %}
Sleuth CTO Don Brown explains how Sleuth measures MTTR
{% endembed %}
## MTTR breakdowns
Sleuth's [**Project Metrics**](/sleuth-dora/modeling-your-deployments/projects) and [**Team Metrics**](/sleuth-dora/modeling-your-deployments/teams) dashboards show the total time spent in a failure state in the period. We also provide a detailed breakdown of the time spent in each type of failure. Failure types currently supported in Sleuth are:
* **Incidents** - any deploy with a status of `Incident` - *Sleuth provides integrations with PagerDuty, Statuspage, and many more, and we're continuously adding new integrations per customer demand. See* [*Integrations*](https://github.com/sleuth-io/sleuth-gitbook-docs/blob/master/accelerate-metrics/broken-reference/README.md) *for an up-to-date list of those we currently support.*
* **Rolled back** - any code deploys that were [detected to be rolled back](/sleuth-dora/modeling-your-deployments/code-deployments/rollbacks)
* **Unhealthy** - any configured [impact sources](/sleuth-dora/integrations-1/impact-sources) and [deploy verification](/sleuth-dora/auto-verify-your-deploys) that has determined a deploy is `Unhealthy`
* **Ailing** - any configured [impact sources](/sleuth-dora/integrations-1/impact-sources) and [deploy verification](/sleuth-dora/auto-verify-your-deploys) that has determined a deploy is `Ailing`
## Feature flags and MTTR
Sleuth [supports feature flags](/sleuth-dora/modeling-your-deployments/feature-flags) as a first class form of change. Because feature flag changes have just as much power to affect failure and recovery as code changes feature flag changes are included in your MTTR calculations. Sleuth's [deploy verification](/sleuth-dora/auto-verify-your-deploys) applies to flag changes in the same way it applies to code deploys.
{% hint style="info" %}
Every deployment, feature flags included, has an advanced setting that allows you to exclude it from impact collection. If this is enabled then feature flags will not affect your MTTR.
{% endhint %}
## Setting up MTTR
Because MTTR is so closely tied to Change failure rate, please see [setting up change failure rate](/sleuth-dora/accelerate-metrics/change-failure-rate#setting-up-change-failure) to configure MTTR.
## Further Reading
For additional information on how Sleuth calculates and presents MTTR and other DORA metrics throughout its various dashboards and views, see [Interpreting metrics in Sleuth](/sleuth-dora/accelerate-metrics/how-we-calculate).
# Interpreting Metrics in Sleuth
In order to trust and effectively improve your DORA metrics, it's helpful to understand exactly how Sleuth calculates and presents each of the four DORA metrics throughout its various dashboards and views.
This article expands on the descriptions of the four core DORA metric calculations already described on the preceding pages, and we highly recommend familiarizing yourself with these before reading on:
* [Deploy Frequency](/sleuth-dora/accelerate-metrics/deploy-frequency) (and included discussion of [Batch size breakdown](/sleuth-dora/accelerate-metrics/deploy-frequency#batch-size-breakdowns))
* [Change lead time](#change-lead-time)
* [Change failure rate](#change-failure-rate)
* [MTTR](/sleuth-dora/accelerate-metrics/mttr)
Parts of this article also assume a basic familiarity with Sleuth's [Project Metrics](/sleuth-dora/modeling-your-deployments/projects), [Team Metrics](/sleuth-dora/modeling-your-deployments/teams), and [Trends](/sleuth-dora/modeling-your-deployments/organization/trends) dashboards.
## Interpreting "Percent Change" in Sleuth
### Percent Change for Project Metrics and Team Metrics
In both the [**Project Metrics**](/sleuth-dora/modeling-your-deployments/projects) and [**Team Metrics**](/sleuth-dora/modeling-your-deployments/teams) dashboards, Sleuth provides the ability to filter by a specific target date range. Sleuth then displays 2 lines within each of the four DORA charts, one for the currently selected period and a second for the prior period of the same length. This overlay is great for comparing and zooming-in on specific points in time, however, what's often more important is understanding how the current period compares *overall* against the prior period.
In addition to plotting the currently selected period and the prior period as two distinct timelines on each graph, Sleuth also displays an overall "percent change" at the top of each graph to help you see at-a-glance how the *average* for the selected period compares to the *average* of the prior period.
Percent change is displayed from prior period to current period
On both the [**Project Metrics**](/sleuth-dora/modeling-your-deployments/projects) and [**Team Metrics**](/sleuth-dora/modeling-your-deployments/teams) dashboards, percent change is calculated as follows:
* First, Sleuth calculates the net difference between the two periods by subtracting the average for the prior period from the average for the currently selected period.
* Then, Sleuth calculates the percent change by dividing this net difference by the average for the prior period and multiplying that result by 100
### Percent Change for the Trends dashboard
The [**Trends**](/sleuth-dora/modeling-your-deployments/organization/trends) dashboard also displays percent change for each of the four DORA metrics, but the calculation for percent change here differs slightly from the Project Metrics and Team Metrics dashboards in that the Trends dashboard display only one period of time (i.e. has no concept of a "prior period" in tis comparison.
Percent change is displayed on the Trends dashboard
For the Trends dashboard, Sleuth calculates percent change by splitting the selected period into two equal halves and calculating the average for each half. From there, the calculation of percent change is similar to the one described for the Project Metrics and Team Metrics dashboards above.
## Interpreting Team-level Metrics
Sleuth does not require users to explicitly associate Teams with Projects. Rather, when calculating Team-level metrics, Sleuth automatically infers which Teams are contributing to which Projects by searching for Team Members within Deploys (and in the case of Code Change Deploys, by searching within the underlying PRs, Branches, Builds, and Issues included in those Deploys). If any Team Member is included as an author on a PR, Branch, or Build within a Deploy, as an owner of an linked Issues listed in the Deploy, or as the initiator of the Deploy itself, then Sleuth will include the entirety of that Deploy in its calculation of Team level-metrics for all Teams to which that Team Member belongs.
As such, Sleuth can present powerful DORA metric "intersections" that show each Team's relative contribution to the DORA metrics for the Projects they're working on. This is evident in views such [Team Metrics](/sleuth-dora/modeling-your-deployments/teams) dashboard's **Projects contributed to** panel below, which shows the DORA metrics at the specific intersection of *this Team* and *those Projects*.
The Team Metrics dashboard shows DORA metrics for the specific intersections between the selected team and the specific projects to which they're contributing
Similarly, from within the [Project Metrics](/sleuth-dora/modeling-your-deployments/projects) dashboard, Sleuth presents a view into **Contributing teams** and their relative impacts on that Project's metrics.
The Project Metrics dashboard shows DORA metrics for each team's specific contributions to the project
## Interpreting Averages across Multiple Projects
When viewing metric averages across multiple Projects in Sleuth, it's important to note that Sleuth calculates cross-Project averages based on the underlying Deploys within each Project.
So, for example, if *Project A* has 2 deploys and *Project B* has 7 deploys, Sleuth will calculate the average CLT across both Projects by adding the CLT for all 9 Deploys and then dividing that sum by 9 (the total number of Deploys across both Projects).
This produces an average CLT result that in most cases will not be equal to the result of adding up the Project-level CLTs and dividing that sum by 2 (the total number of Projects). Sleuth has been intentionally designed around a Deploy-centric point of view, and we believe this Deploy-level handling of cross-project averages provides the most accurate representation of customers' DORA metrics across Projects.
Some specific use cases where this applies include:
* Viewing multiple Projects on the **Trends** dashboard
* Using **Labels** to view cross-project metrics
* Viewing **Team-level metrics** for Teams working on multiple Projects
* Passing multiple Project slugs into the **Sleuth API**
# Deployment tracking
How to represent your existing deployments in Sleuth
Sleuth is designed to allow you to get up and running quickly with all of your existing dev, observability and CI/CD tools. Our light-weight [integrations](/sleuth-dora/integrations-1/about-integrations) allow you to model your existing deployments in Sleuth without having to change your flow or install any invasive libraries.
## How Sleuth models your deployments
### Projects
Within your Sleuth [Organization](/sleuth-dora/settings/organization) you can create any number of [Projects](/sleuth-dora/modeling-your-deployments/projects). Examples of how you might use projects are:
* To represent a product line your organization offers
* To represent a collection of services that many of your products rely on
* To represent a collection of micro-services that, together, offer a specific set of functionality
* To represent business units of your organization
{% hint style="info" %}
Sleuth creates a project for you out-of-the-box. Just name your project in the setup wizard and you're on your way.
{% endhint %}
### Environments
Sleuth allows you to model your existing deployment Environments. Environments are defined and shared under a Project. Examples of how you might use an Environment are:
* To represent your staging and production deploys
* To represent each percentage of a Canary rollout (10%, 25%, etc)
* To represent QA deployments or production deployments across different regions
{% hint style="info" %}
Sleuth creates a Staging and Production environment for every Project by default. You may delete these if they don't correctly represent your deployments.
{% endhint %}
### Code deployments
The heart of most software change in an organization is driven via code. A Code deployment in Sleuth (not to be confused with a [Deploy](#deploys) in Sleuth) represents a direct mapping to a Git repository in your source control system (e.g. GitHub, Bitbucket, GitLab). You may define any number of Code deployments under a Project so Sleuth can track your code deploys. Examples of code deployments include:
* The code used to deploy your main monolith application
* The code used to store and deploy your Terraform infrastructure
* The mono-repo code used to deploy many micro-services (you can create a deployment per service in your mono-repo)
### Feature flags
Many teams use Feature flags to activate new code paths or features for their customers. This is just another source of change and Sleuth will treat them as such. You can connect your LaunchDarkly feature flags to a project.
### Manual changes
Manual changes let you enter anything that you want tracked in Sleuth that isn't covered by code, feature flags, or another type of change that Sleuth [currently supports](/sleuth-dora/integrations-1/about-integrations). They are a free-form entry that can have any name or description you'd like. Examples include:
* A manual resource scaling event
* The restart of a service
* An increase in your infrastructure capacity
### Deploys
Deploys are how Sleuth represents the changes that are made from your code deployments, feature flag and manual changes. Deploys are specific to a project, environment and change source (i.e. a Code deployment, a Feature flag, or a Manual Change) but are visible and searchable at the project and team levels. Deploys can progress through your different environments and Sleuth will show you which environments a deploy has passed through. Deploys collect all the relevant data that went into making your change and, when deploy verification is enabled via Impact tracking, shows the impact your change has made on the health of your service.
# Organization
An **Organization** is the top level container that Sleuth uses to organize your [Projects](/sleuth-dora/modeling-your-deployments/projects). Most Sleuth users will spend the majority of their time in the project-level, deployment-level, and team-level dashboards. However, for managers or executives looking for the 50-foot view of their Organization's engineering efficiency, Sleuth offers multiple ways to slice and dice your organization-wide performance.
* [Compare](/sleuth-dora/modeling-your-deployments/organization/compare) and contrast DORA metrics across any combination of [projects](/sleuth-dora/modeling-your-deployments/projects), [labels](/sleuth-dora/modeling-your-deployments/organization/labels), or [teams](/sleuth-dora/modeling-your-deployments/teams)
* See the overall [Status](/sleuth-dora/modeling-your-deployments/organization/status) of your organizations [projects](/sleuth-dora/modeling-your-deployments/projects) or [teams](/sleuth-dora/modeling-your-deployments/teams) at a glance
* Get a snapshot view into exactly what's been changing across your [projects](/sleuth-dora/modeling-your-deployments/projects) and [teams](/sleuth-dora/modeling-your-deployments/teams)
* [Search](/sleuth-dora/modeling-your-deployments/organization/search) for any deploy across all of your organization's [projects](/sleuth-dora/modeling-your-deployments/projects), [environments](/sleuth-dora/modeling-your-deployments/environment-support), and [teams](/sleuth-dora/modeling-your-deployments/teams) via any dimension (issue key, pull request titles, author and more)
# Labels
Labels allow you to create flexible taxonomies to slice and dice your DORA metrics across your Organization. They are used to combine any number of [projects](/sleuth-dora/modeling-your-deployments/projects) together for metrics comparison and trends.
Do you want to see how your Java-based projects compare to your Python-based projects? Perhaps you’d like to know how your TDD projects stack up against non-TDD projects. Are one week or two week sprints more efficient?
Group your projects by coding language, sprint time or whatever dimension you want insights on.
### Managing Labels
You can manage your Organization's Labels via your Organization settings. You can quickly label multiple projects via the create/edit Label dialog. Labels are a light-weight construct that instantly allow you to generate additional cross-project DORA metric Trends and Comparisons.
# Trends
Sleuth's **Trends** dashboard allows you spot how your initiatives (projects, labels, or teams) are trending over time.
You can use the date filter to see trends over the last two weeks, month, quarter or any custom date range you desire.
Each of the four DORA metrics can be drilled into to see how each metric has broken down. If your MTTR is trending up over the period use the drill-downs to discover if it's an increase in Incidents, Rollbacks or some other cause.
Using the Trends dashboard coupled with [Labels](/sleuth-dora/modeling-your-deployments/organization/labels) and [Teams](/sleuth-dora/modeling-your-deployments/teams) allows you to see trends across any taxonomy within your Organization.
{% hint style="info" %}
Trends for all projects is available on all plans. Filtering by specific projects, labels, or teams requires being on an Enterprise plan.
{% endhint %}
### How is Failure rate calculated?
Each of the four bars represents a period. For each of the 4 periods Sleuth calculates failure rate across all projects. Assume for period 3 one has three projects with corresponding failure rates
* Project A: 30%
* Project B: 0%
* Project C: 0%
The resulting average for the period is somewhere around 10%. You can see this by hovering over independent bars.
The final number at the bottom represents the average across all four periods
## Further Reading
For additional information on how Sleuth calculates and "percent change" for the Trends dashboard and for other dashboards and views, see [Interpreting "Percent Change" in Sleuth](/sleuth-dora/accelerate-metrics/how-we-calculate#interpreting-percent-change).
# Compare
With the Organization Compare dashboard you can finally understand the relative impact of your Engineering initiatives and processes.
Do you want to see how your Java-based projects compare to your Python-based projects? Perhaps you’d like to know how your TDD teams stack up against non-TDD teams. Are one week or two week sprints more efficient?
Using the Compare dashboard coupled with [Labels](/sleuth-dora/modeling-your-deployments/organization/labels) and [Teams](/sleuth-dora/modeling-your-deployments/teams) allows you to compare across any taxonomies or projects within your Organization.
{% hint style="info" %}
Compare is only available on the Enterprise plan.
{% endhint %}
# Search
No good sleuth should be without an amazing magnifying glass. Sleuth's search is one the most powerful tools in your kit, allowing you to instantly find the cause of any bad deploys.
Search in Sleuth allows you to search across all of your [Projects](/sleuth-dora/modeling-your-deployments/projects), [Deployments](/sleuth-dora/modeling-your-deployments/code-deployments), [Environments](/sleuth-dora/modeling-your-deployments/environment-support), and [Teams](/sleuth-dora/settings/organization/team-settings). Use Sleuth search to:
* Quickly discover the root cause of a bug or incident: filter by date and environment to quickly bisect and see where the bad change was introduced
* See when a pull request, commit or issue was deployed to your various environments: filter by pull request id, issue key or commit hash or description
* See which environments are missing a change: filter by the change description and see which environments it's been deployed in
* See what kind of changes are causing your service to be unhealthy: filter by all unhealthy deploys
* Find all the migrations you've made: filter by the migration tag
* Find all the changes made by a specific team or team member: filter by the team or the specific author of the change
* Find all the deploys that might have been just a little too large: filter by the size of the deploy
* And much more, slice and dice your deploys, your way
## Sleuth search filters
| Filter | Description |
| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Date | Limits your search between a date range |
| Project | Limits your search to a specific set of Projects |
| Team | Limits your search to a specific set of Teams |
| Deployment | Limits your search to a specific set of deployments |
| Environment | Limits your search to a specific set of environments |
| Author | Limits your search to users that have participated in a Deploys commits, issue, pull request or was the instigator of the deploy |
| Health | Filters your search by deploys that match the set of health's selected |
| Tag | Filters your search by any tags that exist on the deploys. See [tags](/sleuth-dora/modeling-your-deployments/code-deployments/tags) for the list of tags Sleuth adds automatically |
| Size | Filters your search by deploys of the specified sizes |
| Pull Request | Searches for any deploys that contained the specified pull request ids |
| Commit | Searches for any deploys that contained the specified commit SHAs |
| Issue | Searches for any deploys that contained the specified issue keys |
| Free text | Sleuth allows you to search for deploys by searching for any text that was included in the commit, pull request and issue descriptions that were included in the deploy |
## Sleuth search display
Sleuth supports two views for search results; the default list view and a card view that contains more information.
The views can be toggled via the control in the upper right hand corner of the Search page. Your preferred view preference will be saved.
# Status
The Organization Status dashboard gives you an up-to-the-minute view of how your Organization is performing and what's shipping right now.
Your 50-foot view of your Organization's status, health and activity
The Org Status view show's a quick summary of each project's DORA metrics, if the project is healthy or currently experiencing a failure and the most recent changes to the project.
# Projects
**Projects** are the main container that Sleuth uses to organize your deployments. For example, a project can correlate to a product that your company offers, a collection of micro services or any collection of [deployments](/sleuth-dora/modeling-your-deployments/code-deployments), [environments](/sleuth-dora/modeling-your-deployments/environment-support) and [impact sources](/sleuth-dora/settings/project/impact) that model how you work.
A project gives you a high-level view your Accelerate (DORA) metrics and of all the changes and deploys happening within. With your project you can:
* See, at a glance, if it's safe to deploy your next change
* Track your Project's DORA metrics (deploy frequency, change lead time, change failure rate, MTTR and deploy size)
* Filter Project-level DORA metrics by specific Code deployments, environments, and contributing [Teams](/sleuth-dora/modeling-your-deployments/teams)
* Know the latest change made to each deployment and it's drift from your other environments
* Slice and dice your past deploys via a [powerful search](/sleuth-dora/modeling-your-deployments/code-deployments/search)
* Configure Slack or Email deploy [notifications for your team or individuals](/sleuth-dora/notifications)
* Know the health and Impact of your deploys via your key SLIs
Metrics dashboard for a specific Project
Status dashboard for a specific Project
## Further Reading
For additional information on how Sleuth calculates and presents "percent change" for the Project Metrics dashboard for other dashboards and views, see [Interpreting "Percent Change" in Sleuth](/sleuth-dora/accelerate-metrics/how-we-calculate#interpreting-percent-change).
# Issue trackers
Once you've [connected your issue tracker](/sleuth-dora/integrations-1/issue-trackers), Sleuth will automatically display issues referenced in:
* Pull request titles and descriptions
* Commit descriptions
* Git branch names
There are several places in Sleuth where you can view these related issues. At the lowest level, each deploy includes an Issues tab that lists any issues associated with that deploy along with direct links to those issue source records in your issue tracker.
Sleuth also provides higher-level "Issue breakdowns" on the Project Metrics, Team Metrics, and Work in Progress dashboards, providing visibility into the status and type of any issues referenced across any of the PR/Deploys you have in focus on those dashboards. For Project Metrics and Team Metrics, this means you can see breakdowns for issues you've shipped, while Work-in-Progress displays breakdowns for issues that you're *about* to ship!
From any of these dashboards, the Issue breakdown chart also provides a clickable drill-through to a detailed listing of issues with links to their source records in your issue tracker.
Drill-through to detailed issue listing from "Issue breakdowns" chart
Issue Type is particularly useful for understanding the nature of the work that's going into each release (new features, bugs, tech debt...), while Issue Status is useful for identifying potential breaks in your issue management workflow (e.g. an issue on the Work in Progress dashboard shows as 'Done' even though it has an outstanding PR).
See the p[roject settings details](/sleuth-dora/settings/project/details) documentation on how to configure your issue tracker for your project.
# Environments
Sleuth's environment support lets you model your change sources, such as code deployments, feature flags, and impact, across the multiple deployment environments you maintain. Environments are defined at the project level and all code deployments and feature flags within a project share Environments.
Once setup, Sleuth provides you with a clear view of how your deploys move through your different environments. Sleuth highlights and allows you to drill into the drift that forms between your Environments.
Sleuth is flexible and allows you to map your Environments to your code in the way that you already work. Whether you deploy one branch to multiple environments or you maintain a code branch per environment Sleuth works for you.
# Code deployments
Sleuth uses your code repositories as its main sources of change. Setting up a code deployment in Sleuth allows us to track and alert on what goes into every code deploy. A code deploy surfaces all the **pull requests, linked issues, commits and authors** included in every deploy. This information allows Sleuth to:
* Track your deployment's Accelerate DevOps metrics
* Deploy frequency, change lead time, change failure rate, MTTR and deploy size
* [Lock deployments](/sleuth-dora/modeling-your-deployments/code-deployments/deployment-locking) so you can pump the brakes on deploying when needed
* Slice and dice past deploys with [powerful search](/sleuth-dora/modeling-your-deployments/code-deployments/search)
* Highlight [drift between your different deployment environments](/sleuth-dora/modeling-your-deployments/code-deployments/environment-drift)
* Sleuth [automatically tag deploys](/sleuth-dora/modeling-your-deployments/code-deployments/tags) based on the code that's changed
* and much more...
# Creating a deployment
A code deployment in Sleuth is mapped to a Git code repository and a branch. The repository/branch combination represents the changes your team is deploying to your [environments](/sleuth-dora/modeling-your-deployments/environment-support).
Creating a code deployment is an easy 3 step process. Sleuth guides you through each step and it only takes about 1 - 2 minutes to configure. Creating a code deployment results in pulling in your last 30 days of deploys (DORA metrics) and leaves you fully setup to track your DORA metrics moving forward.
The 3 steps to create a code deployment are:
1. Enable Integrations
2. Add Code Deployment
3. Choose Tracking Type
### Step 1: Enabling Integrations
Sleuth supports most [code repository providers](/sleuth-dora/integrations-1/code-deployment). Choose your code provider and Sleuth will walk you through connecting to your system either via App, OAuth or API key, depending on the provider.
### Step 2: Add Code Deployment
Configuring a code deployment in Sleuth is simple. You just map the repository and branch that your team deploys from and away you go.
#### Mapping the branch you deploy from
Sleuth is very flexible and will support the way that your team works. For teams that deploy the same branch to all pre-production and production environments you will just choose that branch on this screen.
However, some teams maintain a branch for each environment they deploy to. If this is how your team works you will click the checkbox that says `Map separate branches to environments`
This allows you to tell Sleuth which branch it should inspect for which environment.
#### Mapping a set of branches
Some teams will cut a release branch with a predictable name for each deployment they make. If your team works this way you can type in the prefix of the branch name you create and Sleuth will provide you with an option to track `branches matching prefix`.
In this example Sleuth would look for changes on any branch that matches the prefix `released`.
#### Supporting Monorepo's
It's not uncommon for teams to maintain multiple deployable units of code in one large repository , this style of repository structure is called a [Monorepo](https://en.wikipedia.org/wiki/Monorepo). Sleuth supports Monorepo's for code deployments.
The way to set up a Monorepo in Sleuth is to add one code deployment for each deployable unit of code within your Monorepo. When configuring the code deployment you will open the `Advanced` settings and use the `Source path prefix (include)` and the `Source path prefix (exclude)`fields to tell Sleuth which parts of the repository to include or exclude. Once configured Sleuth will only register deploys that had changes made within the patterns defined.
### Step 3: Choosing Tracking Type
Sleuth supports a number of ways to track your code deploys. The easiest, zero conf, way to start tracking is connecting Sleuth to your CI/CD system and letting Sleuth detect when you've made a deploy.
You can read more about all of the options for [How to register a deploy](/sleuth-dora/modeling-your-deployments/code-deployments/how-to-register-a-deploy).
#### Initializing a deployment
By default Sleuth will initialize a code deployment with the last 30 days of deploys we can detect. This will allow you to quickly get a baseline of your DORA metrics for the last 30 days. Sleuth uses past events, based on the tracking type you've chosen to seed the initial deploys.
For instance, if you have connected up CircleCI and mapped your deploy jobs to Sleuth we will find all of the successful jobs run in the last 30 days and create deploys for those. Similarly, if you have chosen to manually tell Sleuth about deploys with a webhook we will use the last 30 days worth of pull requests to initialize your deploys.
#### Manually initializing a deployment with as much past data as you want
{% hint style="warning" %}
Initializing more than 30 days worth of data is not a common thing teams require. It's a manual process and will take more effort on your part. If you do need to do so please make sure to follow the directions throughly.
{% endhint %}
For some teams 30 days of past data won't be enough to meet their needs. If you want to manually import past data into Sleuth it is possible but you must follow these steps, in the right order:
1. Create your code deployment
2. When selecting the tracking type choose `Webhook`
3. On the `Using the webhook` confirmation screen open the `Advanced` configuration and deselect the option that says `Populate deploys from the last 4 weeks`\
4. Once the deployment has been created you will then need to manually register the past deploy SHA's with Sleuth in one of two ways using the [Sleuth API](/sleuth-dora/sleuth-api#deploy-registration)APIs
1. (Recommended) Import past deploys via a CSV file using the `/import_deploys` endpoint
2. Import deploys one by one using the `/register_deploy` endpoint
5. Once you're satisfied with how the older data looks inside of your code deployment you can edit the code deployment to have the Tracking type that you'd like moving forward.
{% hint style="info" %}
If you run into trouble initializing the data you can always delete the deployment, re-create it and try populating again.
{% endhint %}
# How to register a deploy
## Notifying Sleuth when you deploy
How does Sleuth know when you have deployed? There are five different ways Sleuth can be notified:
* [Precise (CI/CD) – automatically detect deploys from completed CI/CD builds](#precise-deploy-detection-from-completed-ci-cd-builds)
* [Precise (webhook) – send Sleuth a webhook so we know exactly when you've deployed](#precise-deploy-registration-via-a-webhook)
* [Approximate – automatically create deploys for every PR merged](#approximate-automatic-tracking-for-each-pr-merged)
* [Approximate – automatically create deploys for every commit created](#approximate-automatic-tracking-for-each-push-to-the-configured-branch)
* [Approximate – automatically create deploys for every tag created](#approximate-automatic-tracking-for-each-tag-made-against-the-configured-branch)
{% hint style="info" %}
We highly **recommend precise deploy registration**. Knowing exactly when you've made your deploy unlocks the truly powerful features of Sleuth such as Impact tracking, notifications and more.
{% endhint %}
### Precise deploy detection from completed CI/CD builds
When this option is selected Sleuth will guide you through a CI/CD mapping step where you'll map a build / job / pipeline name to each Sleuth environment. Sleuth will then automatically register a deploy on a successfully completed build / job / pipeline that matches the mapped name.
When using CI/CD build deploy detection there is no need to modify your build scripts or change the way you deploy, Sleuth does all the work and will track your deploys precisely and automatically.
For example, we use CircleCI where we have many jobs, but only two are relevant for deploy registration. These are `deploy-prod` and `deploy-stage`, so our mapping between CI/CD and Sleuth environments looks like so:
{% hint style="info" %}
Sleuth is only able to do auto deploy detection from CI/CD builds for our supported [CI/CD integrations](/sleuth-dora/integrations-1/builds). If you don't see your CI/CD provider please reach out and let us know. We're always adding new Integrations and are prioritizing as demand dictates.
Keep in mind that even without a supported provider you can still achieve precise tracking using our [webhook registration](#precise-deploy-registration-via-a-webhook).
{% endhint %}
#### Manually entering build and job names
{% hint style="info" %}
This feature is currently only supported when using GitHub Actions.
{% endhint %}
Certain complex configurations such as using [reusable workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows) can produce dynamically named workflows and jobs. Sleuth might not be able to get their names from the provider API, which means they won't be available for selection.
To get around this limitation we provide an alternative UI that lets you manually enter workflow and job names. It is accessed by clicking *Can't find your build?* under the build selection dropdown.
#### Tracking deploys from nested workflows
Some workflows are configured in a **nested structure**; top-level jobs are used to **trigger additional (sub-)jobs**, and any completed sub-job should be considered a deploy by Sleuth.
{% code title="Example Structure" overflow="wrap" %}
```yaml
Workflow name: My Test Workflow
Jobs:
Example Job 1
Example Sub-Job 1
Example Sub-Job 2
```
{% endcode %}
CI/CD tools **concatenate the job and sub-job names**, placing a `/` between them and return them as a **single string**.
To account for such a setup in build mappings, a job "prefix" can be manually specified in the `Job` field, followed by `/*`:
{% code title="Example job " overflow="wrap" %}
```yaml
Job: Example Job 1/*
```
{% endcode %}
#### Only detect builds that match the environment branch
When a code deployment has been configured to [map separate branches to environments](/sleuth-dora/modeling-your-deployments/code-deployments/creating-a-deployment#mapping-the-branch-you-deploy-from), Sleuth's default behavior is to register each specified build against its mapped environment whenever the specified build name is detected, regardless of which branch that build might have been generated from in the CD pipeline. This is the preferred behavior for about 90% of Sleuth customers.
Some customers' CD workflows, however, require that Sleuth register a specified build against its mapped environment only when that build has been generated from the specific branch associated with the mapped environment.
For such cases, Sleuth provides a toggle under Advanced Settings that, when enabled, tells Sleuth to register the specified build only when that build's source branch matches the mapped environment branch.
Note that when using any of Sleuth's native CI/CD integrations, this toggle will also impact which builds are included in the Builds tab on the Deploy Details screen. The toggle will not, however, limit which builds appear on Deploy Details screen for deploys that are registered using our [webhook registration](#precise-deploy-registration-via-a-webhook).
#### Build mapping caveats
There are several edge cases where using CI/CD mapping won't be an option:
* The GitHub Actions workflow uses the matrix feature
* The code repository is in Bitbucket and the CI/CD system is on Azure
* The branch mapped to the target environment in Sleuth is a prefixed branch
* When using Jenkins, only a limited set of jobs is available for mapping. To be applicable, a job must be configured to use [GitSCM](https://plugins.jenkins.io/git/#plugin-content-pipelines).
When build mapping is not an option, the fallback is to register deploys with a webhook.
### Precise deploy registration via a webhook
Ping Sleuth with a Git commit SHA or a tag to tell Sleuth you've deployed by making a `POST` request. You'll need to provide theses values when making the call:
* `YOUR_API_KEY`
* `YOUR_SHA`
* `ENVIRONMENT_SLUG`
* `ORG_SLUG`
* `DEPLOYMENT_SLUG`
{% hint style="info" %}
[Get more detailed information](/sleuth-dora/sleuth-api/deploy-registration) on precisely registering a deploy via the Sleuth API.
{% endhint %}
You can find your *API Key* in top right profile menu > **Organization > Settings** > **Details** > **Api key**:
You can find `YOUR_SHA` using the commands:
```http
git checkout YOUR_BRANCH
git rev-parse HEAD
```
### Approximate – automatic tracking for each PR merged
When this option is selected Sleuth will treat every merged PR on your branch as a deploy.
{% hint style="info" %}
Sleuth allows you to specify a delay in minutes. When this is set Sleuth will only create the deploy after the delay has passed. A delay of 0 will create the deploy immediately.
{% endhint %}
### Approximate – automatic tracking for each push to the configured branch
When this option is selected Sleuth will treat every commit made to your branch as a deploy.
{% hint style="info" %}
Sleuth allows you to specify a delay in minutes. When this is set Sleuth will only create the deploy after the delay has passed. A delay of 0 will create the deploy immediately.
{% endhint %}
{% hint style="warning" %}
It's rarely the case that every commit is a deploy. Only true continuous deployment setups deploy every commit.
{% endhint %}
### Approximate – automatic tracking for each tag made against the configured branch
When this option is selected Sleuth will treat every tag made on your branch as a deploy.
{% hint style="info" %}
Sleuth allows you to specify a delay, in minutes. When this is set Sleuth will only create the deploy after the delay has elapsed. A delay of 0 will create the deploy immediately.
{% endhint %}
{% hint style="warning" %}
If you've chosen this option make sure that your CD system is tagging your code only once it's actually been deployed.
{% endhint %}
# Rollbacks
Sleuth supports the deployment of a previously deployed code revision, otherwise know as a rollback.
It's not too uncommon for a team to deploy a code change and then realize, after some time, that the change isn't behaving how they intended. The most common strategy to deal with this is to roll-forward, making a quick fix for the new error. However, sometimes it's too difficult to make a quick fix and you decide to revert to the last known good code revision.
Sleuth automatically detects rollbacks by checking the SHA of the deploy currently being deployed against the SHAs of all existing deploys in the target environment. If a matching SHA is found on an existing deploy, Sleuth will do the following:
* Register the new deploy as a rollback and tag it with the tag `rollback`
* Identify all deploys registered between the rollback deploy and the initial deploy with the same SHA, and mark them as rolled back, changing their health to `Rolled back` and adding the tag `rolled_back`
Rollbacks, by default, are counted as a failure when determining your [Change failure rate](/sleuth-dora/accelerate-metrics/change-failure-rate).
# Automatic tagging
Automatic tagging for quickly searching through your deploy history
Sleuth allows you to setup rules to automatically tag your deploys. Tagging deploys allows you to flag deploys that may require a different level of attention, such as a database migration. Tags are also a quick way to organize and quickly search similar deploys.
The tags are added by looking for patterns in the files deployed in your code repositories. For example, if Sleuth finds a **pom.xml** file in your deploy, it automatically adds the tag **#dependencies** to the deploy.
Tags are searchable via everywhere [search](/sleuth-dora/modeling-your-deployments/code-deployments/search) is exposed. See the table below for more patterns and tags Sleuth automatically applies to your deploys based on pattern matching.
If tags are not explicitly defined for a deployment, Sleuth detects tags by matching files using patterns either from the *\*\*.sleuth/TAGS \*\** file in your repository, or a set of default patterns:
| Pattern | Tag |
| ------------------- | ------------- |
| \*\*/migration/\*\* | #migration |
| Pipfile.lock | #dependencies |
| requirements.txt | #dependencies |
| package-lock.json | #dependencies |
| pom.xml | #dependencies |
| Dockerfile | #docker |
| \*\*/db/\*\* | #database |
| \*tf | #terraform |
### Adding custom tags
In addition to having Sleuth automatically detect patterns and add tags to your deploys, you can add your own patterns that Sleuth can then use to help you search for your previous deploys. This is easily done by editing the ***.sleuth/TAGS*** in your code repository.
#### To add your own pattern/tag pair:
1. Create a \_\*\*.sleuth/ \*\*\_directory in the root directory of your repo. This repo must be connected as a [code deployment](/sleuth-dora/settings/project/code-deployments) in Sleuth.
2. Create a file **TAGS** in the ***.sleuth*** directory.
3. Create a matching pattern/tag pair; create additional pairs on new lines.\
For example:\
`/db #database`
4. Save the file.
In the example above, a directory with the name `db` would generate a tag `database` in the Sleuth deploy card, which you can then search for quickly using the Sleuth search.
# Deployment locking
The Lock feature in Sleuth prevents pull requests from merging into deployment branches, and generates automatic notifications via your [Slack](/sleuth-dora/integrations-1/slack) channel.
#### Manual locking
You can manually lock a deployment or all deployments within a project with one click of a button from your dashboards. You can view your lock history so you can learn how frequently your team locks their deploys and why.
#### Automatic locking
You can also configure Sleuth to auto-lock deployments [when PRs are merged for the deployment branch](https://help.sleuth.io/settings/project/code-deployments#automatically-lock-deployments).
Read Sleuth CTO [Don Brown's blog post](https://www.sleuth.io/post/prevent-unwanted-changes-with-sleuth-deployment-locking) on how Sleuth's locking feature can make your DevOps life easier!
##
# Environment drift
**Environment drift** is the difference between two unique environments in a [project](/sleuth-dora/modeling-your-deployments/projects), measured in the number of commits and deploys between them as well as the amount of time between the current date and the last commit date for the environment that's lagging.
For example, if you just deployed your staging environment to production, drift will be zero since both environments have the same code. As soon as you make a deploy to your staging environment, Sleuth detects a drift between staging and production as *1 commit and 1 deploy*. An example drift value between *Production* and *Staging* environments could be:*\*\* 1 commit and 3 days\*\**.
Sleuth displays environment drift for individual code deployments within the Code Deployment dashboard, and for Projects with multiple code deployments, it also displays environment drift for each of those code deployments from within the Project Status dashboard.
What's more, when drift is detected, Sleuth also provides visibility into drift details, showing exactly which deploys, PRs, commits, and issues need to be resolved across the two environments.
# Move code deployments
We get it, things change. Sometimes it's a change to your development process or your product architecture, or sometimes you just realize after configuring your initial hierarchy of Projects, Environments, and Code Deployments in Sleuth that there was a better way you could have done things.
Fortunately, Sleuth provides the ability to easily move existing Code Deployments from one Project to another while maintaining each Deployments's full history and impact on project-level metrics.
To move a Code Deployment (and all of its Deploys) from one Project to another, perform these steps:
* Open the "source" Project, navigate to the Code Deployment you'd like to move, expand the cog, and select **Move code deployment**.
* In the next screen, select the "target" Project (i.e. the Project you'd like to move this Deployment to), and specify Environment mappings between the two Projects:
* For any of source environments that do not have a corresponding environment in the target Project, select "Do not map." Note, however, that ***this action will irrevocably delete all historical deploy data*** for that source environment***.*** Sleuth will ask you to confirm:
* It is not currently possible to map a single source environment to multiple target environments
* Click **Save** to initiate the move. Note that a code deployment might be in motion for as long as 30 minutes. You can freely move additional code deployments during this time, but it is recommended that you wait until all code deployments have finished moving before evaluating metrics data for any source or target projects.
# Search everything
Search your entire deployment history using Sleuth's built-in search function. The search field features search-as-you-type functionality; simply start typing and the previous [deploys](/sleuth-dora/modeling-your-deployments/deploy-cards) that contain the entered search terms are displayed.
You can also filter the deploy cards based on the health of the deploy. For example, you might only want to search deploys that were deemed *Unhealthy*. The dropdown value can be set either before or after you begin searching.
#### Sleuth searches through all content contained in a deploy:
* Pull request key, summary and descriptions
* Commit hashes and descriptions
* Issue keys, summaries and descriptions
* File names
* Authors username and full name
* Sleuth [tags](/sleuth-dora/modeling-your-deployments/code-deployments/tags)
#### Use Sleuth search to quickly discover when an issue was deployed:
#### Search to find if a pull request was deployed:
#### Search to find all the deploys made by one of your developers:
### Searching with Slack
If your organization has a Slack integration, you can search directly from the Slack app. You can search from any channel in the integrated organization.
To search using Slack, type `/sleuth` then your search term. For example:
```
/sleuth memory leak
```
Search results are displayed in the same channel. The most recent 5 changes are returned. You can click the View all button to view all search results in Sleuth.
By default `/sleuth MY SEARCH TEXT` will search all projects in the organization with the Slack integration for matching deploys.
# Feature flags
Feature flags are an integral part of agile software development, and an important variable in deploying a successful CI/CD pipeline. Making high-impact changes with minimal risk and maximum control of your applications helps deliver quality software to your customers.
Sleuth integrates with your existing [LaunchDarkly](/sleuth-dora/integrations-1/feature-flags/launchdarkly) account to surface your feature flag changes within your Sleuth project. Setting up feature flags in Sleuth allows us to track and alert on flag changes via the same mechanisms used for your code deploys, providing you a single system of record for all your deploy changes.
Sleuth allows a simple mapping between [LaunchDarkly environments](https://docs.launchdarkly.com/home/managing-flags/environments) and [Sleuth environments](/sleuth-dora/modeling-your-deployments/environment-support).
Feature flag changes also trigger automatic health verification when you've setup Impact Sources.
# Manual changes
Manual changes let you enter anything that you want tracked in Sleuth. Such changes are those not tracked by source code, feature flags, or another type of change that Sleuth [currently supports](/sleuth-dora/integrations-1/about-integrations). They are a free-form entry that can have any name or description you'd like.
To add a manual change:
1. Click **Create** then **Add Manual Change** in the sidebar.
2. Give the manual change a **Name** and **Description**.
3. Press **Create**.
{% hint style="info" %}
A sample webhook with your project data pre-populated is displayed (see image below).
{% endhint %}
{% hint style="danger" %}
**Do not share the webhook link with anyone outside of your organization.** The dynamically-generated command contains your private Sleuth API key and other private information.
{% endhint %}
The manual change will be visible on your project dashboard and displayed just like any other source of change and deploy. Manual changes are not updated nor managed by Sleuth; you'll need to maintain them on your own.
{% hint style="info" %}
Manual changes can be also be [submitted via the Sleuth API](/sleuth-dora/sleuth-api#manual-change).
{% endhint %}
# Deploys
Deploys are how Sleuth represents the changes that are made from your [code deployments](/sleuth-dora/modeling-your-deployments/code-deployments), [feature flag](/sleuth-dora/modeling-your-deployments/feature-flags) and [manual changes](/sleuth-dora/modeling-your-deployments/manual-changes). Deploys are specific to a [project](/sleuth-dora/modeling-your-deployments/projects), [environment](/sleuth-dora/modeling-your-deployments/environment-support) and change source. Deploys aggregate information from the integrations you've connected in Sleuth, these include:
* All [code information](/sleuth-dora/integrations-1/code-deployment) in a deploy with quick links to external systems to view; pull requests, commits, files and authors
* Any [issues referenced](/sleuth-dora/integrations-1/issue-trackers) in commits, pull requests or branch names
* Any [CI/CD builds associated](/sleuth-dora/integrations-1/builds) with the deployed revision
* The who (authors), what (change details) and when (timeline) of the deploy
* How long the deploy was live in your Environment
* The change lead time for the deploy, the time from first commit to deploy
* The size of the deploy, based on number of commits, pull requests, etc
* A timeline of all the deploy events
* The Impact attributed to the change and the Sleuth, auto-determined, health of the deploy
## Deploy card timeline icons
Sleuth uses a variety of timeline icons in the deploy view to quickly and easily communicate the evolution of your deploy. You can hover over the icons to get more detailed information. The icons used are shown below:
| Icon | Description | Meaning |
| -------------------------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|  | **Commit** | A commit was made in your deploy. Commits link, hash, messages, date and time, are also displayed. |
|  | **Pull Request** | A pull request was made in your deploy. The pull request link, description, id, date and time, are also displayed. |
|  | **Issue** | Issues referenced via configured issue tracker integrations are displayed. The issue link, key and description are displayed, along with issue creation date and time. |
|  | **Build** | Builds for the deploy revision via configured build services are displayed. The build link, name, hash and results are displayed, along with build run date and time. |
|  | **Deployed** | Your code has been deployed. |
|  | **Soon** | Indicates that a deploy is coming soon. |
|  | **Replaced** | Indicates that the current deploy has been superseded by another deploy. |
|  | **Too Many** | Shown when there are too many event icons to display. Hovering over this icon will let you know how many icons are currently hidden. |
## Deleting deploys
If, for some reason, you need to delete a deploy you can do so by clicking the trash can icon in the upper-right corner of a deploy. This action only deletes the deploy from Sleuth; your connected systems are not affected.
# Teams
While [projects](/sleuth-dora/modeling-your-deployments/projects), [environments](/sleuth-dora/modeling-your-deployments/environment-support), and [deploys](/sleuth-dora/modeling-your-deployments/deploy-cards) represent the core "targets" of changes in Sleuth (and the DORA metrics associated with those changes), we also understand that Teams represent a critical dimension for interpreting DORA metrics and driving Team-level improvements over time.
Teams often span multiple projects, and vice-versa (e.g. a platform team contributing to multiple projects, multiple micro-services teams contributing to the same project). In order to provide even more value to your Accelerate journey, Sleuth provides the optional ability to define team-centric lenses in addition to your project-level and org-wide views.
One benefit to team-centric views is that they can provide greater insight into the root causes (and recommended remediations) of changes to project-level DORA metrics (e.g. being able to trace a project-level increase in Change Failure Rate down to a particular contributing team).
But team-based views are about more than just understanding each team’s relative contributions to the projects they’re working on. Different teams often have their own unique processes, tools, and relevant targets for “what’s good” when it comes to measuring their performance against the DORA metrics, and so team-centric views provide an invaluable tool for enabling “apples-to-apples” measurements, comparisons, and improvements to each team’s DORA metrics over time.
With Teams, Sleuth user can:
* [Define teams and sub-teams](/sleuth-dora/settings/organization/team-settings) in Sleuth
* Automatically maintain Sleuth teams using [GitHub Team Sync](/sleuth-dora/settings/organization/team-settings#manage-teams-using-github-team-sync) (Enterprise-only)
* Track teams across projects with Team-centric DORA metrics dashboards
* Subscribe to [Team-centric email digests](/sleuth-dora/notifications#to-set-up-at-the-team-level)
* Filter [Filter project-level DORA metrics dashboards](/sleuth-dora/modeling-your-deployments/projects) by contributing teams
* View [team-level trends](/sleuth-dora/modeling-your-deployments/organization/trends) over time
* [Compare teams](/sleuth-dora/modeling-your-deployments/organization/compare) with other teams, projects, or labels (Enterprise-only)
* [Search by team](/sleuth-dora/modeling-your-deployments/organization/search)
Teams are managed and maintained from within Organization Settings. For information on how to set up and maintain your teams, sub-teams, and team members in Sleuth, refer to [Managing Teams](/sleuth-dora/settings/organization/team-settings).
Sleuth also allows users to subscribe to Team-centric digest emails. For more information on notifications, refer to [Slack and Email Notifications](/sleuth-dora/notifications).
## Further Reading
For additional information on how Sleuth calculates and presents Team-level metrics, see [Interpreting Team-level metrics in Sleuth](/sleuth-dora/accelerate-metrics/how-we-calculate#interpreting-team-level-metrics).
For additional information on how Sleuth calculates and presents "percent change" for the Team Metrics dashboard for other dashboards and views, see [Interpreting "Percent Change" in Sleuth](/sleuth-dora/accelerate-metrics/how-we-calculate#interpreting-percent-change).
# Work in Progress
In addition to tracking deploy metrics, Sleuth also provides **Work in Progress** dashboards and notifications for Teams and Projects that provide real-time visibility into in-flight work (i.e. work that has not yet deployed) and highlight risks that teams can address right now.
Work in Progress dashboard for a specific Project
### What is "Work in Progress"?
Work in Progress, or WIP, includes any PRs that have not yet deployed to their target environment.
### Understanding "at-risk" items
While the **Work in Progress** dashboards present in-flight work in part to provide general visibility into the changes that are likely deploy next, the real value of these dashboard is that they highlight "at risk" items that are likely to have a negative impact on your DORA metrics. This early risk identification allows you to take immediate corrective actions on at-risk items before they ship, driving tactical improvements to your DORA metrics.
Sleuth currently highlights the following risk types:
* Batch Size
* Total change lead time (CLT)
* Coding time
* Review lag time
* Review time
* Waiting to deploy
For Batch Size, an item is considered "at risk" if it is either Large or Gigantic.
For change lead time and its four composite breakdowns, an item is considered "at risk" if it's current value exceeds your average by more than 30% (where your "average" is calculated based on the items that *deployed* during the same period as your currently work-in-progress data range selection). Note that an item must accumulate a minimum of 30 minutes in a given CLT bucket before Sleuth will potentially flag it as at-risk relative to your average.
### Understanding Work in Progress filters
The Work in Progress dashboards provide three levels of filtering.
#### Left-hand navigation filters
Like all Project-specific and Team-specific dashboards in Sleuth, the highest-level filters for the Work in Progress dashboard are the Project/Team selector and the Environment selector.
When filtering by a specific Environment, note that in order for Sleuth to track PRs for an environment, that environment must be mapped to a branch. See [Environments](/sleuth-dora/modeling-your-deployments/environment-support) for more information.
#### Top-level dashboard filters
Just like the Metrics Dashboards for Projects and teams, the Work in Progress dashboard can be filtered using top-level filters for Date Range, Projects, Teams, Environments, and Deployments. These filters impact the specific PRs that display in the listing as well as the data that is displayed in all of the dashboard charts.
Top-level work-in-progress dashboard filters
Special considerations when using the Date Range filter:
* The Date Range filter allows you to select the "from" date, but the "to" date will always be the current date. The main reason for specifying a "from" date is to exclude "zombie PRs" (i.e. PRs that have not been updated for a long time and so should not be included in your universe of "current work in progress"
* For WIP risks that rely on comparison against your "average", that average is calculated based on the items that *deployed* in the same period as your currently selected work-in-progress date range.
* When you enable [build tracking](/sleuth-dora/modeling-your-deployments/code-deployments/how-to-register-a-deploy) for a [code deployment](/sleuth-dora/modeling-your-deployments/code-deployments), if you opt to include historical deploy data for the past 4 weeks, then Sleuth will also fetch all work-in-progress updated in that time period so that you can immediately begin analyzing your in-flight work.
* Note that Sleuth has been collecting work in progress data only since November 23, 2022, so it is not possible to view work in progress that has not been updated since before that date.
Special considerations when using the Date Range filter:
* The Date Range filter allows you to select the "from" date, but the "to" date will always be the current date. The main reason for specifying a "from" date is to exclude "zombie PRs" (i.e. PRs that have not been updated for a long time and so should not be included in your universe of "current work in progress"
#### Work-in-progress listing filter
In addition to the top-level filters, the detailed listing of work-in-progress items provides an additional filter to zero-in on items that exhibit a particular risk type.
Work-in-progress listing filters by specific risk types
By default, this filter is set to "No Filters," which displays all work-in-progress items that match the top-level filters (i.e. regardless of what risks they might or might not exhibit).
Selecting "All at risk items" filters the listing to show only those items that exhibit some risk (regardless of which specific risk type or types they might exhibit).
The remaining filters selections show items that exhibit a particular risk type. When these specific risk type filters are active, the listing is also sorted by that risk value from riskiest to least risky.
### Understanding work in progress charts
| Chart | Explanation |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|  |
Work in progress lead time displays a comparison of your current aggregate WIP lead time breakdowns against your average lead time breakdowns for items that deployed in the same period.
Grey columns represent the average for deployed items, while red and green columns represent your current work in progress (red indicates that the current WIP value is at risk relative to the average).
This chart is responsive to selections made in the listing filter.
|
|  |
Batch size breakdown groups your current work in progress by each of Sleuth's four batch size categories.
Large and Gigantic counts are outlined in red when either is greater than 0.
This chart is responsive to selections made in the listing filter.
|
|  |
Summary of work in progress summarized work in progress by showing the total count of all items, the total at risk, and the number of times that each risk type appears in your work in progress.
Clicking on the count for any risk type in this chart will set the listing filter to that risk type. It is not, however, responsive to selections made in the list filter.
|
### Subscribe to daily work in progress risk digests
Daily WIP risk digests help ensure that teams are aware of and responding to risks on a daily basis.
#### Slack and Microsoft Teams digests
We recommend using our [Slack or Microsoft Teams digests](https://marketplace.sleuth.io/?search=work+in+progress) over the email digest because they can be delivered to a team's channel, where everyone can see an comment on them, and because they can be scheduled down the minute (e.g. to arrive in a team's channel just before their daily stand-up.).
#### Email digests
If Slack or Microsoft Teams digests aren't an option, then email digests are a great alternative. Simply click on the notification bell on the top-right of the dashboard and select "Daily." The email digest will be sent to your email inbox each day if and only if there is at least one risk present in the work in progress dashboard.
Note that the digest subscription is specific to your currently selected Team or Project context and currently selected Environment. The date range for the email digest will always be the default look-back period of 28 days.
# Goals
Sleuth's **Goals** dashboard, available for Standard an Enterprise plans, helps teams set, track and achieve their efficiency goals by:
* Allowing them to set actionable goals for Coding time, Review lag time, Review time, and Deploying time
* Automatically notifying PR owners and reviewers in real-time as they begin to drift from their goals
* Providing ongoing visibility into their progress toward achieving their goals over time
### Suggested goals
Goals can be set for Projects or for Teams. By default, the Goals dashboard is pre-populated with suggested goals that Sleuth calculates based on your project's or team's historical performance. Suggested goals are calculated by taking your average performance for the selected time period and reducing it by 10% in order to drive improvement.
Suggested goals are presented along with your historical metrics on greyed-out charts, and they will not send notifications to any users until you explicitly enable them.
### Enabling goals
To enable or modify a suggested goal, perform the following steps:
* Click **Set goal.** A modal appears with the suggested goal pre-populated
* To modify the suggested goal, simply enter your own goal using the **Days**, **Hours**, and **Minutes** fields.
* To learn more about how to set up "nudge" notifications for a goal, refer to [Setting up "nudge" notifications](#setting-up-nudge-notifications-for-goals)
* Click **Save** to enable the goal. Once enabled, the goal chart appears in color, and the goal line updates from a dashed line to a solid line, indicating that the goal is now enabled.
### Setting up "nudge" notifications for goals
One of the most powerful features of Goals is the ability to set up automated Slack notifications that "nudge" teams as their individual PRs begin to approach the goals they've set. Goals do not include notifications by default, but you can easily set them up by performing the following steps:
* For an already-enabled goal, click the ellipsis in the top-right of the chart and select "Edit goal". For a goal that has not yet been enabled, click "Set goal." Either action will present the settings modal for that goal:\\
* Expand the **Notification type** drop-down and select either "Smart escalations" or "Custom schedule"
* **Smart escalations:** This option will automatically send 2 progressively escalating notifications to the selected Slack channel. The first notification is sent at 75% of goal and doesn't specifically mention anyone. The second notification is sent at 90% of goal and @mentions the authors or reviewers on the PR (whichever is appropriate to the specific goal). \\
* **Custom schedule:** This option allows you to schedule up to 5 notifications. Specify the elapsed **Days**, **Hours**, and **Minutes** when the first notification should be sent (this value must be less than the goal value, as the notification is intended to warn teams *before* the goal has passed).To add additional notifications to the schedule, click **+ Add another notification**. \\
The formatting for individual notifications varies slightly depending the goal type and whether they include @mentions or an @channel, but they all generally align to the following examples:
### Interpreting goal charts
Enabled goals are color-coded to highlight the days on which the goal was or wasn't achieved
In the screen shot above, the enabled goal is represented by the solid grey line.
Each column on a goal chart represents the average value for the specific metric (in our example, Coding time) for all of the items that deployed on that day, and the columns are colored red or green according to whether or not the goal was achieved for that day.
The main heading on the top-left of the chart summarizes the daily pass/fail data as an overall percentage of days passing in the selected period, and the sub-heading presents your overall average across the entire period in relation to the goal.
The chart also displays a solid blue line representing your trend over the selected time period, and the chart's footer provides commentary on how the direction of that trend compares to your overall average for the period.
The Goals dashboard aims to answer the question "How have we been doing against the goals we've set?" Since each column on the goal charts represents the average for items that *deployed* on that day, the Goals dashboard is inherently backward-looking. However, Sleuth continuously evaluates individual PRs against your goals in real-time, and if you've [set up "nudge" notifications](#setting-up-nudge-notifications-for-goals) for a goal, Sleuth will notify teams in real-time as PRs begin to approach the thresholds you've defined.
### Disabling and re-enabling goals
Disabling a goal turns it back to grey-scale and pauses any notifications that have been set for it while preserving any goal or notification settings you've already specifie. This allows you to easily re-enable a goal with a single click.
To disable a goal, click on the ellipsis and select **Disable goal**.
To re-enable a disabled goal, click **Enable goal**.
### Resetting goals and generating new suggestions
While disabling a goal preserves your goal settings, **Reset to default** generates a fresh goal suggestion and puts the goal back into its "default" state. Note that when a goal is in the default state, Sleuth will regenerate a fresh suggestion for that goal each time you load the Goals dashboard!
# Sleuth Automations
## Overview
Sleuth Automations is a powerful capability that help teams drive improvements across their entire software delivery workflow. Whether its by removing manual tasks, ensuring consistency, or ensuring teams are continuously informed of what matters, Sleuth Automations takes care of the toil so teams can focus on what they do best: delivering customer value.
### Automations for everything
Sleuth's [Automations Marketplace](/sleuth-dora/sleuth-automations/automations-marketplace) provides a huge catalogue of point-and-click automations that can be install in seconds. The Marketplace is a great place to explore the kinds of things that are possible with Sleuth Automations, and once your creative juices are flowing, Sleuth's [Custom Automations](/sleuth-dora/sleuth-automations/actions) framework lets you take full control by writing your own automations in our highly expressive YAML framework.
### More than just a toolbox
By [suggesting automations](/sleuth-dora/sleuth-automations/automations-marketplace/smart-suggestions) that are designed to address your specific opportunities for improvement, and by providing [efficacy feedback](/sleuth-dora/sleuth-automations/automations-marketplace/understanding-efficacy) to understand which automations are making a difference to your teams, Sleuth Automations combine with Sleuth's powerful DORA metrics to provide a full-circle continuous improvement solution.
Check out this video for a quick overview of how it works!
{% embed url="" %}
# Automations Marketplace
Sleuth's [**Automations Marketplace**](https://marketplace.sleuth.io/) provides a rich offering of easy-to-install automations designed to help teams drive improvements across their entire software delivery workflow.
### Getting there
Access the Automations Marketplace either by clicking the **Marketplace** item in Sleuth's top navigation or by navigating directly to [marketplace.sleuth.io](http://marketplace.sleuth.io)
### Automation Types
The Automations Marketplace is categorized into 4 main "types" of automations:
Automation types
* **PR Checks** analyze pull requests to ensure they conform to industry best practices, cutting down review time and improving morale by relieving reviewers of the onus of “policing” these best practices. Think of them as linters for your PR process!
* **Notifications** drive awareness and help teams respond quickly by automatically alerting them in Slack or Microsoft Teams when Sleuth determines there’s something they need to know.
* **Actions** provide a variety of simple, targeted automations across your software delivery tool chain, like automatically transitioning or commenting on a Jira issue to let stakeholders know immediately when a code change they care about has deployed to its target environment.
* **Workflows** provide powerful automations that can evaluate multiple conditions and execute complex actions across that tool chain. The example I love to use here is auto-promoting a build from Staging to Production once Sleuth determines that it’s had a good soak. Sleuth can do that because it knows exactly when that change hit that environment, it knows the status of all of the monitoring tools connected to that environment, and has the ability to trigger a build in your integrated CI/CD platform.
### Additional Automation Tags
In addition to the 4 automation types above, automations are further tagged by the following dimensions. Simply click on the corresponding items in the left-hand navigation of the Automations Marketplace to filter automations that match these tags:
* **Category** describes which aspects of your software delivery process each automation is designed to improve.\
* **DORA** tags allow you to search for automations designed to improve a specific DORA metric (or sub-metric). Sleuth also uses these DORA tags to [suggest specific automations](/sleuth-dora/sleuth-automations/automations-marketplace/smart-suggestions) in the context of your unique bottlenecks.\
### Search
In addition to the above tags, you can also search across all automation titles and descriptions using the free-text search box:
# Installing Automations
We've made it incredibly easy to install automations from right from the Marketplace.
We see Sleuth Automations as part of continuous improvement *experimentation platform*, so we understand how important is to enable teams to quickly try out an automation, see if it's working for them, and then quickly move on to trying out additional automations.
The install process varies slightly depending on the [automation type](/sleuth-dora/sleuth-automations/automations-marketplace#automation-types), but all automations share the the following easy-install features:
* All automations can be installed (and also [edited or uninstall](/sleuth-dora/sleuth-automations/automations-marketplace/editing-and-uninstalling-automations)ed) from completely from within Sleuth's UI.
* The install process will never require issuing a PR or otherwise modifying your code repositories
* The install process allows you determine a specific Team or Project to which an automation will apply
{% hint style="info" %}
Most automations can be applied to either a Project or a Team, but some are limited to one or the other.
When installing an automation to a Project, the automation will apply by default to all Code Deployments within that Project, but some automations provide a configuration option for limiting their scope to specific Code Deployments within the selected Project
{% endhint %}
{% hint style="warning" %}
In order to use automations that update PRs in your Git provider, you'll need to make sure you've enabled "write" permissions for that Git provider within Sleuth.
{% endhint %}
To install an automation in your sleuth environment, follow these steps:
* [Browse the Marketplace](/sleuth-dora/sleuth-automations/automations-marketplace) to find an automation you'd like to install
* Click on the automation card to view additional details about the automation
* Click **Install in Sleuth** \\
* The next page allows you pick a Project or Team for the automation and might prompt for further configuration parameters depending on the specific automation. In this PR check example, Sleuth is asking for the maximum number of files allowed on a single PR. Other common parameters might include a Slack channel for sending notifications, an Environment selector for determining which environments an automation applies to, etc. \\
Installing and configuring a PR check automation
* Once all required fields have been specified, click **Install and continue**
* Sleuth will display a toast indicating that the installation was successful, and the installed automation now appears under the **Automations** item in the left-hand-navigation pane for the selected Project/Team.\\
Viewing installed automations for a specific Project
# Installing PR "Update" Automations
Some of Sleuth's automations can directly update pull requests by doing things like automatically closing PRs, adding PR labels, commenting on PRs, etc.
These pull request "update" automations are currently available only for GitHub and require the **Sleuth Automations for GitHub** application to be installed in your GitHub organization, as this application grants Sleuth the additional write permissions to perform updates to pull requests. See [Code integrations (write)](/sleuth-dora/integrations-1/code-integrations-write) for more information on the specific permission granted by this application.
Sleuth will let you know before you try to install one of these automations if the **GitHub Automations for GitHub** application is required.
Note that [PR Check automations](/sleuth-dora/sleuth-automations/automations-marketplace#automation-types) do **not** require the **GitHub Automations for GitHub** application. PR Checks can be installed using any of Sleuth's standard [code "read" integrations](/sleuth-dora/integrations-1/code-deployment) for GitHub, GitLab, BitBucket, or Azure DevOps.
# Editing and uninstalling Automations
Automations are easy to edit or uninstall from directly within Sleuth! To edit or uninstall an automation, perform these steps:
* Navigate to the Project or Team whose automations you want to manage and click the **Automation** item in the left-hand-pane
Installed autoamtions for a specific Project
* Click on the automation you want to manage to open its details screen
Installed automation details screen
* Click the **Installed** button to display options for editing or uninstalling the automation.
* Click **Edit** to open the automation's configuration screen, where you can now modify the parameters originally specified during installation. When finished editing, click **Save**
Editing the configuration for an installed automation
* To uinstall the automation, simply click Uninstall. A toast will indicate that automation was successfully uninstalled, and the automation no longer appears on the **Automations** screen for the current Project/Team.
Automation successfully uninstalled from a Project
# Smart suggestions
Sleuth automatically suggests automations designed to address your specific bottlenecks!
Sleuth displays smart suggestions for metrics that have worsened by 30% or more since the prior period
### Viewing smart suggestions
On the Project Metrics and Team Metrics dashboards, when a DORA metric or sub-metric has worsened by 30% or more compared to the prior period, Sleuth displays a suggestion prompt like the one in the screen shot above.
Click **View automations** to navigate to a pre-filtered view of the Automations Marketplace, where you easily view all available automations that are tagged with your ailing metric.
Filtered view of the Automations Marketplace showing automations designed to improve Review time
When multiple metrics have worsened by 30% or more, the suggestion prompt will also include a **More suggestions** link. Click it to view the additional suggestions. Click the arrow next to any suggestion to view automations designed to help that specific metic.
### Dismissing smart suggestions
Click the **X** in the top-right corner of the suggestion banner to dismiss all smart suggestions for the remainder of your current Sleuth session.
Click **Don't show again for this metric** to more permanently dismiss suggestions for a specific metric. Suggestions for that metric will remain suppressed across all Projects and Teams for your Sleuth user. You can re-enable suggestions at any time by clearing your browser cache.
# Understanding efficacy
Sleuth provides an option on both the Project Metrics and Team Metrics dashboards to view the install dates for your installed automations within the context of your metrics, allowing you to see at a glance whether those automations are having the intended impact!
Viewing automation install dates within the context of DORA metrics
To view the install dates for installed automations within the Project Metrics or Team Metrics dashboards, perform the following steps:
* Select a specific Project or Team from the **Switch to** menu.\
* Click the lightning bolt icon to display automation install dates on the chart.
NOTE: If you don't see the lightning bolt icon, it's likely because you have no automations installed for the current Project/Team. To install an automation, refer to [Installing automations](/sleuth-dora/sleuth-automations/automations-marketplace/installing-automations).
* The date range selection auto-updates to a period that starts from the earliest automation installation date and ends one week following the most recent installation (or the current date when it has been less than one week since the last automation was installed).
* The vertical lines on the chart represent the installation dates for installed automations. Hover over any vertical line to view the specific automation that was installed on that date. If multiple automations were installed on that date, they will all be referenced within the tool tip.
* To reset the date range back to your original selection, click **Reset range**
We're working on on adding much more granular visibility into the efficacy of individual automations, so stay tuned for more to come!
# Custom Automations
Sleuth's Custom Automations framework allows you to build bespoke automations to suit your organization's specific workflows by putting our powerful and highly expressive YAML-based automation capability into your hands.
### How it works
All of the automations in Sleuth's [Automations Marketplace](/sleuth-dora/sleuth-automations/automations-marketplace) are built on an underlying YAML-based rules framework that allows you to define an automation by combining one or more **triggers** (e.g. a PR update, a code deployment, or a user-defined schedule), **conditions** (e.g. a code deployment is "healthy"), and **actions** (e.g. send a Slack message, trigger a build). Here's an example of what an automation rule might look like in YAML:
```
rules:
- stage-to-prod:
description: Automatically promotes a healthy staging deployment to production
conditions:
- environment='Staging'
- health='Healthy'
- deployed_for>'10m'
actions:
- auto_approve_build: test-and-deploy
- add_to_deploy_message_thread: >-
Build promoted automatically on a healthy staging deploy
```
A rule can be named whatever you want, `stage-to-prod` in this example, and can have a description to display in the Sleuth interface. In this example, the automation's conditions check that a deploy has soaked healthily for a specified amount of time, and then the actions automatically approve a pending build to the next environment and send a notification to the Slack deployment message thread. Another common and powerful action is the [webhook](/sleuth-dora/sleuth-automations/actions/webhook), which can be used to notify external systems.
For more ideas on what tasks can be automated, see [our cookbook](/sleuth-dora/sleuth-automations/actions/cookbook).
### Two methods for installing and maintaining custom automations
There are two ways to install and maintain custom automations.
#### Method 1: Add a YAML file to your code repositories
The first method is to define your rules in a YAML file and then insert that YAML file into each code repository where you want those rules to be evaluated (located at `.sleuth/rules.yml`). The advantage of this approach is that you can modify your custom automations "as code" (e.g. enforcing your SDLC process around changes to rules and ensuring preservation of historical states). The disadvantage is that it's more effort to deploy rules to multiple repositories (since each requires its own rules.yaml file) and to make changes to rules over time (since rules changes will be treated like any other code changes).
#### Method 1: Add custom automations from Marketplace
The second method is to use the [Custom Automation template](https://marketplace.sleuth.io/?filter=custom) provided in Sleuth's Automations Marketplace.
This option allows you to quickly write a new automation (using YAML) and deploy it to multiple teams or projects, all from within Sleuth's UI. You can also modify or uninstall those automations quickly from within Sleuth's UI.
This option is great for quickly experimenting with automations, putting them out to team quickly, tweaking them based on feedback, and if they aren't producing the intended effect, removing them as quickly as you added them. While this option streamlines experimentation by removing the need to manually add a rules.yaml files to multiple repositories or to issue pull requests each time you want to change those rules, it might not conform to some organizations' policies for managing "rules as code".
The quickest way to get started with custom automations is to copy the YAML either from an existing automation or from our automations [cookbook](https://help.sleuth.io/sleuth-automations/actions), then tweak it to meet your organization's unique needs. Access the YAML for any existing automation on the Marketplace by navigating to its YAML tab as shown here:
### Ready to automate?
For detailed information on the YAML-based framework, including what specific triggers, condition variables, and actions are available, click on "Help" from within Sleuth, then click "Custom Automations".
# Automations Cookbook
To more easily start using Sleuth Custom Automations, here are several recipes that you can either try today or if labeled with "(coming soon)", may be available in the future. To vote for these proposed use cases or suggest new ones, please [let us know](mailto:support@sleuth.io)
## Deployment promotion
### Auto-promote staging deploys to production
When code is deployed to the Staging environment and has been determined to be healthy by Sleuth, auto approve any manual approval steps in the associated build.
```
rules:
- staging-promotion:
conditions:
- environment='Staging'
- health='Healthy'
- deployed_for>='5m'
actions:
- auto_approve_build: 'test-and-deploy'
```
### Auto-promote quickfix staging deploys to production
When you want your code shipped quickly or know you are shipping something can't break production, add a 'quickfix' label to your pull/merge request and it'll automatically be promoted to production.
```
rules:
- quick-fixes:
conditions:
- environment='Staging'
- pr_labels='quickfix'
actions:
- auto_approve_build: 'test-and-deploy'
```
### Auto-approve a specific job
When you want to auto-approve a specific job in your build, because you have several approvals in your workflow.
```
rules:
- dev-to-staging:
conditions:
- environment='Staging'
actions:
- auto_approve_build:
build: 'test-and-deploy'
job: 'approve-to-staging'
```
### Slack approvals-based promotion
The first rule responds to the staging deployment message in Slack to let people know how to promote. The second rule fires the number of :thumbsup: reactions is 3 or more and there are no :thumbsdown: veto votes. If successful, the deploy is promoted to production.
```
rules:
- promote-staging-to-prod-notify:
conditions:
- environment='Staging'
actions:
- add_to_deploy_message_thread: |
Please vote to approve +/-1 to promote this deployment {{slack_mention_authors}}
- promote-staging-to-prod-on-approval:
conditions:
- environment='Staging'
- health='Healthy'
- deployed_for>='5m'
- deploy_message_reaction_plus_1>=3
- deploy_message_reaction_minus_1=0
actions:
- auto_approve_build: 'test-and-deploy'
```
### Auto-promote after soak time
When code is deployed to staging for at least 4 hours and is still healthy, then promote to production.
```
rules:
- staging-to-prod-after-soak:
conditions:
- environment='Staging'
- health='Healthy'
- deployed_for>'4h'
actions:
- auto_approve_build: 'test-and-deploy'
```
### Separate approval workflow for third-party PRs (coming soon)
Pull/merge request merges, from non-trusted developers, aren't autopromoted from staging to production, but instead send a Slack message to the `#deploy-requests` channel for someone to promote it for them.
Note that the slack message is markdown and created with the [Jinja](https://jinja.palletsprojects.com/en/2.11.x/) template language.
```
rules:
- staging-to-prod-third-party:
conditions:
- environment='Staging'
- pr_authors!='teamlead1' AND pr_authors!='srdev1'
actions:
- auto_approve_build: 'test-and-deploy'
- slack_channel_message:
channel: '#deploy-requests'
message: |
A {{deployment_name}} is awaiting approval:
* <{{pr.url}}|{{pr.title}}> - by {{ pr.author}}
```
## Deployment notifications
### Custom message to author when deploy is unhealthy
When a deployment to production is determined by Sleuth to be unhealthy, notify any author involved in the deployment, usually commit and pull request authors. Send them a personal slack notification with a custom message containing links to key dashboards, logging systems, runbooks, and whatever other resources they need to resolve the incident.
```
rules:
- author-unhealthy:
conditions:
- environment='Production'
- health='Unhealthy'
actions:
- slack_personal_message:
group: authors
meessage: |
Your code was <{{deploy_url}}|deployed> and is unhealthy. Check these links:
*
*
*
```
### Notify the project lead on certain deploys (coming soon)
Sleuth can tag deploys, either based on file paths matched from the contents of the deploy, or via tags passed explicitly when registering the deploy. In this rule, when production deployment is created and the deploy is tagged with `api_change` or `database_migration`, notify the team lead with a personal slack message.
```
rules:
- notify-sara:
conditions:
- environment='Production'
- tags='api_change' or tags='database_migration'
actions:
- slack_personal_message:
email: sara@example.com
message: |
An important <{{deploy_url}}|deployment> went out tagged with {{deploy_tags}}.
```
### Notify key issues on certain deploys (currently for Jira and Linear only)
When a blocker issue is resolved with a deployment to production, add a comment on the issue to notify any watchers of that issue that it has been fixed.
```
rules:
- notify-important-issues:
conditions:
- environment='Production'
- issue_priority='Blocker'
actions:
- notify_mentioned_issues
```
### Notify support team on bugs and new features that impact them (coming soon)
When code is deployed to production, find any issues that are either bugs or labeled with `support`, and send them to the Slack `#support` channel to notify the support team so that they can update the customer.
```
rules:
- notify-support:
conditions:
- environment='Production'
- issue_labels='support' or issue_type='Bug'
actions:
- slack_channel_message:
channel: '#support'
message: |
{{deployment_name}} was deployed with the following relevant issues
* <{{issue.url}}|{{issue.key}} - {{issue.title}}>
```
### Notify in Slack when drift is too high
When a deploy hits staging, and staging is more than 10 deploys different than production, send a Slack channel notification to `#dev` warning them that they need to promote to production soon.
```
rules:
- drift-too-high:
conditions:
- environment='Staging'
- drift_to_production>10
actions:
- slack_channel_message:
channel: '#dev'
message: |
Drift to production too high
```
### Notify an internal app when code is deployed
When a deploy hits production, send a webhook to an internal system. See [webhook](/sleuth-dora/sleuth-automations/actions/webhook) for more information.
```
rules:
- notify-internal-app:
conditions:
- environment='Production'
actions:
- webhook: https://myapp.example.com/deployment/production
```
## Deployment miscellaneous
### Transition issues on deploy (currently for Jira and Linear only)
When code is deployed to production, find any referenced issues and transition them into the `Deployed` state.
```
rules:
- transition-issues-on-deploys:
conditions:
- environment='Production'
- health='Healthy'
- deployed_for>='5m'
actions:
- transition_mentioned_issues:
state: Deployed
comment: |
This issue has been deployed by {{deploy_author_name}} in <{{deploy_url}}|{{deploy_name}}>
```
### Create revert PR when unhealthy (coming soon)
When a deployment to production is unhealthy, create a pull/merge request that reverts the deployment. Send a personal Slack message to the authors of the deployment with a link to the revert PR so that they can quickly fix the deployment.
```
rules:
- revert-pr-on-unhealthy:
conditions:
- environment='Production'
- health='Unhealthy'
actions:
- create_revert_pr
- notify_deploy_authors: |
Your code was <{{deploy_url}}|deployed> and is unhealthy, and a PR was created to revert {{pr_url}}
```
### Create backport PR on deploy (coming soon)
When a deployment goes to production, create a pull/merge request that backports the changes to staging. Useful if hot fixes were applied directly to the production branch and need to be backported to the staging branch.
```
rules:
- backport-pr:
conditions:
- environment='Production'
actions:
- create_pr:
target_env: Staging
source_env: Production
title: |
Backport of production to staging for release {{deploy_name}}
- notify_deployer: |
Your code was <{{deploy_url}}|deployed> and here is the backport PR: <{{backport_pr_url}}>
```
### Slow rollout of feature flags after delivery (coming soon)
When code is shipped to production, and it comes from a pull/merge request that has the `auto-flag-rollout` label, and is healthy, then gradually enable related feature flags over a period of time.
```
rules:
- feature-flag-rollout:
conditions:
- environment='Production'
- pr_label='auto-flag-rollout'
- health='Healthy'
- deployed_for>='5m'
actions:
- gradually_enable_feature_flag:
increment: 10%
interval: 20m
on_unhealthy: revert
```
# Webhook Actions
The `webhook` action will send an HTTP POST to the URL of your choosing. The body contains information about the deployment, and a signature to verify the webhook is valid and originated at Sleuth.
## Body
The body of the webhook will look something like this:
```
{
"organization": {
"slug": "myorg"
},
"name": "791f0e0",
"on": "2021-02-03T00:35:09.823706+00:00",
"branch": "master",
"revision": "791f0e0ce8612717da226d73caac7790e0de4032",
"environment": {
"slug": "production"
},
"changeSource": {
"slug": "sleuth-test"
},
"health": "healthy",
"author": {
"email": "nobody@example.com"
}
}
```
## Signature
The webhook will contain two headers you can use to safely validate the message:
* `X-SLEUTH-TIMESTAMP` - The unix timestamp, in seconds, of when the webhook was created
* `X-SLEUTH-SIGNATURE` - The signature of the timestamp and request body, signed with the API key
The signature follows the format Slack uses to validate their webhooks. For more information how it works and how to validate the signature, see [the Slack docs](https://api.slack.com/authentication/verifying-requests-from-slack)
The only difference is Sleuth uses the API key from your Sleuth [organization](/sleuth-dora/settings/organization/details) as the signing key.
# Trigger Build Actions
The `trigger_build` is an extremely powerful action that will trigger your repository's build. You can control what workflow is executed by specifying the parameters. Your [CI/CD integration](/sleuth-dora/integrations-1/builds) defines the required parameters for Sleuth to trigger the build process.
The `trigger_build` action is best used in concert with Sleuth's [Slack based Approvals](/sleuth-dora/slack-mission-control/approvals). Together they can be used to define your deployment workflow.
| Integration |
| --------------------------------------------------------------------------------------------------------- |
| [Bitbucket Pipelines](/sleuth-dora/sleuth-automations/actions/trigger_build/trigger-build-bb-pipelines) |
| [CircleCI](/sleuth-dora/sleuth-automations/actions/trigger_build/trigger-build-circleci) |
| [Github Actions](/sleuth-dora/sleuth-automations/actions/trigger_build/trigger-build-gh-actions) |
| [Jenkins](/sleuth-dora/sleuth-automations/actions/trigger_build/jenkins) |
# Bitbucket Pipelines
## Usage
The only **required parameter** for Sleuth is the **name of the pipeline** specified in your`./bitbucket-pipelines.yml` file. Sleuth will pass any additional specified parameters to Bitbucket API when triggering the build in case your workflow requires additional input parameters. See [Bitbucket Pipelines documentation](https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Bworkspace%7D/%7Brepo_slug%7D/pipelines/) for additional informations.
This example triggers a `deploy-prod` pipeline in Bitbucket when code is deployed to the "Staging" environment for more than 4 hours and is healthy:
```
rules:
- run-deploy:
conditions:
- environment='Staging'
- deployed_for>'4h'
- health='Healthy'
actions:
- trigger_build:
parameters:
name: 'deploy-prod'
my-custom-non-required-param: 'we are going live'
```
# CircleCI
## Usage
This example triggers a deployment in CircleCI when code is deployed to the "Staging" environment for more than 4 hours and is healthy:
```
rules:
- run-deploy:
conditions:
- environment='Staging'
- deployed_for>'4h'
- health='Healthy'
actions:
- trigger_build:
parameters:
run_deploy: true
environment: production
```
It passes several parameters to be used by CircleCI: `run_deploy` and `environment`. For more information about how to declare parameters and filter workflows in CircleCI, see the [CircleCI documentation](https://circleci.com/docs/2.0/pipeline-variables/#pipeline-parameters-in-configuration).
This is an example of the CircleCI configuration that uses the `run_deploy` parameter to selectively execute a workflow, while the `environment` parameter is used within the job to perform the deployment:
```
parameters:
run_deploy:
default: false
type: boolean
environment:
type: string
default: staging
jobs:
run-deploy:
docker:
- image: circleci/python:3.8.6
steps:
- run:
command: |
echo "Deploying to <>"
workflows:
deploy:
when: << pipeline.parameters.run_deploy >>
jobs:
- run-deploy
```
# Github Actions
## Usage
By default Github Actions can not be triggered via the API. To enable Sleuth to trigger the workflow, you need to add `workflow_dispatch` option in `on` section:
```
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events
# but only for the master branch
push:
branches: [ master ]
pull_request:
branches: [ master ]
# Allows Sleuth to triggered the workflow through API
workflow_dispatch:
```
The only **required parameter** for Sleuth is the **name of the workflow file** located in your repository `./github/workflows/` folder. Sleuth will pass any additional specified parameters to Github Actions API when triggering the build in case your workflow requires additional input parameters. See [Github Actions documentation](https://docs.github.com/en/actions) for additional informations.
This example triggers a `deploy-prod` workflow in Github when code is deployed to the "Staging" environment for more than 4 hours and is healthy:
```
rules:
- run-deploy:
conditions:
- environment='Staging'
- deployed_for>'4h'
- health='Healthy'
actions:
- trigger_build:
parameters:
name: 'deploy-prod.yml'
my-custom-non-required-param: 'we are going live'
```
# Jenkins
## Usage
The only **required parameter** for Sleuth is the \*\*name of the Jenkins job \*\*. Sleuth will pass any additional specified parameters to Jenkins when triggering the job in case you are using parameterised builds. See [Jenkins documentation](https://www.jenkins.io/doc/book/using/remote-access-api/) for additional informations.
This example triggers a `deploy-prod` job in Jenkins when code is deployed to the "Staging" environment for more than 4 hours and is healthy:
```
rules:
- run-deploy:
conditions:
- environment='Staging'
- deployed_for>'4h'
- health='Healthy'
actions:
- trigger_build:
parameters:
name: 'deploy-prod'
my-custom-non-required-param: 'we are going live'
```
# Slack & Email Notifications
Sleuth can send emails or Slack notifications when something of significance happens, including messages for your entire team, yourself, and/or to commit and pull request authors.
{% hint style="info" %}
Using Microsoft Teams? Check out the [Automations Marketplace](https://marketplace.sleuth.io/?filter=microsoft_teams) to see our rich library of notifications for Microsoft Teams
{% endhint %}
## Setting up Slack notifications
Before you set up project-level Slack notifications, the [Sleuth DORA App for Slack](/sleuth-dora/integrations-1/slack#about-the-integration) must first be enabled in your Slack workspace. Once that's done, you can configure who and how those notifications are sent and who receives them.
{% hint style="info" %}
You must be the owner of or have admin access to an organization to setup an integration. See [Access control](/sleuth-dora/settings/access-control) for more information.
{% endhint %}
### Setting up project-level Slack notifications
1. Select a project in the sidebar, then click **Project Settings**.
2. Click **Slack Notifications**.
3. In the *Slack channel* dropdown, start typing the name of the Slack channel that will receive change notifications. If the channel is private, you will need to invite the *Sleuth bot* to the channel first.\
4. Click **Save**.
At this point, you can notify users in your Slack organization that a notifications channel for this project has been set up and that they should join it to receive any and all change notifications that occur in that project.
### Setting up personal Slack notifications
In the previous section, you created a project-level Slack notification. Team members will only receive change notifications if they join the corresponding Slack channel. However, you might want customized notifications sent directly to yourself. You can do this by configuring your user-level Slack notifications preferences:
1. Select your username in the bottom of the sidebar, then click **Manage Account**.
2. Click **Notifications**.
3. Any [email notifications](#setting-up-email-notifications) you've already set up will be displayed here. Enable the **Slack notifications** toggle. sends a notification only if a deployment occurs in which the impact of your code on production errors is anything except
4. You can enable or disable **Deployed code** and/or **Impact of your Code**:
* **Deployed code**: Selecting **All** means you will receive a Slack notification every time code in which you are the author is deployed.\
Selecting **Exclude my deployments** notifies you of all deployments except those in which you're the author. This option is great if you're up to speed on your own code but want to keep tabs on how the rest of your team's deployments are doing.
* **Impact of your Code**: Selecting **All** means you will always receive a Slack notification about the impact of your code on the rate of production errors.\
Selecting \*\*Exclude healthy \*\*will only notify you when a deployment is not fully healthy\_; \_this includes any that are marked as *Unhealthy*, *Ailing*, or *Improved*.\
{% hint style="info" %}
Read the [Sleuth Privacy Policy](https://www.sleuth.io/privacy) for information regarding the collection, use, and disclosure of Personal Information we collect.
{% endhint %}
## Setting up email notifications
Email notifications are sent at the frequency you select, and can be configured individually at the project and change source level.
### To set up at the project level
1. Select a project in the sidebar.
2. Click **My Notifications** in the upper-right corner of the Dashboard.
3\. Select a notification frequency in the dropdown. More than one can be selected.
### To set up at the code deployment or feature flag level
1. Select a **project** in the sidebar. The project's dashboard is displayed.
2. Select a **code deployment or feature flag** in the sidebar and click on its title link. The dashboard for the code deployment or feature flag is displayed.\\
3. Click the bell icon for **My Notifications** in the upper-right corner of the dashboard. \\
4. Select a notification frequency in the dropdown. More than one can be selected.
{% hint style="info" %}
Add **** to your email provider's or your email application's spam filter whitelist to prevent the notification email from getting caught up in spam filters.
{% endhint %}
The notification email provides a digest of the following activity during the selected timeframe:
* Number of deployments made from each project in your organization
* Number of pull requests
* Number of commits
* Number of issues
* Number of changed files
* Number of unique authors
### To set up at the Team level
Team-level notifications keep you and your teams up to date on how they're performing across projects. To set up team-level email notifications, perform the following steps:
1. Select a **Team** in the sidebar. The Team's dashboard is displayed.
2. Click the bell icon for **My Notifications** in the upper-right corner of the dashboard.
3. Select a notification frequency in the dropdown. More than one can be selected.
{% hint style="info" %}
Add **** to your email provider's or your email application's spam filter whitelist to prevent the notification email from getting caught up in spam filters.
{% endhint %}
# Auto-verify deploys
Sleuth integrates with your Error and Metrics trackers to automatically verify the health of your deploys
Once a deploy has shipped how does your team know if the change was good or bad? How long does it take for your team to discover this? Minutes, hours, days or weeks?
[Impact tracking](/sleuth-dora/integrations-1/impact-sources) in Sleuth uses your [key SLIs](https://en.wikipedia.org/wiki/Service_level_indicator) to let you know if your deploys are getting the job done or causing new problems. Coupled with personal [Slack notifications](https://help.sleuth.io/notifications), developers know the impact of their changes moments after they deploy.
Sleuth monitors the SLIs that define your application’s health and applies [anomaly detection](/sleuth-dora/auto-verify-your-deploys/anomaly-detection) to automatically verify the health of your deploys. Sleuth [integrates with your existing best-in-class](/sleuth-dora/integrations-1/impact-sources) error trackers and observability tools. Get Impact tracking setup in just 5 minutes!
Auto-verification of your deploys with Impact tracking allows you to:
* Easily include production health in your developers definition of done, no more tracking down graphs and trying to understand what normal should look like, Sleuth alerts authors directly in Slack when a deploy is unhealthy
* Avoid creeping, incremental declines in performance, memory, cpu usage and more. Know when your changes take your service out of its normal range!
* Automate deploy locking or creating rollback pull requests when Sleuth sees an unhealthy deploy
# Anomaly detection
Sleuth applies anomaly detection algorithms to your impact data to do the heavy lifting required to determine what is normal and what is not.
All developers want to ship changes that are making systems better, not worse. However, in our modern environments it's often difficult to know:
* What metrics are important?
* Where do you find those metrics?
* And once found, what's normal and what's something to be worried about?
Trust Sleuth to let you know when your [metrics](/sleuth-dora/auto-verify-your-deploys/metric-impact) or [errors](/sleuth-dora/auto-verify-your-deploys/error-impact) are out of your normal range. We'll let you know within 2 minutes of your last deploy, right in Slack, where your developers already live.
The more data Sleuth collects the more we know what's normal for your team. For example, perhaps you're a team that keeps your errors empty by fixing every error that occurs. Or perhaps, which is often more realistic, you always have a "normal" number of errors that you've decided you're willing to accept. Sleuth's anomaly detection means your developers don't need to "magically know" which errors are normal and which are out of your norm.
# Error impact
Error monitoring services help DevOps teams discover, triage, and prioritize the errors their applications produce in real-time. Error impacts are specific to a Sleuth [project](/sleuth-dora/modeling-your-deployments/projects) and [environment](/sleuth-dora/modeling-your-deployments/environment-support) and will be used to determine the health of that project/environment combination.
Sleuth uses anomaly detection to determine your teams healthy range for the number of errors your application generates and will alert you and mark deploys as unhealthy when they are outside of your normal range. With each deploy, the error rate is sampled and correlated to the change that may have caused it.
Sleuth supports the existing tools you already use such as [Sentry](/sleuth-dora/integrations-1/impact-sources/errors/sentry), [Rollbar](/sleuth-dora/integrations-1/impact-sources/errors/rollbar), [Bugsnag](/sleuth-dora/integrations-1/impact-sources/errors/bugsnag) and [Honeybadger](/sleuth-dora/integrations-1/impact-sources/errors/honeybadger).
Error rates are displayed on the health graphs throughout the application.
# Metric impact
**Metric trackers** or Observability tools track SLIs and data from key stats reported by your applications. Connect your cloud metrics provider to Sleuth so we can automatically verify the health of your deploys. Metrics impact are specific to a Sleuth [project](/sleuth-dora/modeling-your-deployments/projects) and [environment](/sleuth-dora/modeling-your-deployments/environment-support) and will be used to determine the health of that project/environment combination.
{% hint style="info" %}
Health verification is different from alerting! Alerting is for when your system is in strife and the on-call must be contacted. Impact lets you know when a metrics has changed in a significant way. For example, a new code change cause 5% more memory to be used. This may not breach an alerting level but it is something the developer who made the change should know and choose to accept or to fix.
{% endhint %}
Metrics trackers such as [AWS CloudWatch](/sleuth-dora/integrations-1/impact-sources/metrics/aws-cloudwatch), [Datadog](/sleuth-dora/integrations-1/impact-sources/metrics/datadog), [NewRelic](/sleuth-dora/integrations-1/impact-sources/metrics/newrelic) and [SignalFx](/sleuth-dora/integrations-1/impact-sources/metrics/signalfx) allow users to specify a query, specific to the provider, that allows you us retrieve complex data and transforms from the system. When [creating a Metric impact](/sleuth-dora/settings/project/impact) in Sleuth we will ask for a query that defines the value we will track:
You will need to be familiar with your metrics providers query language to configure a metric impact in Sleuth. Sleuth allows you to test your query to make sure we'll get the right values for your metric.
# Ignoring pull requests
Whether it's because of Dependabot PRs or those one-off, unusually long deploy times, rest assured you can always ignore specific PRs from Sleuth's DORA metrics, Work in Progress, Goals, and notifications.
You can ignore one-off PRs from within Sleuth, and you can also use [Sleuth Automations](https://marketplace.sleuth.io/?filter=action\&search=ignore) to quickly set up rules that will automatically ignore PRs that meet certain conditions (e.g. those pesky Dependabot PRs).
### Automatically ignoring PRs
Check out the canned ignore rules available in Sleuth's [Automations Marketplace](https://marketplace.sleuth.io/?filter=action\&search=ignore) for ignoring PRs based on PR author, label, title, description, and more, or create your own ignore rules using Sleuth's [Custom Automations framework.](/sleuth-dora/sleuth-automations/actions)
### Manually ignoring PRs in Sleuth
#### Igoring PRs from the Work in Progress dashboard
To manually ignore a PR from Sleuth's [Work in Progress](/sleuth-dora/work-in-progress) dashboard, click **Ignore** from the actions ellipsis menu on any pull request. Click **Unignore** to re-include the PR in your work in progress metrics. Note that any PRs ignored from Work in Progress dashboard will continue to be ignored from the DORA Metrics dashboard once those PRs ship. They will also be excluded from any [Goals charts](/sleuth-dora/goals) or ["nudge" notifications](/sleuth-dora/goals#setting-up-nudge-notifications-for-goals).
#### Ignoring PRs from the DORA Metrics dashboard
To manually ignore a PR that has already deployed, open the deploy's details page from the Metrics dashboard, and click the **Ignore** icon for any PRs within the deploy that you want ignore from your DORA metrics. Click the icon again to re-include the PR in those metrics. Lead time metrics are recalculated on-the-fly as you ignore/unignore PRs.
# Slack mission control
Slack is where developers live, Sleuth understands this and provides a deep integration with Slack so you can get the right notifications and get your deploy-related work done without having to leave Slack.
The [Sleuth DORA App for Slack ](/sleuth-dora/integrations-1/slack)is your mission control for all your deploy needs, providing:
* [Team notifications](/sleuth-dora/slack-mission-control/team-notifications) for deploys and health impact, keep everyone informed
* [Personal notifications](/sleuth-dora/slack-mission-control/personal-notifications) so you know right when your code ships and its impact
* [Search your deploys](/sleuth-dora/slack-mission-control/search-sleuth-in-slack) to find out if your code or that fix you've been waiting on has shipped
* [Instant standup reports](/sleuth-dora/slack-mission-control/developer-standup), let Sleuth remember what you've worked on today
* [Collect approvals](https://help.sleuth.io/actions/cookbook#slack-approvals-based-promotion) via a 👍 to promote deploys to new environments (beta feature)
# Approvals
Approvals are an important component of Sleuth Actions that allows you to set up custom approval-based workflows. They are typically triggered by particular deploys, display an interactive dialog in Slack and execute various actions upon successful approval.
Let's walk through a real-life scenario and see how they work.
## Setting up an approval
Imagine we'd like to set up an approval that triggers every time a deploy is made to our *Staging* environment and allows developers to promote code to the *Production* environment.
First, visit the deployment page of a code deployment and select *Add an approval* from the cogwheel menu in the top right corner.
This will open the approval creation wizard and guide you through the process.
Read the introduction and move to the second step by clicking *Next*.
Specify a name for your approval, select the *Staging* environment (because we want approvals to show up after we deploy to *Staging*) and define various aspects of the approval dialog, such as the Slack channel to display it in.
Under *Action*, select the action that should be executed once the approval is successfully approved. In our case, we want to approve an existing preconfigured CircleCI build that will promote that same code to *Production*.
The most powerful way to use approvals is to have an action that triggers a CI/CD build. In this way you can use Sleuth to define your deployment workflow. See our documentation on how to configure build triggers for more details.
{% content-ref url="/pages/-MUK-Su0bG2FIeXu4HGv" %}
[Trigger Build Actions](/sleuth-dora/sleuth-automations/actions/trigger_build)
{% endcontent-ref %}
Once you're happy with the configuration, move to the final step by clicking *Next* again.
The final step will provide you with the contents of a configuration file and specific instructions on how to enable it by adding it to your code repository. Once you do that, Sleuth will automatically parse it and display an approval on your next deploy.
## Interacting with the approval
The configuration we just defined in the example above will display the following approval dialog in Slack every time a deploy is made to the relevant code deployment.
Once a member of your team clicks *Approve* or *Reject*, their vote will be recorded and the approval dialog will update.
If the approval is successful, the configured action(s) are automatically executed in the background - in this case that means our CircleCI build is approved.
## Inspecting approval logs
Sleuth keeps track of your approvals to provide additional visibility. If you ever need to find out exactly what happened on a specific deploy, visit its detail page and open the *Approvals* tab.
Here, you can see which approvals were triggered by the deploy, how your team members voted on them and what actions were triggered as a result.
# Project notifications
When your team is deploying frequently keeping everyone in the loop about what's changing is crucial.
Project-level team Slack notifications provide the context your teams need to keep the code flowing —automated, easily-digestable messages for:
* when your deploy occurred
* what was deployed
* who were the authors of the deploy
* how many commits, PRs, issues, and changes were in the deploy
* any issues associated with the deploy
* the [auto-generated health](/sleuth-dora/auto-verify-your-deploys) of the deploy
Team notifications are flexible and Sleuth allows you to configure a different channel for each Sleuth [environment](/sleuth-dora/modeling-your-deployments/environment-support).
# Personal notifications
As a developer, how do you know when you code has really been deployed? If you are lucky your deploy process takes 5 minutes and you're the one running it. However, for most of us, deploys take longer, or maybe even don't involve you when things actually ship to production.
Sleuth's got your back! Personal Slack notifications will notify the authors of change when their code is deployed.
Even better, if your team is using [auto-deploy verification](/sleuth-dora/auto-verify-your-deploys) Sleuth will tell you when your change has caused a problem.
Personal notifications aren't just for Developers. Sleuth will alert anyone involved in the [issues](/sleuth-dora/integrations-1/issue-trackers) deployed as well. If you are PM or Designer that's trying to understand when work is really hitting your customers, Sleuth is your best friend.
## Slack deploy summaries
Sleuth provides individuals a daily, weekly, bi-weekly or monthly summary of what was deployed, delivered right to you via Slack!
These can be configured via the [project](/sleuth-dora/modeling-your-deployments/projects) or [deployment](/sleuth-dora/modeling-your-deployments/code-deployments) dashboards.
Once enabled you'll receive the same rich deploy Slack notifications you've come to expect from Sleuth delivered right to you.
# Search Sleuth in Slack
When was the last time you were in a team standup and someone asked the question "Did that pull request ship yet?"
Sleuth is your, well ... sleuth, for all your deploy questions. Just ask Sleuth in slack and never wonder again.
```
/sleuth billing
```
# Project/Deployment history
Sometimes you just need to know what the last few deploys were for your [project](/sleuth-dora/modeling-your-deployments/projects) or [deployment](/sleuth-dora/modeling-your-deployments/code-deployments).
With the Sleuth history Slack command you can see that context right from Slack.
```
/sleuth history [project name/slug | deployment name/slug]
```
# Developer standup
Tired of trying to remember what you did yesterday? Me too. The Sleuth Slack standup command helps you build your standup report, pre-populating it with your deployments and other pull request development activity. Run this in any Slack channel:
```
/sleuth standup
```
And you'll see a dialog that when submitted, reports your standup to the channel:
{% embed url="" %}
# Sleuth API
The Sleuth REST API provides methods that enables users to:
* Register or import their deploys
* Create [manual changes](/sleuth-dora/modeling-your-deployments/manual-changes)
* Register [custom impact values](/sleuth-dora/integrations-1/impact-sources/metrics/custom)
Sleuth's main public API is built using GraphQL. It's the same API we use internally for developing our applications.
If you're new to GraphQL, Apollo has [resources for beginners](https://blog.apollographql.com/the-basics-of-graphql-in-5-links-9e1dc4cac055). [The official documentation](https://graphql.org) is another good starting point.
{% hint style="info" %}
NOTE: the GraphQL API is still under heavy development and is subject to change
{% endhint %}
Sleuth's GraphQL endpoint is:
```
https://app.sleuth.io/graphql
```
We expose the [GraphiQL](https://github.com/graphql/graphiql) client so you can explore and query the API.
{% hint style="info" %}
See [GraphQL examples](/sleuth-dora/sleuth-api/graphql-examples) to see how to authenticate your requests using Sleuth API Key.
{% endhint %}
## Authentication
The Sleuth REST API requires authentication using the API key from your Sleuth [organization](/sleuth-dora/settings/organization/details).
## Provisioning Sleuth with Terraform
For Organizations with many [Projects](/sleuth-dora/modeling-your-deployments/projects), [Code Deployments](/sleuth-dora/modeling-your-deployments/code-deployments) and [Impact Sources](/sleuth-dora/integrations-1/impact-sources) configuring Sleuth via the UI can be cumbersome. The Sleuth API can be used to provision resources directly. However, many teams already rely on [Terraform](https://www.terraform.io/) to provision their infrastructure and other resources.
Instead of using the API directly to provision Sleuth resources, you can use Terraform and our [terraform provider](https://registry.terraform.io/providers/sleuth-io/sleuth/latest).
## Organization and Deployment Slugs
Note that the organization and deployment slugs are not the semantic name of your organization and deployment as shown in the organization settings, which can contain spaces and capitalized characters.
The slugs displayed are the URL of your organization and deployment, with spaces replaced by a hyphen (-) and non-alphabetical characters (e.g., ()@#$%^, etc.) ignored.For example, if you're viewing a deployment called plugin picker (dev) and your organization is called Amazing Software, the URL will display as . Thus, the organization slug is amazing-software, the deployment slug is plugin-picker-dev.
## Errors
* Codes in the `2xx` range indicate success
* Codes in the `4xx` range indicate incorrect or incomplete parameters
* Codes in the `5xx` range indicate an error with Sleuth servers
## REST API Details and Examples
# Deploy Registration
Use this endpoint with the POST method to register deploys.
## Path
{% hint style="info" %}
**ENDPOINT**
`ORG_SLUG`*/`DEPLOYMENT_SLUG`*/register\_deploy
{% endhint %}
The endpoint path takes **2 slugs** which direct the deploy to the correct code deployment:
* `ORG_SLUG`: found in the URL of your Sleuth org, immediately following `https://app.sleuth.io/`
* `DEPLOYMENT_SLUG`: found in the URL, following the prefix `https://app.sleuth.io/org_slug/deployments/`
## Authentication
{% hint style="success" %}
Each request must contain an `Authorization` header including an **Access Token**. We recommend using an **Access Token with** **limited scope** which can only be used for deploy registration.
{% endhint %}
You can manage your org's tokens it in the `Organization Settings` -> `Access Tokens` page.
## Parameters
{% tabs %}
{% tab title="Mandatory parameters" %}
Name
Type
Comments
sha*
string
The git SHA of the commit to be registered as a deploy.
The environment to register the deploy against. If not provided Sleuth will use the default environment of the Project.
date
string
ISO 8601 deployment date and time string
branch
string
If your code deployment's target environment is mapped to a branch prefix (rather than a specific branch), you must include the deploy’s full branch name as the parameter branch.
tags
string
A single string or a comma-delimited list of tags. Defaults to tags calculated by matching paths defined in your .sleuth/TAGS file.
ℹ️ Please note that tags must start with the # symbol, and can only contain:
lowercase letters,
numbers, and
symbols - and _ .
ignore_if_duplicate
string
If the value is provided and set to true Sleuth won't return a 400 error if we see a SHA that has already been registered.
email
string
Email address of author
links
string
A key/value pair consisting of the link name and the link itself in the following format:
mylink=http://my.link
If you need to send multiple then send a JSON body POST where the links are a dictionary of values.
Returned if any of the input parameters are invalid, e.g.: - sha isn't provided - branch doesn't match the configured branch - date format isn't valid - author is not a valid email - we're unable to validate if the sha exists in the remote system
The response text will indicate the nature of the error:
String of message problem
401
API key not valid or the deployment is not in the specified organization
String of message problem
{% endtab %}
{% endtabs %}
## Examples
{% hint style="warning" %}
Make sure you **replace the values** surrounded by**`<`** and **`>`**with your **own values.**
{% endhint %}
cURL
:information\_source: Please note that tags must start with the `#` symbol, must be separated with commas, and cannot contain the `.` symbol.
PowerShell
cURL using Custom Git
{% code overflow="wrap" lineNumbers="true" %}
```bash
curl -X POST -v \
'https://app.sleuth.io/api/1/deployments///register_deploy' \
-H 'Authorization: Bearer ' \
-H 'Content-Type: application/json' \
-d '{
"sha": "",
"environment": "",
"ignore_if_duplicate": "true",
"commits": [
{
"revision": "",
"message": "",
"author": {
"name": "Jane",
"email": "jane@email.com",
"username": "jane@email.com"
},
"date": "2022-08-01T00:10:10+00:00",
"files": [
"/some/path/to/a/file.txt"
],
"parents": [
""
],
"url": "http://www.commits/aaa"
}
],
"files": [
{
"path": "http://www.example.com/some/path.txt",
"additions": 3,
"deletions": 0,
"url": "http://www.example.com"
}
]
}'
```
# Deploy import
Use this endpoint to import deploys using a CSV file.
Delete all existing deploys in your Code Deployment and create a new history with only the deploys you specify via a CSV file. This might be useful if you wish to populate more than the default 28-days' worth of data when creating a new Code Deployment in Sleuth, or when you want to only import specific deploys, and don't want to register them individually.
## Path
{% hint style="info" %}
**ENDPOINT**
`ORG_SLUG`*/`DEPLOYMENT_SLUG`*/import\_deploys
{% endhint %}
The endpoint path takes **2 slugs** which direct the deploys to the correct code deployment:
* `ORG_SLUG`: found in the URL of your Sleuth org, immediately following `https://app.sleuth.io/`
* `DEPLOYMENT_SLUG`: found in the URL, following the prefix `https://app.sleuth.io/org_slug/deployments/`
## Parameters
{% tabs %}
{% tab title="Mandatory parameters" %}
Name
Type
Comments
api_key*
string
Can be found in the Organization Settings -> Details -> Api Key field in your Sleuth org.
csv_file*
file
The attached CSV file containing the list of deploys to import.
environment
string
The Sleuth environment slug. Defaults to the primary environment of the project.
{% endtab %}
{% tab title="Responses" %}
Code
Comments
Response Text
200
Deploys imported successfully.
Success
400
Returned if any of the input parameters are invalid, e.g.: - sha isn't provided - branch doesn't match the configured branch - date format isn't valid - author is not a valid email - we're unable to validate if the sha exists in the remote system
The response text will indicate the nature of the error:
String of message problem
401
API key not valid or the deployment is not in the specified organization
String of message problem
{% endtab %}
{% endtabs %}
{% hint style="danger" %}
Importing deploys into a pre-populated code deployment will **delete all existing deploys** in that code deployment and **create a new history** with only the deploys specified in the CSV file.
{% endhint %}
## CSV File Structure
The deploys should be imported using a **CSV file** that is uploaded as part of the request. The CSV file should contain the headers `sha` and `date`, the `sha` column should contain the **full SHA** of the commits at the point of release, and the `date` column should contain the **date and time** of the deploy in the [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format.
{% code title="Example CSV file" lineNumbers="true" %}
```csv
sha,date
3c02378,2002-04-03T08:34:02
6f0d7cf,2002-04-01T15:03:55
```
{% endcode %}
## Examples
{% hint style="warning" %}
Make sure you **replace the values** surrounded by**`<`** and **`>`**with your **own values**. Adjust the `csv_file` path accordingly or run the command from the directory that contains the `csv_file`.
{% endhint %}
{% hint style="warning" %}
When using the Organization API token found under **Organization Settings > Details**, the `Authorization` header needs to pass the API key via `apikey` e.g. `Authorization: apikey `
When using API tokens created under **Organization Settings > Access Tokens** the `Authorization` header needs to pass the API token via `Bearer` e.g. `Authorization: Bearer `
{% endhint %}
cURL with API key in Header
{% code overflow="wrap" lineNumbers="true" %}
```bash
curl \
'https://app.sleuth.io/api/1/deployments///import_deploys' \
-H 'Authorization: apikey ' \
-F 'csv_file=@.csv'
```
{% endcode %}
cURL with API key in Body
{% code overflow="wrap" lineNumbers="true" %}
```bash
curl \
'https://app.sleuth.io/api/1/deployments///import_deploys' \
-F 'api_key=' \
-F 'csv_file=@.csv'
```
{% endcode %}
# Manual Change
Use this endpoint with the POST method to register manual changes.
Manual changes are any changes not tracked by source code, feature flags, or any other type of change not supported by Sleuth. They are free-form entries that include a name and description. Although the description is optional, the form data in the manual change must contain a name as one of its parameters.
## Path
{% hint style="info" %}
**ENDPOINT**
`ORG_SLUG`*/`PROJECT_SLUG`*/register\_manual\_deploy
{% endhint %}
The endpoint path takes **2 slugs** which direct the manual changes to the correct code project:
* `ORG_SLUG`: found in the URL of your Sleuth org, immediately following `https://app.sleuth.io/`
* `PROJECT_SLUG`: found in the URL, following the prefix `https://app.sleuth.io/org_slug/`
### Parameters
{% tabs %}
{% tab title="Mandatory parameters" %}
Name
Type
Comments
api_key*
string
Can be found in the Organization Settings -> Details -> Api Key field in your Sleuth org.
The environment to register the change against. If not provided Sleuth will use the default environment of the Project.
tags
string
A comma-delimited list of tags.
author
string
Email address of change author.
email
string
Email address of the user associated with the project receiving the manual change.
{% endtab %}
{% tab title="Responses" %}
Code
Comments
Response Text
200
Manual change registered successfully.
Success
400
Returned if any of the input parameters are invalid, e.g.: - date format isn't valid - author is not a valid email
The response text will indicate the nature of the error:
String of message problem
401
Returned if the API key provided doesn't exist.
Unauthorized
404
Returned if the project does not exist.
Project not found
422
Returned if name is not provided
Name is required.
{% endtab %}
{% endtabs %}
### Examples
{% hint style="warning" %}
Make sure you **replace the values** surrounded by**`<`** and **`>`**with your **own values**.
{% endhint %}
cURL with API key in Header
cURL with API key in Body
{% code overflow="wrap" lineNumbers="true" %}
```bash
curl -X POST \
'https://app.sleuth.io/api/1/deployments///register_manual_deploy' \
-H 'Content-Type: application/json' \
-d '{
"name": "",
"description": "",
"api_key":
}'
```
{% endcode %}
PowerShell with API key in Header
{% code overflow="wrap" lineNumbers="true" %}
```powershell
Invoke-RestMethod -Method POST `
-Uri 'https://app.sleuth.io/api/1/deployments///register_manual_deploy' `
-Headers @{
'Authorization' = 'apikey '
'Content-Type' = 'application/json'
} `
-Body '{
"name": "",
"description": ""
}'
```
{% endcode %}
PowerShell with API key in Body
{% code overflow="wrap" lineNumbers="true" %}
```powershell
Invoke-RestMethod -Method POST `
-Uri 'https://app.sleuth.io/api/1/deployments///register_manual_deploy' `
-Headers @{
'Content-Type' = 'application/json'
} `
-Body '{
"api_key": "",
"name": "",
"description": ""
}'
```
{% endcode %}
# Custom Incident Impact Registration
Use this endpoint with the POST method to register Custom Incident Impact values.
Some teams track incidents outside of traditional Observability tools. [Custom impact sources](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/custom) allow you to submit these values to Sleuth and get your Failure Rate and MTTR values.
## Path
{% hint style="info" %}
**ENDPOINT**
`ORG_SLUG`/`PROJECT_SLUG`/`ENVIRONMENT_SLUG`/`IMPACT_SOURCE_SLUG`/register\_impact/`APIKEY`
{% endhint %}
The endpoint path takes **4 slugs** which direct the manual changes to the correct code project:
* `ORG_SLUG`: found in the URL of your Sleuth org, immediately following `https://app.sleuth.io/`
* `PROJECT_SLUG`: found in the URL, following the prefix `https://app.sleuth.io/org_slug/`
* `ENVIRONMENT_SLUG`: found at the end of the URL of your Sleuth org when navigating to the target project and selecting the target custom incident impact source: `env_slug=``ENVIRONMENT_SLUG`
* `IMPACT_SOURCE_SLUG`: found in the URL of your Sleuth org when navigating to the target project and selecting the target custom incident impact source, just before the `?env_slug`
The `API key` must also be added to the **end of the path** in this instance.
### Parameters
{% tabs %}
{% tab title="Mandatory parameters" %}
Name
Type
Comments
api_key*
string
Can be found in the Organization Settings -> Details -> Api Key field in your Sleuth org.
type*
string
Valid types are triggered, resolved, and reopened.
Use it with "type": "triggered" to register past incident event.
title
string
The human-readable title of the incident.
url
string
URL to the incident in your external system.
{% endtab %}
{% tab title="Responses" %}
Code
Comments
Response Text
200
Manual change registered successfully.
Success
400
Returned if any of the input parameters are invalid, e.g.: - date format isn't valid - value is not a valid float
The response text will indicate the nature of the error:
Bad Request - impact value must be a number
401
Returned if the API key provided doesn't exist.
Unauthorized
404
Returned if the IMPACT_ID does not exist.
MetricImpactSource Not Found
429
Returned if your requests are more frequent than one every 120 seconds. A Retry-After header is provided with the number of seconds you should wait until you try again.
You may only register a custom metric once every 120 seconds
{% endtab %}
{% endtabs %}
{% hint style="warning" %}
Make sure you **replace the values** surrounded by**`<`** and **`>`**with your **own values**.
{% endhint %}
cURL with API key in Path
PowerShell with API key in Path
{% code overflow="wrap" lineNumbers="true" %}
```powershell
Invoke-RestMethod -Method POST `
-Uri 'https://app.sleuth.io/api/1/deployments/////register_impact/' `
-Headers @{
'Content-Type' = 'application/json'
} `
-Body '{
"type": ""
}'
```
{% endcode %}
# Custom Metric Impact Registration
Use this endpoint with the POST method to register Custom Metric Impact values.
You can submit custom metric impact values to Sleuth. Sleuth will perform its anomaly detection on these values and they will inform the health of your deploys. Values can represent any metric that matters to you and must be represented via a float.
## Path
{% hint style="info" %}
**ENDPOINT**
`IMPACT_ID`*/*register\_impact
{% endhint %}
The endpoint path takes **1 ID** which uniquely identifies the Impact Source to register a value against:
* `IMPACT_ID`: must be an `integer`, you can find the full path (*including the ID*) in your Sleuth org by navigating to your [**Custom Metric Impact Source**](https://help.sleuth.io/integrations-1/impact-sources/metrics/custom), clicking the **gearwheel icon** in the top-right corner, and selecting "**Show register details**"
### Parameters
{% tabs %}
{% tab title="Mandatory parameters" %}
Name
Type
Comments
api_key*
string
Can be found in the Organization Settings -> Details -> Api Key field in your Sleuth org.
The date and time at which the metric value should be registered. If left blank, it defaults to the current time. Must be in ISO-8601 format.
{% hint style="danger" %}
Please note that **backfilling of metrics data is not currently supported**, and the supplied date therefore **cannot be older/before than the Impact Source creation date**.
{% endhint %}
{% endtab %}
{% tab title="Responses" %}
Code
Comments
Response Text
200
Manual change registered successfully.
Success
400
Returned if any of the input parameters are invalid, e.g.: - date format isn't valid - value is not a valid float
The response text will indicate the nature of the error:
Bad Request - impact value must be a number
401
Returned if the API key provided doesn't exist.
Unauthorized
404
Returned if the IMPACT_ID does not exist.
MetricImpactSource Not Found
429
Returned if your requests are more frequent than one every 120 seconds. A Retry-After header is provided with the number of seconds you should wait until you try again.
You may only register a custom metric once every 120 seconds
{% endtab %}
{% endtabs %}
### Examples
{% hint style="warning" %}
Make sure you **replace the values** surrounded by**`<`** and **`>`**with your **own values**.
{% endhint %}
cURL with API key in Header
cURL with API key in Body
{% code overflow="wrap" lineNumbers="true" %}
```bash
curl -X POST \
'https://app.sleuth.io/api/1/impact//register_impact' \
-H 'Content-Type: application/json' \
-d '{
"value": ,
"api_key": ""
}'
```
{% endcode %}
PowerShell with API key in Header
{% code overflow="wrap" lineNumbers="true" %}
```powershell
Invoke-RestMethod -Method POST `
-Uri 'https://app.sleuth.io/api/1/impact//register_impact' `
-Headers @{
'Authorization' = 'apikey '
'Content-Type' = 'application/json'
} `
-Body '{
"value":
}'
```
{% endcode %}
PowerShell with API key in Body
{% code overflow="wrap" lineNumbers="true" %}
```powershell
Invoke-RestMethod -Method POST `
-Uri 'https://app.sleuth.io/api/1/impact//register_impact' `
-Headers @{
'Content-Type' = 'application/json'
} `
-Body '{
"api_key": "",
"value":
}'
```
{% endcode %}
# Deprecation information
Parts of the Sleuth GQL API currently in a deprecated state
This page lists all the fields in our GQL API that have been marked as deprecated along with suggestions on possible alternatives.
We will try to keep all these fields available for at least 3 months after they've been tagged deprecated. You're highly encouraged to update any code still using them to avoid issues once the deprecation period runs out and the fields are removed.
## 2023-12-05
* Field `organization.apiKey` was deprecated. Use `organization.accessTokens` instead.
## 2023-11-21
* Field `manuallySetHealthThreshold` was deprecated. Use `earliestNonhealthyThreshold` instead, it has the same logic as the deprecated field.
## 2023-10-17
* Object type `DeployInProgress` was deprecated. We stopped creating `WorkInProgressItemType` items for future deploys. Instead we now create `PrInProgress` items for merged pull requests.
## 2023-08-02
* Field `builds` was deprecated. Use field `organization.buildDefinitions` instead.
## 2023-05-10
* Mutation `triggerIssueStatusesSync` was deprecated. Use `triggerRemoteObjectSync` instead.
* Field `canSeeOrgDashboard` on `UserPermissionsType` was deprecated. The permission in question is enabled for all users, so the field is no longer required.
## 2023-04-04
* Top level field `projects` was deprecated. Use field `organization.projects` instead.
## 2023-03-10
* Top-level field `context` was deprecated. See subfield deprecation messages for more information about alternatives.
* Field `project` was deprecated from object type `ContextType`. Use field `organization.projects` with `slug` argument query type instead.
* Field `environment` was deprecated from object type `ContextType`. Use field `organization.project.environment` query type instead.
## 2023-03-08
* Field `org` was deprecated from object type `ContextType`. Use field `organization` on root query type instead.
## 2023-03-07
* Field `visibleOrganizations` was deprecated from object type `ContextType`. Use field `organizations` on type `UserType` instead.
* Field `visibleProjects` was deprecated from object type `ContextType`. Use field `projects` on type `OrganizationType` instead.
## 2023-03-06
* Field `user` was deprecated from object type `ContextType`. Use field `user` on root query type instead.
* Field `messages` was deprecated from object type `ContextType`. Use field `flashMessages` on root query type instead.
* Field `baseurl` was deprecated from object type `ContextType`. Use field `baseurl` on root query type instead.
## 2023-03-02
* Field `flags` was deprecated from object type `ContextType`. Use field `flags` on type `OrganizationType` instead.
## 2022-06-24
* Enum `OrderField` 's option `health` is deprecated. Use option `health_time` instead.
* Field `orgChanges` has the following arguments deprecated:
* `healths`, use `health_times` instead
* `start_date`, use `start` instead
* `end_date`, use `end` instead
* For the following fields the arguments `start_date` and `end_date` are deprecated, use `start` and `end` instead:
* `ProjectType.metricMTTR`
* `ProjectType.metricFrequency`
* `ProjectType.metricLeadTime`
* `ProjectType.metricFailureRate`
* `ProjectType.metricRecap`
* `ProjectType.metricSlowestPrs`
* `ProjectType.metricBiggestDeploys`
* `ProjectType.metricTopUnhealthyDeploys`
* `ProjectType.insights`
## 2022-06-07
* Field `impactHistory` was deprecated from object type `Environment`. Use field `healthEvents` instead.
* Field `impactHistory` was deprecated from object type `ChangeModelType`. Use field `healthEvents` instead.
## 2022-05-23
* Field `health` was deprecated from object type `ChangeType`. Use field `currentHealth` on type `Environment` instead.
## 2022-04-26
* Mutation `create_nr_integration` was deprecated. Use mutation `create_newrelic_integration` instead.
* Mutation `create_dd_integration` was deprecated. Use mutation `create_datadog_integration` instead.
# GraphQL Queries
Example GraphQL queries with authentication.
### GraphiQL
To get started, login into Sleuth and open [GraphiQL](https://app.sleuth.io/graphql)
#### Logged in user ([example link](https://app.sleuth.io/graphql#query=%7B%0A%20%20user%20%7B%0A%20%20%20%20display%0A%20%20%7D%0A%7D))
{% code title="GraphQL Query" overflow="wrap" lineNumbers="true" %}
```graphql
{
user {
display
}
}
```
{% endcode %}
#### Team metrics recap ([example link](https://app.sleuth.io/graphql#query=query%20GetNumberOfTeamDeploys\(%24orgSlug%3A%20ID!%2C%20%24start%3A%20DateTime!%2C%20%24end%3A%20DateTime!%2C%20%24teamSlugs%3A%20%5BID%5D\)%20%7B%0A%20%20organization\(orgSlug%3A%20%24orgSlug\)%20%7B%0A%20%20%20%20metricsRecap\(start%3A%20%24start%2C%20end%3A%20%24end%2C%20filters%3A%20%7BteamSlugs%3A%20%24teamSlugs%7D\)%20%7B%0A%20%20%20%20%20%20numOfDeploys%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A\&operationName=GetNumberOfTeamDeploys\&variables=%7B%0A%20%20%22orgSlug%22%3A%20%22sleuth%22%2C%0A%20%20%22start%22%3A%20%222022-07-01T00%3A00%3A00Z%22%2C%0A%20%20%22end%22%3A%20%222022-07-31T00%3A00%3A00Z%22%2C%0A%20%20%22teamSlugs%22%3A%20%5B%0A%20%20%20%20%22frontend-2%22%0A%20%20%5D%0A%7D))
{% tabs %}
{% tab title="GraphQL Query" %}
{% code overflow="wrap" lineNumbers="true" %}
```graphql
query GetNumberOfTeamDeploys($orgSlug: ID!, $start: DateTime!, $end: DateTime!, $teamSlugs: [ID]) {
organization(orgSlug: $orgSlug) {
metricsRecap(start: $start, end: $end, filters: {teamSlugs: $teamSlugs}) {
numOfDeploys
}
}
}
```
{% endcode %}
{% endtab %}
{% tab title="Query Variables (JSON)" %}
{% code overflow="wrap" lineNumbers="true" %}
```json
{
"orgSlug": "sleuth",
"start": "2022-07-01T00:00:00Z",
"end": "2022-07-31T00:00:00Z",
"teamSlugs": [
"frontend-2"
]
}
```
{% endcode %}
{% endtab %}
{% endtabs %}
#### Project metrics recap ([example link](https://app.sleuth.io/graphql#query=query%20GetNumberOfProjectDeploys\(%24orgSlug%3A%20ID!%2C%20%24start%3A%20DateTime!%2C%20%24end%3A%20DateTime!%2C%20%24projectSlugs%3A%20%5BID%5D\)%20%7B%0A%20%20organization\(orgSlug%3A%20%24orgSlug\)%20%7B%0A%20%20%20%20metricsRecap\(start%3A%20%24start%2C%20end%3A%20%24end%2C%20filters%3A%20%7BprojectSlugs%3A%20%24projectSlugs%7D\)%20%7B%0A%20%20%20%20%20%20numOfDeploys%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A\&operationName=GetNumberOfProjectDeploys\&variables=%7B%0A%20%20%22orgSlug%22%3A%20%22sleuth%22%2C%0A%20%20%22start%22%3A%20%222022-07-01T00%3A00%3A00Z%22%2C%0A%20%20%22end%22%3A%20%222022-07-31T00%3A00%3A00Z%22%2C%0A%20%20%22projectSlugs%22%3A%20%5B%0A%20%20%20%20%22sleuth%22%0A%20%20%5D%0A%7D))
{% tabs %}
{% tab title="GraphQL Query" %}
{% code overflow="wrap" lineNumbers="true" %}
```graphql
query GetNumberOfProjectDeploys($orgSlug: ID!, $start: DateTime!, $end: DateTime!, $projectSlugs: [ID]) {
organization(orgSlug: $orgSlug) {
metricsRecap(start: $start, end: $end, filters: {projectSlugs: $projectSlugs}) {
numOfDeploys
}
}
}
```
{% endcode %}
{% endtab %}
{% tab title="Query Variables (JSON)" %}
{% code overflow="wrap" lineNumbers="true" %}
```json
{
"orgSlug": "sleuth",
"start": "2022-07-01T00:00:00Z",
"end": "2022-07-31T00:00:00Z",
"projectSlugs": [
"sleuth"
]
}
```
{% endcode %}
{% endtab %}
{% endtabs %}
### cURL and API key
{% hint style="info" %}
Using cURL you can call Sleuth GraphQL API from any environment.
{% endhint %}
#### Get API Key through GraphiQL ([example link](https://app.sleuth.io/graphql#query=%7B%0A%20%20context%20%7B%0A%20%20%20%20org%20%7B%0A%20%20%20%20%20%20apiKey%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D\&variables=))
{% code title="GraphQL Query" overflow="wrap" lineNumbers="true" %}
```graphql
{
context {
org {
apiKey
}
}
}
```
{% endcode %}
Now use API Key in `authorization` header (required headers are `content-type` and `authorization`)
{% code title="" overflow="wrap" lineNumbers="true" %}
```bash
curl 'https://app.sleuth.io/graphql' \
-H 'content-type: application/json' \
-H 'authorization: apikey ' \
-d '{"query":"query{organization(orgSlug:\"sleuth\"){metricsRecap(start:\"2022-07-01T00:00:00Z\",end:\"2022-07-31T00:00:00Z\",filters:{teamSlugs:\"frontend-2\"}){numOfDeploys}}}"}'
```
{% endcode %}
{% hint style="danger" %}
Authorization header starts with **apikey** and not **Bearer**!
{% endhint %}
# GraphQL Mutations
Example GraphQL mutations with authentication.
To get started, login into Sleuth and open [GraphiQL](https://app.sleuth.io/graphql).
#### Create new Code Deployment
{% tabs %}
{% tab title="Mutation Example" %}
{% code overflow="wrap" lineNumbers="true" %}
```graphql
mutation CreateCodeDeployment($input: CreateCodeChangeSourceMutationInput!) {
createCodeChangeSource(input: $input) {
changeSource {
name
slug
repository {
provider
owner
name
url
}
notifyInSlack
includeInDashboard
}
}
}
```
{% endcode %}
{% endtab %}
{% tab title="Variables (with webhooks)" %}
{% code overflow="wrap" lineNumbers="true" %}
```json
{
"input": {
"projectSlug": "",
"repository": {
"name": "",
"owner": "",
"url": "https://gitlab.com//",
"provider": "GITLAB"
},
"name": "Test Code Deployment",
"deployTrackingType": "MANUAL",
"environmentMappings": {
"environmentSlug": "production",
"branch": "main"
},
"environmentMappings": {
"environmentSlug": "staging",
"branch": "main"
}
}
}
```
{% endcode %}
{% hint style="info" %}
Supported values for the **provider** field are **AZURE**, **BITBUCKET**, **CUSTOM\_GIT**, **GITHUB**, **GITHUB\_ENTERPRISE**, or **GITLAB**.
{% endhint %}
{% endtab %}
{% tab title="Variables (with build tracking)" %}
{% code overflow="wrap" lineNumbers="true" %}
```json
{
"input": {
"projectSlug": "",
"repository": {
"name": "",
"owner": "",
"url": "https://gitlab.com//",
"provider": "GITLAB"
},
"name": "Test Code Deployment",
"deployTrackingType": "BUILD",
"environmentMappings": {
"environmentSlug": "production",
"branch": "main"
},
"environmentMappings": {
"environmentSlug": "staging",
"branch": "main"
},
"
}
}
```
{% endcode %}
{% hint style="info" %}
Supported values for the **provider** field are **AZURE**, **BITBUCKET**, **CUSTOM\_GIT**, **GITHUB**, **GITHUB\_ENTERPRISE**, or **GITLAB**.
{% endhint %}
{% endtab %}
{% endtabs %}
# Query batching
Sleuth GQL API supports batching, which allows you to execute multiple queries in a single request. To use it, you need to make two small changes to your requests: use a different API URL and wrap the payload in a list.
The two examples below demonstrate the same query being requested individually or in a batch.
### Individual query
#### URL
`https://app.sleuth.io/graphql`
#### Request body
```json
{
"query": "{ organization(orgSlug: \"sleuth\") { name } }",
"variables": null
}
```
### Batched query
#### URL
`https://app.sleuth.io/graphql-batch`
#### Request body
```json
[
{
"query": "{ organization(orgSlug: \"sleuth\") { name } }",
"variables": null
}
]
```
{% hint style="info" %}
When batching queries, the server response will match the request and return a list of objects instead of a single object.
{% endhint %}
# Integrations
Connect Sleuth DORA to the rest of your toolchain — source control, CI/CD, issue trackers, observability, chat, and feature flags.
Sleuth DORA reaches its full value when it is wired into the tools your team already uses. This section covers every supported integration, grouped by what the integration does for Sleuth.
* [About Integrations...](/sleuth-dora/integrations-1/about-integrations) — start here for the conceptual overview.
* [Code integrations (read-only)](/sleuth-dora/integrations-1/code-deployment) — source control connectors that feed deploys and commits into Sleuth.
* [Code integrations (write)](/sleuth-dora/integrations-1/code-integrations-write) — connectors that let Sleuth write back to source control.
* [Feature flag integrations](/sleuth-dora/integrations-1/feature-flags) — LaunchDarkly and others.
* [Impact integrations](/sleuth-dora/integrations-1/impact-sources) — error trackers, metric trackers, incident trackers, and CI/CD build trackers.
* [Sleuth DORA App for Slack](/sleuth-dora/integrations-1/slack)
* [Microsoft Teams integration](/sleuth-dora/integrations-1/microsoft-teams-integration)
* [CI/CD integrations](/sleuth-dora/integrations-1/builds) — register deploys directly from your build pipelines.
* [Issue tracker integrations](/sleuth-dora/integrations-1/issue-trackers) — Jira, Linear, Shortcut.
* [Fixing broken integrations](/sleuth-dora/integrations-1/fixing-broken-integrations)
# About Integrations...
Sleuth works with the tools you already have in place. Every integration has been designed to setup in Sleuth in under 5 minutes. Our **webhook** approach to [deploy registration](/sleuth-dora/modeling-your-deployments/code-deployments/how-to-register-a-deploy) means we support **every** possible way you deploy your code.
Integrations are what enable Sleuth to communicate with the tools in your DevOps arsenal. Sleuth is able to collect the information it needs to provide a comprehensive view of your deployments, ensuring that you always know what is happening with your code—commits, PRs, deploys, issues, errors, metrics, feature flags, authors, and so much more—and communicating successful—or failed—deployments through any [chat ops integrations ](#chat-ops)you have set up.
Sleuth communicates with your tools via the APIs they provide. You will need access to your tools' API keys so that Sleuth can connect to your integrations and obtain the information it needs to build a comprehensive overview of your applications' health.
### Sleuth Integrations
| Integration | Type | Additional info... |
| ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ | ------------------------------------------------ |
| [AppDynamics](/sleuth-dora/integrations-1/impact-sources/metrics/appdynamics) | Impact metric tracking | Metrics tracker |
| [AWS CloudWatch](/sleuth-dora/integrations-1/impact-sources/metrics/aws-cloudwatch) | Impact metric tracking | Metrics tracker |
| [Azure DevOps](/sleuth-dora/integrations-1/code-deployment/azure-devops) | Code deploys and issue tracker | Code deployments and issue tracking |
| [Azure Pipelines](/sleuth-dora/integrations-1/code-deployment/azure-devops) | Deployment CI/CD | Track and trigger your deploys and builds |
| [Bitbucket](/sleuth-dora/integrations-1/code-deployment/bitbucket) | Code deploys and issue tracker | Code deployments and issue tracking |
| [Bitbucket Pipelines](/sleuth-dora/integrations-1/builds/bitbucket-pipelines) | Deployment CI/CD | Track and trigger your deploys and builds |
| [Blameless](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/blameless) | Incident impact | Track failure rate and MTTR with incidents |
| [Buildkite](/sleuth-dora/integrations-1/builds/buildkite) | Deployment CI/CD | Track and trigger your deploys and builds |
| [Bugsnag](/sleuth-dora/integrations-1/impact-sources/errors/bugsnag) | Impact error tracking | Error tracker |
| [Custom incident](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/custom) | Incident impact | Track failure rate and MTTR with incidents |
| [Custom metric impact](/sleuth-dora/integrations-1/impact-sources/metrics/custom) | Impact metric tracking | Submit custom metrics |
| [CircleCI](/sleuth-dora/integrations-1/builds/circleci) | Deployment CI/CD | Track and trigger your deploys and builds |
| [Custom Git](https://github.com/sleuth-io/sleuth-gitbook-docs/tree/8c6f655818b14806b9a76252e4224c2ef29d58f6/integrations-1/code-deployment/custom.md) | Code deploys | Code deployments |
| [Datadog (metrics)](/sleuth-dora/integrations-1/impact-sources/metrics/datadog) | Impact metric tracking | Metrics tracker |
| [Datadog Monitors](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/datadog) | Incident impact | Track triggered monitors as incidents |
| [GitHub](/sleuth-dora/integrations-1/code-deployment/github) | Code deploys and issue tracker | Code deployments and issue tracking |
| [GitLab](/sleuth-dora/integrations-1/code-deployment/gitlab) | Code deploys and issue tracker | Code deployments and issue tracking |
| [Gitlab CI/CD Pipelines](/sleuth-dora/integrations-1/code-deployment/gitlab) | Deployment CI/CD | Track and trigger your deploys and builds |
| [FireHydrant](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/firehydrant) | Incident impact | Track failure rate and MTTR with incidents |
| [Honeybadger](/sleuth-dora/integrations-1/impact-sources/errors/honeybadger) | Impact error tracking | Error tracker |
| [Jenkins](/sleuth-dora/integrations-1/builds/jenkins) | Deployment CI/CD | Track and trigger your deploys and builds |
| [Jira (Cloud/Data Center)](/sleuth-dora/integrations-1/issue-trackers/jira) | Deploy issue tracker | Automatically associate issues with deploys |
| [Jira (Cloud/Data Center)](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/jira-cloud-data-center) | Incident impact | Track failure rate and MTTR with incidents |
| [LaunchDarkly](/sleuth-dora/integrations-1/feature-flags/launchdarkly) | Feature flag changes | Feature flags as a change source |
| [Linear](/sleuth-dora/integrations-1/issue-trackers/linear) | Deploy issue tracker | Automatically associate issues with deploys |
| [Microsoft Teams](/sleuth-dora/integrations-1/microsoft-teams-integration) | Notifications | Keep teams informed with real-time notifications |
| [NewRelic](/sleuth-dora/integrations-1/impact-sources/metrics/newrelic) | Impact metric tracking | Metrics tracker |
| [Opsgenie](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/opsgenie) | Incident impact | Track failure rate and MTTR with incidents |
| [PagerDuty](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/pagerduty) | Incident impact | Track failure rate and MTTR with incidents |
| [Rollbar](/sleuth-dora/integrations-1/impact-sources/errors/rollbar) | Impact error tracking | Error tracker |
| [Sentry](/sleuth-dora/integrations-1/impact-sources/errors/sentry) | Impact error tracking | Error tracker |
| [ServiceNow](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/servicenow) | Incident impact | Track failure rate and MTTR with incident |
| [Shortcut](/sleuth-dora/integrations-1/issue-trackers/shortcut) | Deploy issue tracker | Automatically associate issues with deploys |
| [SignalFx](/sleuth-dora/integrations-1/impact-sources/metrics/signalfx) | Impact metric tracking | Metrics tracker |
| [Statuspage](/sleuth-dora/integrations-1/impact-sources/incident-tracker-integrations/statuspage) | Incident impact | Track failure rate and MTTR with incidents |
| [Slack](/sleuth-dora/integrations-1/slack) | Notifications and control | Notify your entire team or specific individuals |
| [Terraform Cloud](https://www.terraform.io/cloud) | Code deploys | Automatically register a deploy via a Webhook |
# Code integrations (read-only)
**Code** integrations give Sleuth read-only access to your code repositories, allowing Sleuth to analyze your pull requests, commits, files and authors to provide a meaningful and clear view of the changes you make in code when you deploy.
| Integration | Type | Additional info... |
| ----------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | ----------------------------------- |
| [Azure DevOps](/sleuth-dora/integrations-1/code-deployment/azure-devops) | Code Deployment | Can be used a change source. |
| [Bitbucket](/sleuth-dora/integrations-1/code-deployment/bitbucket) | Code Deployment | Can be used a change source. |
| [GitHub](/sleuth-dora/integrations-1/code-deployment/github) | Code Deployment | Can be used as a change source. |
| [GitLab](/sleuth-dora/integrations-1/code-deployment/gitlab) | Code Deployment | Can be used as a change source. |
| [Custom Git](https://github.com/sleuth-io/sleuth-gitbook-docs/tree/8c6f655818b14806b9a76252e4224c2ef29d58f6/integrations-1/code-deployment/custom.md) | Code Deployment | Can be used as a change source. |
| [Terraform Cloud](https://www.terraform.io/cloud) | Code Deployment | Auto register deploys via a webhook |
### Version Control Workflow
Sleuth encourages and works best with a [trunk-based developmen](https://trunkbaseddevelopment.com/)t workflow. As observed by [Accelerate](https://en.wikipedia.org/wiki/Accelerate_\(book\)):
> Developing off trunk rather than on long-lived branches is correlated with higher delivery performance
#### Scaled trunk-based development
If your teams create short-lived branches that are merged into a core branch (trunk, main, master) at least daily then you're working consistent with commonly accepted continuous integration best practices. Sleuth is an excellent tool for your team.
#### Strict trunk-based development caveats
If your team is small or nimble enough to commit directly to *one* branch, more granular data may not be available or relevant (e.g., review time and review lag time in change lead time) depending on your process and provider. However, Sleuth will still surface the important metrics to help your team succeed in its software delivery optimization goals.
# Azure DevOps
## About the integration
Integrating Azure DevOps with Sleuth is simple. If you're connecting to a personal Azure DevOps repo, you just need your credentials. If you're part of an organization and aren't the owner, you will need permission to allow Sleuth to connect to the repo—after you connect you'll be able to select individual private or public repositories.
The integration is best tested against Azure DevOps Services, however, it should work for Azure DevOps Server as well. Which is which?
* Azure DevOps Services (formerly known as Visual Studio Team Services, or VSTS\_)\_ is a cloud-based solution
* Azure DevOps Server (formerly known as Team Foundation Server, or TFS) is an on-premises offering
## Setting up the integration
To set up the Azure DevOps integration:
1. Click **Add** in the top navigation bar and select **Integration** from the list.
2. Select **Code** from the drop-down located in the top right.
3. In the **Azure DevOps** tile, click **Enable**.
4. Enter the details of the account with which you wish to authenticate your Azure DevOps integration. You will have the chance to select specific repo(s) for your Sleuth project(s) later.\
5. In a separate browser tab or window, visit your Azure DevOps account, and under **User settings**, click on **Personal Access Tokens** and generate a token with the required scopes. The `Work Items` and `Build` scopes are only necessary if you want to configure issue and build integration. Once generated, paste the token into the Sleuth form and click **Save**.
6. On successful integration, you'll see **Azure DevOps** marked as **Enabled** and there will be a list of connections (*you can have more than one*) displayed on the tile when expanded:\\
### Custom HTTP headers
If you using Azure DevOps on-premise behind Cloudflare access or similar, Sleuth might need to include some HTTP headers in order to reach your instance. In order to set Sleuth to send any custom HTTP headers when making requests:
1. In the Azure DevOps dialog, click on the **Advanced setting**.
2. Enter a comma-separated list of custom headers you want Sleuth to include.
## Configuring the integration
After the initial setup is complete, the Azure DevOps integration can be used to set up:
* a **code deployment**: select a Sleuth project from the list and then follow the instructions for [creating a code deployment](https://help.sleuth.io/modeling-your-deployments/code-deployments/creating-a-deployment)
* a **build server**: select a Sleuth project from the list to set AzureDevops as the `Build integration provider` for the selected project
* an **issue tracker**: select a Sleuth project from the list to set AzureDevops as the `Issue integration provider` for the selected project
## Removing the integration
#### If you wish to remove the **Azure DevOps** integration for the organization:
1. Click the **Add** button in the top nav and select **Integrations** from the list.
2. Expand the **Azure DevOps** integration card, and click **Remove** next to the connection you wish to remove. If you want to remove all of your Azure DevOps connections, you'll need to repeat this step for each connection. A confirmation screen will appear warning you of the consequences of this action and prompting you to confirm your decision -> click **Confirm**.
After all connections are removed, the Azure DevOps integration is disconnected and no longer available for any projects within that organization.
# Bitbucket
## About the integration
Integrating Bitbucket with Sleuth is simple. If you're connecting to a personal Bitbucket repo, you just need your credentials. If you're part of a Bitbucket organization and aren't the owner, you will need permission to allow Sleuth to connect to the repo (after you connect you'll be able to select individual private or public repositories).
If you are using Bitbucket issues to track issues, Sleuth will automatically discover your referenced issues once the integration is configured. You can still use other [issue tracker integrations](/sleuth-dora/integrations-1/issue-trackers) if you don't use Bitbucket's issues.
{% hint style="info" %}
Check out the Sleuth for Bitbucket integration [in the Atlassian Marketplace](https://marketplace.atlassian.com/apps/1223448/sleuth-for-bitbucket?hosting=cloud\&tab=overview).
{% endhint %}
## Setting up the integration
To set up the Sleuth Bitbucket integration:
1. Click **Add** in the top navigation bar and select **Integration** from the list.
2. Select **Code** from the drop-down located in the top right.
3. In the **Bitbucket** tile, click **Enable**.
4. Grant Sleuth access to your Bitbucket account by clicking **Grant access** in the confirmation dialog. You will have the chance to select specific repo(s) for your Sleuth project(s) later.\
5. On successful integration, you'll see **Bitbucket** marked as **Enabled** and the connection listed in the format **Connected as ``**.\
## Configuring the integration
After the initial setup is complete, the Bitbucket integration can be used to set up:
* a **code deployment**: select a Sleuth project from the list and then follow the instructions for [creating a code deployment](https://help.sleuth.io/modeling-your-deployments/code-deployments/creating-a-deployment)
* an **issue tracker**: select a Sleuth project from the list to set Bitbucket as the `Issue integration provider` for the selected project
## Removing the integration
#### If you wish to dissolve the Bitbucket integration for the organization:
1. Click the **Add** button in the top nav and select **Integrations** from the list.
2. Expand the **Bitbucket** integration card, and click **Remove** next to the connection you wish to remove. A confirmation screen will appear warning you of the consequences of this action and prompting you to confirm your decision -> click **Confirm**.
After all connections are removed, the Bitbucket integration is then disconnected and no longer available for any projects within that organization.
---
[Next Page](/llms-full.txt/1)