Running pentests

Create a pentest, define its scope, authorize it with Rules of Engagement, assign agents, launch it, and follow it through to a sealed report.

The shape of a pentest

A pentest is one engagement against a defined scope. When you create it you choose two things that never change afterwards:

  • Type — web (a web application), api (an HTTP API) or server (a host or network service).
  • Mode — guided (you pick the agents for each phase and advance one phase at a time) or automatic (default agents run every phase end to end with no intervention).

Agents work through three ordered phases: 1 Reconnaissance, 2 Enumeration, 3 Analysis. For background on the model, read Pentests, assets & phases.

Create a pentest

  1. From the dashboard at aleex-rank.ai, open Pentests and click New pentest.

  2. Give it a name and optional description, then choose the type and mode.

  3. If you belong to one or more teams, pick the team that owns this pentest. This is required when you have teams, because it determines whose budget and members apply.

  4. Add at least one asset to define the scope. Each asset has a type — url, domain, ip or api — and exactly one should be marked primary, which agents attack first.

  5. Click Create. The pentest is saved in draft, then becomes configured once it has a type, methodology and at least one asset.

A scope with a primary domain plus a few supporting assets is a common setup:

{
  "name": "Acme web assessment",
  "type": "web",
  "mode": "guided",
  "team_id": 4,
  "assets": [
    {"asset_type": "domain", "asset_value": "acme.example.com", "is_primary": true},
    {"asset_type": "ip", "asset_value": "192.0.2.10"},
    {"asset_type": "url", "asset_value": "https://acme.example.com/api"}
  ]
}

You can keep editing assets from the pentest detail screen until you launch.

Authorize the run (RoE)

Assets are inventory. Before a phase can start, the pentest also needs an active Rules of Engagement that covers every asset — CIDRs and/or domains, an authorization_ref, and at least one escalation contact. Without that, the orchestrator is fail-closed (no_active_roe).

How you declare and activate RoE, decide in-run approvals, and use the kill switch is documented at Engagement & control. From code, use the Engagement API or the Python SDK. The CLI does not create RoE.

Assign agents

How agents get onto the phases depends on the mode you chose. Either way the platform requires 3 to 4 agents per phase.

Guided mode

You select agents one phase at a time, and you must fill phase N before moving to phase N+1.

  1. Open the pentest and go to the Phase 1 — Reconnaissance configuration screen.

  2. The agent picker lists every agent available for that phase. Default agents are marked with a star; you can also pick agents you own or that teammates share.

  3. Select 3 or 4 agents and drag them into the order you want them to run.

  4. Click Save & run to assign them and launch the phase. When it finishes you repeat for Phase 2, then Phase 3.

Automatic mode

You don’t assign anything by hand. Opening the run screen loads the default agents for every phase, and the platform auto-assigns them in one step — you only confirm and launch. If you want to customise an automatic run, you can still override the defaults from the agent picker before launching; the same 3-to-4-per-phase rule applies.

Guided mode gives you full control and a review step between phases. Automatic mode is the fast path: assign once, run everything, get findings.

Launch and monitor

When you launch, the browser connects directly to the streaming backend and the run begins. Progress streams back live as a timeline of agent activity:

  • Queued / Ready — if there’s a queue, you see your position, then “starting…” when a slot opens.
  • Phase start — the phase begins and its agents spin up, often in parallel.
  • Agent activity — per agent: its plan, each iteration, thinking, tool calls and results, and a short interpretation step. Subagents appear nested under their parent.
  • Consolidation — at the end of a phase, results are consolidated into a written summary that streams in.
  • Phase complete — in guided mode this is your cue to review and decide whether to continue; in automatic mode the next phase starts on its own.

You can pause a running pentest or cancel the current stream from the same screen. Cancel asks the streaming backend to stop this run; it is not the kill switch. Kill is a terminal emergency halt on the control plane: the orchestrator must stop, evidence is preserved, and the pentest cannot return to an active state. Use cancel to detach a run you started; use kill when the RoE is breached or you need a hard stop. See Engagement & control.

If you close the tab mid-run, reopening the pentest reconnects to the live stream where it left off. For the full event vocabulary, including control_stop and window_warning, see Streaming.

A pentest moves through running → phase_completed between phases, then processing and finally completed. Idle pentests left in configured or running for too long are archived automatically and you’re notified by email.

Process vulnerabilities and finish

Before a guided pentest can finish, its findings must be processed — the platform extracts vulnerabilities from the agents’ work and stores them for triage. You’ll see a Process vulnerabilities action once the phases you want are done.

  1. After the last phase you care about completes (you don’t have to run all three), click Process vulnerabilities.

  2. Processing runs as a background job. The pentest shows a processing state while it works; the count and severities appear once it finishes, not instantly. You can leave the screen and come back — the job keeps running.

  3. When processing completes, review what was found, then click Finish pentest. The pentest must be in phase_completed (or processing) — you can’t finish while a phase is still running. A guided pentest also needs at least one processed vulnerability before it will finish.

In automatic mode none of this is manual: the streaming backend processes vulnerabilities and marks the pentest completed for you. Either way, the findings land in the triage queue — see Vulnerability triage.

Generate the report

Once a pentest is finished you can generate a sealed report. Rank issues one file per format — pdf, html, markdown, docx or json — hashes the payload, stores the issuance, and (for PDF) emails it only if sealing succeeded. Audience (executive, technical, attestation) and formats can be chosen for that generation without changing the stored profile. SARIF is a finding export, not a report format.

Custom branding is a visual overlay (logo, colours, footer) on the same content. The sealed history and download URLs live with the pentest. See Reports.

Where to go next