Skip to content

buildBenchmarkPrompt

reference
1 min readUpdated

Kind: Function

Source: atloria-monorepo/libs/agent-core/src/autonomous/adaptive-prompt.ts

Build a benchmark-optimized system prompt.

Unlike buildAdaptivePrompt (which has a 5-phase workflow including "run baseline tests first"), this prompt is action-first:

  1. Read the issue → identify the source file and function
  2. Call edit_file immediately with the fix
  3. Run the targeted test to verify

The "establish baseline first" step is intentionally omitted because:

  • Baselines consume 50-80% of the turn budget on analysis
  • The model then produces text describing its plan instead of calling edit_file
  • Azure Responses API expires previous_response_id on completion → no second turn
  • SWE-bench eval applies the diff, not the agent's text response

Used by BenchmarkRunner instead of buildAdaptivePrompt.

Signature

ts
function buildBenchmarkPrompt(options: {
  workspace: string;
  toolGuide: string;
  repoMap?: string;
  domainRules?: string[];
  /** Specific test names that must pass (from SWE-bench FAIL_TO_PASS metadata) */
  failToPassTests?: string[];
}): string

Parameters

NameType
options`{
workspace: string;
toolGuide: string;
repoMap?: string;
domainRules?: string[];
/** Specific test names that must pass (from SWE-bench FAIL_TO_PASS metadata) */
failToPassTests?: string[];
}`

Returns: string

Was this page helpful?

Download as PDF