SWEBenchAdapter

reference

1 min readUpdated 2026-07-29

Kind: Class

Source: atloria-monorepo/libs/agent-core/src/benchmark/swebench-adapter.ts

Implements: BenchmarkAdapter

Methods

Method	Signature	Returns
`loadTasks`	`loadTasks(config: BenchmarkRunConfig)`	`Promise<BenchmarkTask[]>`
`setupTask`	`setupTask(task: BenchmarkTask, workDir: string)`	`Promise<void>`
`buildPrompt`	`buildPrompt(task: BenchmarkTask)`	`string`
`evaluate`	`evaluate(task: BenchmarkTask, workDir: string)`	`Promise<{ passed: boolean; details: string }>`
`cleanup`	`cleanup(_task: BenchmarkTask, workDir: string)`	`Promise<void>`

Where it refuses work

SWEBenchAdapter stops the work with Error when !fs.existsSync(tasksFile).

When something fails

SWEBenchAdapter handles failure in 13 places: it discards it silently in 5, logs it and continues in 4, and turns it into a return value in 4.

Was this page helpful?

Download as PDF

ADocs generated by Atloria — from the code, always in sync →