Skip to content

Chunker

reference
1 min readUpdated

Kind: Class

Source: atloria-monorepo/libs/agent-core/src/semantic/chunker.ts

Methods

MethodSignatureReturns
chunkFilechunkFile(filePath: string, rootDir: string)CodeChunk[]
chunkDirectory`chunkDirectory(rootDir: string, config: {
  maxFiles?: number;
  includeExtensions?: string[];
  excludeDirs?: string[];
  targetChunkTokens?: number;
})` | `CodeChunk[]` |

| extractImports | extractImports(content: string) | string |

Where it refuses work

  • Chunker stops the work with an early return when results.length >= maxFiles, in 2 places.
  • Chunker stops the work with an early return when stat.size > MAX_FILE_SIZE.
  • Chunker stops the work with an early return when content.length > MAX_FILE_SIZE.
  • Chunker stops the work with an early return when boundaries.length === 0.
  • Chunker stops the work with an early return when foundOpen && depth === 0.
  • Chunker stops the work with an early return when lines[i].trimEnd().endsWith(';').

When something fails

  • Chunker handles failure in 2 places: it turns it into a return value in all 2.

Was this page helpful?

Download as PDF