# tokenize

**Kind:** Function

**Source:** [`atloria-monorepo/libs/agent-core/src/semantic/bm25.ts`](https://github.com/sherkety/atloria/blob/main/atloria-monorepo/libs/agent-core/src/semantic/bm25.ts#L19)

BM25 (Okapi BM25) — Inline Implementation

Standard BM25 ranking with k1=1.2, b=0.75.
Zero external dependencies — pure TypeScript.

## Signature

```ts
function tokenize(text: string): string[]
```

## Parameters

| Name | Type |
|---|---|
| `text` | `string` |

**Returns:** `string[]`