tokenize

Kind: Function

Source: atloria-monorepo/apps/api/src/technical-docs/search-index.ts

Lowercased, deduped, stopword-filtered significant tokens (capped).

Signature

ts
function tokenize(text: string): string[]

Parameters

Returns: string[]