KeywordExtractorService

Kind: Service

Source: atloria-monorepo/apps/api/src/documentation/services/keyword-extractor.service.ts

Keyword Extraction Service

Extracts searchable keywords from documentation content. Leverages Phase 2.1 & 3 context for rich keyword extraction:

• State machines → states, transitions, action verbs
• UI interactions → button labels, form fields
• Code → method names, class names
• Content → natural language terms

Used by DocumentIndexingService to create search indexes.

KeywordExtractorService extracts searchable keywords from documentation content to support fast, relevant search. It analyzes multiple signal sources—state machines, UI interactions, code identifiers, and natural language terms—to produce a rich keyword set. The resulting keywords are consumed by DocumentIndexingService when building and updating search indexes.

Methods

Where it refuses work

• KeywordExtractorService stops the work with an early return when !text.

Diagram

mermaid
sequenceDiagram
  autonumber
  participant DI as DocumentIndexingService
  participant KE as KeywordExtractorService
  participant IDX as Search Index

  DI->>KE: extractKeywords(docContent, context)
  Note over KE: Parse and normalize tokens<br/>from Phase 2.1 & 3 context
  KE->>KE: State machines → states/transitions/action verbs
  KE->>KE: UI interactions → button labels/fields
  KE->>KE: Code → class/method names
  KE->>KE: Content → natural language terms
  KE-->>DI: keywords[]
  DI->>IDX: upsertDocument({ id, keywords, ... })

Usage

ts
// Example (NestJS): using KeywordExtractorService within an indexing workflow

import { Injectable } from '@nestjs/common';
import { KeywordExtractorService } from './keyword-extractor.service';

type DocContext = {
  stateMachines?: Array<{ states: string[]; transitions: string[] }>;
  ui?: { buttons?: string[]; fields?: string[] };
  code?: { classes?: string[]; methods?: string[] };
};

@Injectable()
export class ExampleIndexingWorkflow {
  constructor(private readonly keywordExtractor: KeywordExtractorService) {}

  async buildIndexPayload(docId: string, markdown: string, context: DocContext) {
    const keywords = await this.keywordExtractor.extractKeywords(markdown, context);

    return {
      id: docId,
      keywords, // store on the document index record
      content: markdown,
    };
  }
}

// Example (non-Nest): direct instantiation (if it has no DI-only dependencies)
async function extractForSearch(content: string, context: DocContext) {
  const svc = new KeywordExtractorService();
  return svc.extractKeywords(content, context);
}

AI Coding Instructions

• Preserve deterministic output: normalize case/whitespace, deduplicate, and keep keyword ordering stable (or explicitly sort) to avoid index churn.
• When adding new keyword sources, route them through a single normalization/tokenization pipeline so scoring and filtering remain consistent.
• Be careful with noise: avoid indexing stop-words, very short tokens, and overly-generic UI terms unless they improve search recall.
• Integration point: DocumentIndexingService depends on this output—keep the return type and semantics stable; any changes should update indexing and tests together.
• Prefer additive changes (new extractors/weights) over breaking changes; validate with real docs to ensure keywords remain relevant and not overly broad.

Referenced By

• DocumentationModule (MODULE_PROVIDES)
• DocumentIndexingService (DEPENDS_ON)