DocsSyncQueue

Kind: Service

Source: atloria-monorepo/apps/api/src/git-sync/docs-sync-queue.service.ts

The 'docs-sync' queue registration (B1 outbound mirror). Present in EVERY api pod for ENQUEUE (settings save / sync-now / the cron sweep just add a job); processing lives in GitSyncMirrorWorker, gated by DOCS_REPO_WORKER — the mirror reads the substrate repo (read-only git show/ls-tree at a pinned sha) which only the worker deployment mounts. Mirrors the docs-repo / docs-cr enqueue/worker split exactly.

Jobs are DEDUPED per project via a deterministic jobId (mirror-<projectId> — BullMQ rejects ':' in custom ids): a burst of substrate commits collapses to one pending mirror. removeOnComplete AND removeOnFail are BOTH true: a retained job with the same jobId makes BullMQ silently ignore a later add(), wedging the next mirror. The ProjectDocsSync cursors (lastSourceSha vs headCommit) are the retry source of truth — the cron sweep re-enqueues.

DocsSyncQueue registers the outbound docs-sync BullMQ queue in every API pod and provides the enqueue boundary for documentation mirror requests. Settings saves, manual “sync now” actions, and cron sweeps add deduplicated per-project jobs; actual Git processing is performed separately by GitSyncMirrorWorker when DOCS_REPO_WORKER is enabled.

Methods

Dependencies

• ConfigService

Where it refuses work

• DocsSyncQueue stops the work with an early return when !this.queue.

When something fails

• DocsSyncQueue handles failure in 2 places: it logs it and continues in 1, and turns it into a return value in 1.

Diagram

mermaid
sequenceDiagram
  participant API as API Pod
  participant Queue as DocsSyncQueue
  participant BullMQ as BullMQ / Redis
  participant Worker as GitSyncMirrorWorker
  participant Repo as Substrate Repository

  API->>Queue: enqueueMirror(projectId)
  Queue->>Queue: Build jobId: mirror-<projectId>
  Queue->>BullMQ: add("docs-sync", payload, { jobId })

  alt Job already pending for project
    BullMQ-->>Queue: Deduplicated / existing job
  else New job
    BullMQ-->>Queue: Job queued
  end

  Worker->>BullMQ: Consume docs-sync job
  Worker->>Repo: git show / git ls-tree at pinned SHA
  Worker->>Worker: Mirror docs to docs repository
  Worker->>BullMQ: Complete or fail job
  Note over BullMQ: removeOnComplete and removeOnFail are true

Usage

ts
import { Injectable } from '@nestjs/common';
import { DocsSyncQueue } from './git-sync/docs-sync-queue.service';

@Injectable()
export class ProjectSettingsService {
  constructor(private readonly docsSyncQueue: DocsSyncQueue) {}

  async saveDocsSettings(projectId: string): Promise<void> {
    // Persist project settings first.
    // await this.projectRepository.update(projectId, settings);

    const queued = await this.docsSyncQueue.enqueueMirror(projectId);

    if (!queued) {
      // A mirror job may already be pending for this project.
      // The cron sweep can retry later if cursors remain out of sync.
      return;
    }
  }
}

AI Coding Instructions

• Use enqueueMirror(projectId) as the only API-side path for requesting outbound documentation mirrors; do not perform Git mirror work in API pods.
• Preserve deterministic job IDs in the mirror-<projectId> format. BullMQ custom job IDs cannot contain :.
• Keep removeOnComplete: true and removeOnFail: true; retaining completed or failed jobs can cause later add() calls with the same ID to be silently ignored.
• Treat ProjectDocsSync.lastSourceSha versus headCommit as the retry source of truth. The cron sweep should re-enqueue projects whose cursors are not aligned.
• Keep worker-only repository access inside GitSyncMirrorWorker, gated by DOCS_REPO_WORKER; API pods should only register and enqueue queue jobs.

Relationships

• DEPENDS_ON → configservice

Referenced By

• GitSyncModule (MODULE_PROVIDES)
• GitSyncService (DEPENDS_ON)