Kind: Function
Source: atloria-monorepo/libs/parser-core/src/utils/web-crawler.ts
Crawl multiple pages with concurrency control
Signature
tsasync function crawlPages(urls: string[], concurrency): Promise<CrawledPage[]>
Parameters
| Name | Type |
|---|---|
urls | string[] |
concurrency | any |
Returns: Promise<CrawledPage[]>
Was this page helpful?