// PlaywrightCrawler crawls the web using a headless // browser controlled by the Playwright library. const crawler = new PlaywrightCrawler({ // Use the requestHandler to process each of the crawled pages. async requestHandler({ request, page, enqueueLinks, log }) { const title = await page.title(); log.info(`Title of ${request.loadedUrl} is '${title}'`);
// Save results as JSON to ./storage/datasets/default await Dataset.pushData({ title, url: request.loadedUrl });
// Extract links from the current page // and add them to the crawling queue. await enqueueLinks(); }, // Uncomment this option to see the browser window. // headless: false, });
// Add first URL to the queue and start the crawl. await crawler.run(['https://crawlee.dev']);
GitHub 地址→https://github.com/apify/crawlee
2.4 云原生数据库:surrealdb
本周 star 增长数:2,850+,主语言:Rust
一个用于实时网络的可扩展、分布式、可协作的文档图数据库。它通过简化数据库和 API 栈来减少应用程序的开发时间。SurrealDB 可以作为单机运行,也支持以高可用、高可扩展的分布式形式运行。它支持来自客户端设备的 SQL 查询、GraphQL、ACID、WebSocket、结构化和非结构化数据处理、全文索引、地理空间查询及权限控制。