trafilatura
A Python tool for web scraping and crawling that extracts main text, metadata, and comments from web pages. Designed for creating text corpora and extracting structured content. More information: https://trafilatura.readthedocs.io/en/latest/usage-cli.html#further-information.
- Extract text from a URL:
trafilatura
- Extract text and save to a file:
trafilatura
- Extract text in JSON format:
trafilatura --json
- Extract text from multiple URLs listed in a file:
trafilatura
- Crawl a website using its sitemap:
trafilatura --sitemap
- Extract text while preserving HTML formatting:
trafilatura --formatting
- Extract text including comments:
trafilatura --with-comments
- Display help:
trafilatura
来源:tldr-pages contributors · 作者与贡献记录 · CC BY 4.0。离线整理了版式、链接和媒体显示。
