Web Scraping Library with Proxy Rotation

[armanist]

No description provided.

[armanist]

Develop a streamlined web scraper library with key functionality:

Proxy Rotation: Automate proxy usage to avoid rate limits and IP blocks.
Data Extraction: Allow customizable data extraction patterns (e.g., CSS selectors, XPath) for broad compatibility across sites.
HTTP Requests: Use existing HttpClient library for managing requests and responses.
Requirements:

Proxy Rotation:
Design a ProxyManager component that rotates proxies based on predefined rules.
Allow users to set proxies manually or load from a list.
Data Extraction:
Include support for flexible, user-defined selectors.
Allow modular selectors for various page structures and content types.