Python Web Scraping Developer

需求說明

About the Role We are looking for an experienced Python Web Scraping Developer to help us build and maintain reliable scrapers for extracting product data and images from multiple e-commerce websites. The ideal candidate will have strong technical skills, attention to detail, and the ability to adapt scrapers to changing site structures while ensuring clean, organized data output. Responsibilities Develop and maintain robust web scraping scripts to extract product data (text and images) from various e-commerce websites. Implement systems to ensure accurate and complete image extraction, troubleshooting issues when images fail to pull. Design and build a hierarchical folder structure for categorizing and storing product images (e.g., Shoes/Nike/…). Write clean, well-documented, and efficient code in Python using libraries like Scrapy, BeautifulSoup, Selenium, requests. Monitor scraper performance, handle errors gracefully, and adapt quickly to changes in site layouts (HTML, CSS, JavaScript). Ensure optimized image handling, storage, and compatibility across different formats. Required Qualifications Proven experience in web scraping, including dynamic/JavaScript-heavy sites. Strong proficiency in Python and scraping libraries (Scrapy, BeautifulSoup, Selenium, requests). Solid understanding of HTML, CSS, XPath, and DOM structures for locating and extracting data. Hands-on experience with image handling and storage optimization. Familiarity with version control systems, especially Git. Desired Skills (Nice-to-Have) Experience with cloud platforms (AWS or Google Cloud) for deploying/running scrapers at scale. Knowledge of databases (MongoDB, PostgreSQL) for structured storage of extracted data. Familiarity with anti-bot techniques (e.g., proxies, rotating user agents, CAPTCHA solving). Why Join Us? Work on a project that directly impacts how e-commerce data is collected and organized. Flexible remote work with growth opportunities. Exposure to cutting-edge scraping techniques and scaling on cloud platforms. How to Apply Please include: A short summary of your relevant experience.