Scrapy Jobs

valid email addresses harvested from a list I'll give you. Objective: outreach to sell books. You might use the stack you are most comfortable with Python + BeautifulSoup, Scrapy, Selenium, Node.js + Puppeteer, or similar 'as long as the final data is clean and deduplicated' Deliverables: • A CSV file containing each email address alongside the exact page URL where it was found • A brief note on the toolchain or script used (for reproducibility) Accuracy matters more than sheer volume Do Not give me a bloated list full of bounces.. I only neeed proper clean verified and contactable info.

We need a robust yet lightweight script that can automatically pull business details from a publicly accessible government website. The information to capture will centre on business details, such as registration numbers, business name, registration date, address, etc. The workflow should: • Navigate every relevant section of the site (pagination, search filters, subsidiary pages). • Extract the required fields accurately • Export clean, structured data to CSV and JSON A Python solution leveraging requests/BeautifulSoup or Scrapy is preferred, but I’m open to other dependable stacks if they handle rate-limits, retries, and potential CAPTCHA gracefully. The script must be easy to rerun on demand, with clear instructions for environment setup and any dependencies...

Seeking a skilled web designer to create a clean, minimalistic website to showcase services. The website's primary function will be to highlight our offerings in a sleek, digestible format that engages visitors and inspires a sense of professionalism and quality. Features needed include: - A contact form to facilitate customer inquiries - Subscriber sign-up option for customers to receive updates and newsletters - A blog/news section to post updates and relevant content - An 'add listing' feature to display products effectively - Banner slide out et - Social Scroll It's crucial for the designer to have a keen eye for creating layouts that promote readability and user experience. Experience in minimalistic design can use template. Images & website examples will be...

# Python Developer (Scrapy / FastAPI / Docker) for Crawling Project We are looking for an experienced Python developer to build a structured web crawling system. This is not a large corporate project. It is a technically clean, well-defined system with a long-term perspective. We value clean architecture, stability, and understandable code. The system will process approximately 30,000 profiles per week and will be connected to an existing Laravel-based admin dashboard. --- ## Responsibilities * Develop a crawler using **Scrapy** * Build a small **FastAPI interface** (start / stop / status / statistics) * Integrate with **MariaDB** * Implement a media pipeline: download → local cache → upload to Wasabi (S3) * Implement hash-based change detection * Implement soft-delete logi...

I am looking for an experienced Web Scraping and Automation Developer to build an automated daily workflow for my business. I respond to over 100 government tenders a year and need to automate the discovery and document retrieval process. The Goal: Scrape 9 Australian Government tender websites daily for newly published tenders in two specific categories (UNSPSC 43000000 - IT, and 81000000 - Engineering/Research). Extract key details (Title, Agency, Closing Date, URL) and insert them into a centralized Google Sheet. Automatically download the associated Tender Documents and save them into uniquely named folders in my Google Drive. Periodically monitor these specific tenders for any newly published Addendums, and automatically download them to the respective Google Drive folder. The Website...

I have a list of roughly 1500 URLs—each coming from the same automotive website—that together cover the top 100 makes, models, grades and variants sold in Australia. I need every data point the site makes available for each of those vehicles, from the obvious specs such as year, make, model and variant right through to driveway prices, engine type, transmission, drive configuration, warranty details, fuel-economy figures, in-car technology features, seating layouts and any other attributes exposed on the page. The end goal is a clean, analysis-ready Excel workbook that lets me run market-wide comparisons, so consistency is critical: headings must be standardised, units normalised and categorical values written the same way across the entire sheet. I am happy for you to use P...

I need a reliable script that can pull live pricing details for car rentals from both the rental companies’ own sites and the big aggregator platforms in one pass. The goal is to feed it pickup / drop-off locations, dates, and driver age, then receive a clean CSV or JSON that lists vehicle class, daily and total price, currency, taxes & fees, and the URL it was scraped from. The scraper has to: • Navigate each target site automatically, including date pickers and location selectors. • Rotate user-agents / proxies or apply any other anti-bot tactics necessary to stay undetected. • Capture and log errors so a failed request never silently drops a row. • Be easy for me to rerun on demand—command-line or small web UI is fine, as long as setup is s...

I need a clean, well-documented script that automatically gathers publicly available biographies of football players directly from their official websites and exports everything into a single CSV file. The focus is strictly on biographical details—no match statistics or transfer news for now—so the crawler should identify, parse, and normalise information such as full name, date of birth, nationality, position, current club, height/weight (where listed), and any notable career highlights that appear on the player’s own site. Because these pages vary in structure, the code should be resilient: graceful error handling, user-agent rotation, and clear selectors or XPath rules that are easy for me to extend later. I’m comfortable running Python, so libraries like Reques...

I want to build a clean, well-structured dataset of restaurants throughout Portugal by scraping publicly available information on Google (Search / Maps). The core of the job is to capture three things for every venue you find: • complete contact details (name, address, phone, email or website if listed) • ratings plus review count pulled exactly as Google displays them • whether the place offers a fixed-price menu (“menu do dia”) and, if so, its price and what is included (possible 5 stages: Entrance, Plate, Drink, Dessert, Coffee) but it can be only 2 stages out of 5 (example: Plate+Drink) Please also tag each record with its district and municipality so the file can be filtered regionally. Deliverables 1. A single CSV or Excel file containing one row ...

I need you to collect 1,000 product images, titles, and links from Temu. Just these three items. I think this should be pretty straightforward. If you do a good job, I’ll hire you on a long-term basis, as I’ll need someone to help me collect product information from Temu on an ongoing basis.

I need a reliable developer to build a fully-automated system that gathers fresh, publicly listed email addresses from Google search results (pulled through a SERP API or an equivalent method) every single day, verifies them for validity, and delivers a clean CSV ready for my marketing campaigns. Here’s what I’m after: • Workflow 1. Submit a set of keywords or niche phrases. 2. Crawl the top Google result pages returned by a SERP service, extract any visible email addresses, and capture the source URL and page title. 3. Run each address through an SMTP-level verifier (ZeroBounce, NeverBounce, or an in-house Python verifier—whichever you prefer, as long as it returns status codes for valid, invalid, catch-all, disposable, and role accounts). 4. O...

I'm looking for a skilled web scraper to extract product images and descriptions from Yupoo. The scraped data should be organized and delivered in a CSV file. Requirements: - Experience with web scraping tools and technologies - Ability to handle dynamic content on Yupoo - Attention to detail to ensure data accuracy - Proficient in data organization and CSV formatting Ideal Skills: - Python or similar programming languages - Familiarity with libraries like BeautifulSoup or Scrapy - Previous experience scraping ecommerce websites

Hi, I need a data scraper who can scrap a data from provided sources. Skills : Core Technical Skills The freelancer should know Python (the most common scraping language) with libraries like Scrapy, BeautifulSoup, or Playwright. They should also be comfortable with browser automation tools like Selenium or Puppeteer (JavaScript-based), since sites like TipRanks are JavaScript-heavy and need a real browser to render. Anti-Bot Bypass Experience This is the most critical skill for your specific case. Look for someone experienced with handling CAPTCHAs (2Captcha, Anti-Captcha services), rotating proxies and residential IPs, spoofing browser headers and fingerprints, and bypassing Cloudflare or similar bot protection. TipRanks specifically uses these protections, so this experience is non-...