Robust Data collection/scraping powered by AWS

Selesai Dipasang 5 tahun yang lalu Dibayar saat pengiriman
Selesai Dibayar saat pengiriman

Requirements:

1. Continuously and reliably scrape and collect job posting data from websites like indeed, dice, careerbuilder, Monster, etc. (any one or two would be sufficient). The best solution would be rotating among those sites.

2. It queries jobs based on a randomly generated combination of keywords, such as "java, Dallas Texas".

3. It should be disruption-free and utilize AWS Spot EC2 instance to power the scraping. That means, the solution should include programmatically create a spot instance and start working there.

4. The collected data should be saved to a central server, in a format of zipped csv file or Mongodb.

Amazon Web Services NoSQL Couch & Mongo Parallel Processing Web Scraping

ID Proyek: #16990869

Tentang proyek

4 proposal Proyek online Aktif 5 tahun yang lalu

Diberikan kepada:

zekovicm

Hi there,I am Miljan,web scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I can start immediately and finish it within Lebih banyak

$155 USD dalam 3 hari
(71 Ulasan)
6.7

4 freelancer rata-rata menawar $176 untuk pekerjaan ini

mantislin

Hi sir, This is Lin and I am scraping expert, i have checked all details for your project. can we discuss more info then i can provide example data for you? Please message me then we can discuss more ASAP. Lebih banyak

$172 USD dalam 5 hari
(260 Ulasan)
7.5
cyberskytech

We are a small team of experienced IT professionals who excel in System Engineering, DevOps, Cloud, Web Development and Cyber Security. Our primary goal is to provide the best solutions for the least cost. Managed I Lebih banyak

$155 USD dalam 10 hari
(3 Ulasan)
2.2