Screen-scraping comments for academic research purposes
$30-250 USD
Dibatalkan
Dibuat sekitar 12 tahun yang lalu
$30-250 USD
Dibayar ketika dikirim
I am looking for java developers who can write a java code* that screen-scrape a specific site. I am only interested in collecting the comments found in that site for academic research purposes. It is an Arabic news site, where it has major sections like: Political news, Financial News, Sports news, Technology news,... each major section has subsections, e.g. Political News has the following subsections: Middle East News, Global News, ... . Each subsection has news items, and each news item may or may not have comments. The comments are paged.
I need the scraping code to collect the comments and put them in the following structure:
ID | Section Title | Subsection Title | News Item Title | URL | Comment | Comment Author | Timestamp
example:
1 | Political News | Global News | Mission impossible diplomacy in Beijing | [login to view URL] | this is a comment | someone | 2012-03-01 12:00:00
2 | Political News | Global News | Mission impossible diplomacy in Beijing | [login to view URL] | this is 2nd comment | som2 | 2012-03-01 12:00:00
3 | Financial News | Banking | IMF to support Some country | [login to view URL] | this is a comment | someone | 2012-05-11 12:00:00
I will run the script daily, and the output should be a CSV file.
The code must be provided.
* If you prefer to code in a different language, like Python, you may still bid on this project, but putting in mind that you must then deliver an annotated and explained code to be used and run by someone who only knows java.
Hi,
This is Nitin having HUGE experience in scraping HUGE data in least amount of time.
I code in perl & php, and scrapers written by me are being used to scrape more than 20 million pages per day without being blocked.
I would like to help you in getting all the data you are looking for.
Please pm me in case you find my bid suitable.
And don't forget to check my reviews here :
http://www.freelancer.com/users/1303125.html
Cheers,
Nitin
Expert scrapper here. I am perfect for this job, since I also know only java. I have coded all my scrappers in java only. I am confident to handle this job and provide the required output(csv).
Please check your PMB.
I have been writing web scraping applications for about 6 years and am currently employed full time as a software developer. I have approx 10-15 working scrapers currently being used.
I have put a low bid for this as I am a new member and want to boost my reputation. I have also put a large delivery time for 3 reasons, 1 I have not seen the website yet, 2 its better to overestimate than under-estimate and 3, I like to take my time and produce something worth paying for. I have also put 0% initial milestone, you can decide what you would like to see from me to improve your confidence in my work.