I have a web crawler that crawls the web and it is written in Perl.
Right now, it only gets the URL, keywords, description and outputs to a text file delimited by tab. I would like it so that the crawler will get the title correctly, with the proper error processing routine.
The output file needs to be in this format:
URL KEYWORDS DESCRIPTION TITLE per line, delimited by Tab.
The less overhead for the crawler, the better.
Hi sir,
I am scraping expert, I have did too many similar projects, please check my feedback then you will know.
Can you tell me more details? then I will provide demo data for you.
Thanks,
Kimi