I need a web crawler that does the following:
(1) A url is entered of a site that sells products online
(2) The web crawler then visits the site and enters the following into a datrabase
- product url
- product title
- product image url
- product price
- product description
- product currency
(3) The types of sites I want crawled will include [login to view URL], [login to view URL] and [login to view URL]
The difficulty with some sites will be that they are flash based images and some are standard images (jpeg,gif, bmp)