We're an insurance company from to Santiago of Chile. We need to scrape a website and pull prices of our competitors using dummy data as an input. We are mainly interested in the code algorithm of a scraper but under certain conditions. This should be done with Python+Selenium with random proxies and random agents, this is to avoid banning our bots.
We have a code Jupyter-Notebook to scrape the website but without random proxies-agents, we are attaching the code as a file. The main goal is to create a Python-Selenium algorithm (3.8 version or upper, and we're working with jupyter-notebook) under Chrome or Mozilla browser, furthermore is very important to use random proxies (public proxies desirably), random agents and this should be working for every requests on the website without bans or failures (bans for ip, bans for requesting many times the same customer id, etc...). The process starts in "[login to view URL]", you should click the button "Cotiza Aquí" and will load a page of customer information (this could be a dummy input), after providing this information you would get to prices of all competitors, you should save these prices like an object in Python.
We offer $$$$ US for who is interested in this project and meet the profile, the needed skills are:
- Python + Selenium for scraper websites
- Knowledge about the parameters proxies and agent in Selenium
IMPORTANT:
• If accepted, the above project shall be undertaken under the utmost confidentiality, both as regards its results and the dummy data provided therein.
• Without prejudice to above, if selected, you shall be required to enter into a binding NDA and a Data Processing Agreement under terms satisfactory to the Company.
• The project is to be carried out within the limits set forth under applicable regulation and your commitment to comply with all applicable regulation is expected.
Hi there, I have good experience in python with selenium and also have good experience in proxy handling.
Message me and lets take this forward.
Thanks
$240 USD dalam 10 hari
5,0 (23 ulasan)
4,4
4,4
8 freelancer menawar dengan rata-rata $169 USD untuk pekerjaan ini
Hi there,
Let’s have a quick chat to discuss this project.
I am expert in Python, PHP, JavaScript,Web Scraping,MYSQL.I do have expertise for this project.
You can check my portfolio here:- https://www.freelancer.com/u/PoojaRautela417?page=portfolio&w=f&ngsw-bypass=
Looking forward to hear from you soon.
Regards
Pooja Bohra
Hi
I had gone through description and notebook. I had solved the same hurdle u are facing for another client in US. For that I used HMA proxy and configured it to change every minute. Thus I was able to get it running without ban. This is the easiest way to solve it. More could be shared in chat
Hi there,
I would love to work on your project with the strong skills needed for the project.
However I have to clarify few more questions which can be discussed to get scope locked.
Please initiate chat to move further.
Hi there I use python with selenium on a daily basis doing pretty much exactly what you require. I would use browser profiles to set user agents and proxies would be scraped from a public site as the bot runs. I do also have access to a large private pool of residential US proxies if you’d prefer the bot to run completely undetected, as public proxies can be iffy.
Hoping to hear from you, a little more information would be appreciated. Have a good day