Create a script to check websites for their http status (100,000 sites per minute)

$250-750 USD

Ditutup

Dibuat

lebih dari 9 tahun yang lalu

$250-750 USD

Dibayar ketika dikirim

We're looking for a solution that will check millions of websites for their https status. Basically we need a script or software that will run literally tens of thousands of simultaneous threads. We have huge servers capable of processing scripts that resource heavy. We will upload lists of 1-10mil domains at a time, and the script will check each domain for the response code. Example codes: 200 - alive 301 - redirect 404 - not found Etc No proxies will be necessary as the script will be calling a different domain/host in each instance and won't be hammering the same ip/host. We need a solution that will be able to check at least 100,000 domains per minute for their response codes. The results will need to be able to be stored in a MySQL database with a simple access interface. Looking forward to your solutions!

ID Proyek: 6575636

Tentang proyek

15 proposal

Proyek remot

Aktif 9 tahun yang lalu

Ingin menghasilkan uang?

Alamat email

Keuntungan menawar di Freelancer

Tentukan anggaran dan garis waktu Anda

Dapatkan bayaran atas pekerjaan Anda

Uraikan proposal Anda

Gratis mendaftar dan menawar pekerjaan

15 freelancer menawar dengan rata-rata $513 USD untuk pekerjaan ini

@kssoftphp

I can make script who can check 1 lac domain in every minute. I am expert in Programming and I can develop these type of scripts.

$700 USD dalam 10 hari

4,9

(101 ulasan)

7,0

@odessky

Hi This looks a simple task I can create console script You need have powerful server for run it!

$250 USD dalam 10 hari

4,8

(164 ulasan)

6,6

@websoftech

I already make that kind of script for my client, but let me explain very clearly that its not depend on script that fetch 100000 data in one minute, but its all depend on server where its run, and most impotent its depend on Internet speed on server, some web hosting block that kind of script, if you have very strong internet then its possible, but if some one said that he can make script which can run on simple internet and it can grab 100000 result so its all fake, You need very strong Internet pipe line for that , because script run on internet , if it have about 40 to 50 mb per second so it can fetch result. but i can give you maximum results. lets talk in detail

$333 USD dalam 10 hari

4,8

(10 ulasan)

6,9

@virtuarchitects

Hi. I have understood exactly how you want a script to run millions of threads to check the HTTP status as per the response code mentioned in the description & want them stored in MySQL database which shall be view-able via admin panel in a simple format. I can deliver the required work within 5 days, starting immediately. I look forward to work on this project with my full potential targeting for a long term business relation with you. Please message me for further discussion & proceedings. Thanks! :)

$250 USD dalam 5 hari

5,0

(162 ulasan)

6,3

@ankasky

hi my priority is to provide a quality work and i am ready to start work in this project i am a software engineer i am good and experienced web developer we can discuss more in chat

$578 USD dalam 20 hari

4,8

(92 ulasan)

5,9

@saymrs9

Hi, I've a team of well experienced web and mobile app developers, our developers have 4 years of experience in average. Your project is very interesting that's why we are desperate to develop this application. Please message me if there's any other requirement about this awesome project. Regards Samya

$750 USD dalam 18 hari

4,5

(30 ulasan)

5,1

@dapu92

Only reason to consider my proposal should be that i am not asking you to award me project until your job is done. And also i will cut down my rates. Regards, Darpan

$555 USD dalam 2 hari

4,5

(17 ulasan)

4,8

@osfreelanceruk

Hi, I’m a London based software engineer with more than 10 years of experience working on enterprise level applications builded with Python, Java and PHP. I can guarantee highest code quality (best design patterns, libraries and framework) which makes my solution easy to maintain and extend with highest performance. I have a lot of experience with custom solutions builded with most popular frameworks like Symfony, Zend or Spring. I can build distributed well scalable architecture based on high-performance solutions including nonblocking IO operations, threads and queues. If you have any questions feel free to contact with me.

$666 USD dalam 3 hari

4,6

(7 ulasan)

4,7

@kanata1988

100,000 domains per minute means 1,666 domains per seconds. How many servers do you have to execute it? What about the internet speed, is it fast enough to fetch 1,666 domains per second? As for the script, I could create a multi-threading task with PHP using pThreads. Maybe we should make 20-50 threads per server. It needs to be tested though to achieve maximum result. Maybe node.js is better suited for this job than PHP. You can contact me to further discuss this. Thanks, Riko Nagatama

$700 USD dalam 10 hari

5,0

(7 ulasan)

4,0

@vigidis

Hello, My proposition for the implementation would be a C# or Java application (based on server architecture) that would spawn a thread establishing a connection to the server via http, read the response code and save it to the MySQL DB or die after a certain time (in order to manage timeouts). I have experience working on similar projects for my university degree in Computer Science. Please feel free to contact me for further information. Regards, Sabin.

$444 USD dalam 10 hari

0,0

(0 ulasan)

0,0

@paulohrpinheiro

Hello! The project can only be executed in PHP? I have extensive experience in management and development of systems for Internet infrastructure, mainly programming in python. What types of queries you need? I present budget for this project keeping in mind three reports: 1) summarization by error code (choosing a period of time) 2) information about a specific domain (also per period) 3) list of domains (choosing the return code and a period of time) Any questions, feel free to ask.

$277 USD dalam 3 hari