Java web crawler

oleh teguhpanjaitan
Gambar tidak ditemukan

I build java web crawler my self as company internal software. I build this crawler from concept to crawl many of our merchants website such as lazada, zalora, rakuten, poplook, etc (mostly are the biggest ecommerce in Malaysia). I design this crawler to able to fetch data around 1000+ page crawl per second but due to merchant complain I build web base control panel to control speed and another feature. The last of my day in this company I was designed the crawler to fit with AWS infrastructure using lambda and EC2 (more advance technique) to reduce server cost. Ability to achieve this project : - Good knowledge in Java, as I build this crawler all using java. The reason why we using java is a lot of tutorial out there where another developer can learn so easy to handover the job. - Advance MySql knowledge, we desingned this crawler as fast as it can. So we need to optimize MySql to fit with our requirement. - Advance HTML knowledge.

image of username teguhpanjaitan Flag of Indonesia Medan, Indonesia

Tentang Saya

Thanks for visiting my profile. I am a senior software developer with over 10 years of coding experience, 8+ years of web & mobile programming and over 4 years of cloud computing (especially AWS) for IOT and high availability server solutions. Skills: - Native PHP and PHP framework (CodeIgniter, Zend Framework 3, Symfony & Laravel) - Wordpress development - NodeJS, Python & Java - SQL (MySQL) - NoSQL (Redis, Firestore, MongoDB, DynamoDB) - Linux shell scripting for server automation - HTML, CSS, Javascript, JQuery, React & Vue - Blockchain & Cryptography - Python, experience in development of Raspberry Pi software solutions - C/C++, experience in development of arduino software solutions I am a fast learner, problem solver and focus to deliver high quality works. I also friendly & communicative person.

$19 USD/jam

64 Ulasan
5.4

Tag