Consulting to improve Apache Spark program performance

$100-400 USD

Ditutup

Dibuat

lebih dari 8 tahun yang lalu

$100-400 USD

Dibayar ketika dikirim

I have a Spark application (coded with Scala) running on a cluster of 5 nodes. Each node has 125GB memory and 32 cores. There is a function/method containing 8000 loops totally. Inside the loop it calls MLlib k-Means, so the k-Means runs about 8000 thousand times. The running time of the big loop is around 5.5 hours currently on the Spark clusters. I want to reduce the running time significantly. May need to tune JVM parameters, Spark configuration parameters, or restructure the Scala application code, etc.

Scala

Spark

ID Proyek: 9103352

Tentang proyek

3 proposal

Proyek remot

Aktif 8 tahun yang lalu

Ingin menghasilkan uang?

Alamat email

Keuntungan menawar di Freelancer

Tentukan anggaran dan garis waktu Anda

Dapatkan bayaran atas pekerjaan Anda

Uraikan proposal Anda

Gratis mendaftar dan menawar pekerjaan

3 freelancer menawar dengan rata-rata $348 USD untuk pekerjaan ini

@ahmsak

Hello Sir... I am a computer science tutor, I teach (among others) Scala, Haskell, Prolog and Algorithms. I have done many projects like this, and I'm the top Scala developer, you can check here please: https://www.freelancer.com/freelancers/skills/Scala/ Please contact me for more details when possible. I look forward to work for you Sir. Best Regards.

$300 USD dalam 3 hari

5,0

(3 ulasan)

3,5

@HappyRainy

First of all, I Need to know what the current settings you are using for running the applivation now. Kmeans could be very expensive if you have big number of clusters and too many of the dimensions.

$444 USD dalam 4 hari