Find Jobs
Hire Freelancers

AWS Lambda / Python Guru - Re-Architecture

$750-1500 USD

Selesai
Dibuat lebih dari 6 tahun yang lalu

$750-1500 USD

Dibayar ketika dikirim
Each month we receive a CSV data file that has about 5-6k records. We need that data processed and put into a MySQL data base. Here is the background and workflow: 1) We receive the raw data file as a CSV (via email) 2) We have a script that uploads that CSV to a AWS S3 bucket (called raw_data) 3) A Python Lambda is triggered that iterates over each of the records in the file.....for each record we pull a unique identifier and use that identifier to call an external api and pull in other relevant infoation 4) the results from #3 are inserted into a AWS MySQL RDS instance. 5) Move the raw data file from the raw_data bucket to a processed_data bucket So if it's working what else do we need? We'll the max time a Lambda can run is 5 mins, and the script is timing out before if finished so we need to refactor it. What we would like, is to update it to the following pattern: 1 Master lambda that reads all of the records and fans out the processing via SQS (as 5-6k messages). Then have another lambda that is triggered that will only process one message at a time. The deliverables would include: 1) All source code (the current lambda python script would need to be updated) 2) Cloud Formation Templates 3) Configured security groups such that ports, etc aren't open to the world and the only the applicable resources can talk to one another (S3, SQS, Lambda, RDS, etc) Open to fair estimates for this project
ID Proyek: 15775786

Tentang proyek

23 proposal
Proyek remot
Aktif 6 tahun yang lalu

Ingin menghasilkan uang?

Keuntungan menawar di Freelancer

Tentukan anggaran dan garis waktu Anda
Dapatkan bayaran atas pekerjaan Anda
Uraikan proposal Anda
Gratis mendaftar dan menawar pekerjaan
Diberikan kepada:
Avatar Pengguna
Great idea! I just think Redis is faster than SQS since you want to use the queues just from s3. SQS is fine when you want to use the queues from outside of AWS. Relevant Skills and Experience - AWS Lambda, SQS, RDS - Redis, Celery - Ansible Proposed Milestones $950 USD - all
$950 USD dalam 10 hari
4,8 (234 ulasan)
8,3
8,3
23 freelancer menawar dengan rata-rata $1.168 USD untuk pekerjaan ini
Avatar Pengguna
Thank you for looking at our bid. Separate Lambada per record that would process (instead of one Lambda processing all records as being done currently), request your response. (guess externalizing api call through an independent Lambda would not make your current master process faster) Relevant Skills and Experience 100+ projects through Freelancer - Firebase, AWS, S3, dynamo, kindly visit our profile on Freelancer to review our past. Proposed Milestones $1500 USD - to be discussed Just need to review few things (1-2 days) & discuss before accepting award. We have an AWS dev account, may be inactive post trial period. We request your response to discuss.
$1.500 USD dalam 21 hari
4,9 (79 ulasan)
7,9
7,9
Avatar Pengguna
Hello, I'd be glad to complete this python project for you. Loking forward to chat with you soon for more details. Best regards, Relevant Skills and Experience Python , data processing, automation Proposed Milestones $882 USD - payment after project completion
$750 USD dalam 7 hari
5,0 (180 ulasan)
7,4
7,4
Avatar Pengguna
Hello sir I am a python developer with 7 years of experience. I am very familiar with aws s3, lambda, sqs and etc. I am confident with this job and can start the work right now. Best, Zhang Relevant Skills and Experience python, aws Proposed Milestones $888 USD - Re-Architecture
$888 USD dalam 10 hari
4,9 (127 ulasan)
7,7
7,7
Avatar Pengguna
I have read your project details. I have to ask a few questions. Can you please message me via chat so we can discuss all the details to elicit all the requirements and hence start the development? Relevant Skills and Experience I will showcase all the skills and my experience for this project over the chat. Please message me so we can resume this discussion. Stay tuned, I'm still working on this proposal.
$1.250 USD dalam 20 hari
5,0 (42 ulasan)
7,6
7,6
Avatar Pengguna
Hello, I always provide fastest results with quality,I have done more than 300 projects here..I have an experience of more than 10 years in web development and maintenance. Relevant Skills and Experience I have in-depth knowledge of php, mysql, javascript , jquery, paypal integrations, API's, css, html, html5, wordpress. I will look forward to discuss with you. Thank you. Proposed Milestones $988 USD - milestone
$988 USD dalam 20 hari
4,6 (125 ulasan)
7,7
7,7
Avatar Pengguna
Hi I am interested in this project. I am high skilled Python developer and know well about Amazon Lambda. I understood your requirements and ready to discuss. Let us discuss in detail Relevant Skills and Experience Python, Amazon Lambda Proposed Milestones $1411 USD - disucss
$1.411 USD dalam 20 hari
5,0 (42 ulasan)
7,0
7,0
Avatar Pengguna
Hi, .I specialise in web development. I live in Oxford. Your project looks exciting to me and I’ve got the skills required Relevant Skills and Experience Amazon Web Services, Aws Lambda, MySQL, Python, Software Architecture Proposed Milestones $1250 USD - project milestone I've done someting similar recently, will be happy to share.
$1.250 USD dalam 20 hari
5,0 (18 ulasan)
6,1
6,1
Avatar Pengguna
This bid is to modify an existing AWS lambda function by leveraging the SQS service and a sub-lambda to process the file incrementally in order to avoid timing out. Relevant Skills and Experience Python, AWS, S3, Data Processing, MySQL Proposed Milestones $1000 USD - Modify / test the existing Lambda function to overcome timeout issue.
$1.000 USD dalam 10 hari
5,0 (44 ulasan)
6,3
6,3
Avatar Pengguna
Thanks for sharing the details. I can help you with programming and infrastructure both. I have been dealing with these services since 14+ years. Relevant Skills and Experience PHP MySQL JAVAScript Ajax CSS HTML5 IOS Android Proposed Milestones $1250 USD - To be decided after discussion
$1.250 USD dalam 30 hari
5,0 (51 ulasan)
5,9
5,9
Avatar Pengguna
We love challenging opportunities. Looks like AWS configuration/services is a major issue here. We are clear about the requirements and our certified AWS solution architect can certainly solve it. Relevant Skills and Experience AWS Certified Developer Proposed Milestones $1323 USD - AWS Configuration + Python Script review Are you open to changes in AWS services being used?
$1.323 USD dalam 12 hari
4,3 (51 ulasan)
6,8
6,8
Avatar Pengguna
Amazon AWS Expert Team We are a competent team for AWS and proud member of Amazon AWS global partner network (APN) and a certified AWS service provider. Skills and Expertise We help our clients at every step of their Cloud journey including - Planning, Procurement, Deployment, Migration, Optimization, Support and security. Our managed service helps optimizing Cloud efficiently for Cost, Speed, Scale, Security and Automation. In order to understand more about your project requirements. Kindly initiate chat. Thanks! aastha
$750 USD dalam 7 hari
4,9 (46 ulasan)
6,0
6,0
Avatar Pengguna
Hello AWS Lambda / Python Guru - Re-Architecture Relevant Skills and Experience Amazon Web Services, Aws Lambda, MySQL, Python, Software Architecture Proposed Milestones $1250 USD - when i did it I have some questons for your project now thx
$1.250 USD dalam 20 hari
4,9 (26 ulasan)
5,7
5,7
Avatar Pengguna
Dear sir alredy done this type of job having 8 year exp in php , pls ping me for more detail am waiting your reply ! Relevant Skills and Experience 10 year exp in amazon web services Proposed Milestones $1666 USD - complete Stay tuned, I'm still working on this proposal.
$1.666 USD dalam 25 hari
4,9 (30 ulasan)
5,8
5,8
Avatar Pengguna
A cloud engineer with 2.5 years of hands-on experience with AWS clouds. Has delivered high quality, scalable, available, reliable and fault-tolerant solutions using AWS. seeks quality work Relevant Skills and Experience I have expertise in following areas 1. Lambda(Integration with SNS, SQS, S3, dynamodb, API gateway, Kinesis etc) 2. SQS(Decoupled architecture, SAM) 3. S3, EC2, ECS, Kinesis,Dynamodb etc 4. python etc Proposed Milestones $150 USD - Project Award $300 USD - First Milestone(deliverable tbd) $300 USD - Final Milestone(deliverable tbd) Can we discuss further and refine the architecture?
$750 USD dalam 20 hari
5,0 (21 ulasan)
5,2
5,2
Avatar Pengguna
I have been in this field for more than 7 years. I have experience in php,perl,wordpress,Html,postgress ,Web Services, Rest Api, Zend ,YIi,Mysql. Relevant Skills and Experience so I am right candidate for your project because I have experience in C#Aws,,Perl,php,node.js,Crypto currency,Wordpress,Ebay,Amazon,Instagram,Mysql,Mongo Db,Api projects. Proposed Milestones $1250 USD - AWS Lambda / Python Guru - Re-Architecture
$1.250 USD dalam 20 hari
5,0 (20 ulasan)
5,4
5,4
Avatar Pengguna
Hi, I am Python expert and have rich experiences with AWS. I can complete this project successfully. Hope to work with you. Stay tuned, I'm still working on this proposal.
$1.323 USD dalam 20 hari
5,0 (11 ulasan)
4,8
4,8
Avatar Pengguna
Can help you with this. Message me . I will deliver in 10 days. Relevant Skills and Experience more than 5 years experience in full stack development Proposed Milestones $1500 USD - code,deployment
$1.500 USD dalam 10 hari
5,0 (6 ulasan)
4,0
4,0
Avatar Pengguna
Hi, I am an experienced programmer and I'm sure I can complete your project asap. Please contact me and tell about your project details. I promise a high quality and punctual work. Thanks. AWS Lambda / Python Guru - Re-Architecture Amazon Web Services, Aws Lambda, MySQL, Python, Software Architecture,
$1.250 USD dalam 6 hari
4,9 (8 ulasan)
4,2
4,2
Avatar Pengguna
Lets introduce our company Gropse Technologiesthat has been in business for the last 5 years developing mobile apps and websites. We have an excellent track record for the best customer satisfaction. Stay tuned, I'm still working on this proposal.
$1.250 USD dalam 20 hari
5,0 (7 ulasan)
4,2
4,2
Avatar Pengguna
As a python expert, I will like to complete your project. we can discuss details over chat.
$750 USD dalam 5 hari
0,0 (0 ulasan)
0,0
0,0

Tentang klien

Bendera UNITED STATES
BALTIMORE, United States
5,0
97
Memverifikasi Metode pembayaran
Anggota sejak Jul 18, 2007

Verifikasi Klien

Terima kasih! Kami telah mengirim Anda email untuk mengklaim kredit gratis Anda.
Anda sesuatu yang salah saat mengirimkan Anda email. Silakan coba lagi.
Pengguna Terdaftar Total Pekerjaan Terpasang
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Memuat pratinjau
Izin diberikan untuk Geolokasi.
Sesi login Anda telah kedaluwarsa dan Anda sudah keluar. Silakan login kembali.