1. There are a series of json file each around 15GB contains (IP+port)
2. There are series of json files each around a GB contains (traceroutes)
3. There are 3 group of csv files, each group contains 30 csv file
1st step
----------------
1. Load all the Json (1) in a MySQL table and delete the duplicates. (tbl_ports)
2. Load all the Json (2) in a MySQL table as they are traceroute IP's, Ignore the first two hop and last hop and store the rest in a SQL table. (tbl_tracceroutes)
3. Load each group of CSV file in a seprate table in MySQL and and add new column to this table and write the name of the csv file. (Do this step for the two rest of csv files group)
group1(tbl_xxx) - I will tell you the exact name when you start working
group2(tbl_yyy) - I will tell you the exact name when you start working
group3(tbl_zzz) - I will tell you the exact name when you start working
2nd Step
-----------------
1. in tbl_zzz, tbl_yyy, tbl_xxx there is a column contains subnetmask(IP+Cidr), take each of that (IP+Cidr) and look on tbl_ports on column IP, if any IP there matches to any (IP+Cidr) of tbl_zzz, you do this process for all of (IP+Cidr) for table tbl_zzz.
Create a new table (report_zzz), take some column from tbl_zzz (I will tell you which columns) and add a new column (port). for the ones that you find match write in report_zzz that row of that plus the port number you find for that (IP+Cidr).
** DO this process for tbl_xxx and tbl_zzz exactly the same way.
2. from the table report_zzz take the (IP+Cidr) data and look if any IP in tbl_traceroute match to that subnet, add new column to report_zzz named as (seen), if you find match write yes and if you don't find match write no.
** DO this process for report_xxx, report_yyy and report_zzz.
Requirements:
optimized code
As it is relatively big data, it should be fast enough
Bug free
Dynamic style
Thank you for your detailed project description. Very.
Surely I can help you. I feel confidence all the requirements you want.
I will do my best to satisfy you.
Let's have a quick chat to discuss it further.
Thanks again.
Hello,
I read out your task completely and I am interested in your post. I have confident that I will provide you quality work within your deadline. I have a deep passion for research and I guarantee that all of my work is 100% efficient I am available 24 hours at this position. A number of my samples are in my portfolio. If you want more sample then message me.
Please, allow me for this task to secure this bid and this amount is demo.
Thank you for your consideration.
Hello. How are you today?
I've read your job post carefully. This job really attracts me and also your required skills are completely matched with my major skills.
I have good knowledge and rich experience with it. I built many websites by using this so it will be an easier and suitable job for me, I think. Mainly I want a long term relationship with you after this job.
As you can see on my profile, I have been working as a full stack developer of over 7 years and is specialized in WordPress, PHP, JS frameworks and libraries and so on.
I received good reputations from clients with their successful jobs.
So once you hire me, I can start working immediately and can finish as you want.
Please ping me asap when you need me.
Thanks.
Hi, How are you?
I've carefully read your description and I am very interested in your project.
I am ready for your project, I can start immediately and finish it within the agreed deadline.
I have expertise with Python , MySql and I am sure I can provide you the best service you are looking for.
No worries, if you give me a chance I will do my best, too.,
I am sure I can complete your project on time and within budget.
I can show you my awesome skills based on my rich experiences and I will guarantee the success of you and your project.
Thanks.
Hi, I have a lot of experience optimizing databases to makes searches quicker as well as filling databases faster. I have lots of tips which can make the whole process a lot faster
My bid is higher than others, but i think thats more accurate.
I will import the json files with a short java program.
Then the csv files will be imported with "load data", the rest will be queries
Relevant Skills and Experience
Java, mysql