Task: We want to bulk load our elasticsearch cluster:
> SNS topic
> Kinesis Firehose
> Maybe event bridge
1.) We will need to transform the message before loaded into elasticsearch
2.) We have over 30 million records we want fast bulk load FIFO.
3.) We want to publish a bulk amound to the SNS topic, but we dont not want multiple copies of the same records sent.
4.) We want to predict how long to complete all 30 million records
5.) We also want to keep using the process for regular updating of elastic search.
Dear Client.
I've read your job post carefully and I can complete it perfectly.
I have rich experience about Elastic Stack.
My goal is to provide high-quality service and product to clients.
Please chat with me and then discuss in details.
Thanks.