AWS - Kinesis

• Streaming data is data generated continuously by thousands of sources, which typically send data simultaneously in small sizes.
• Kinesis Streams:
• Consists of Shards
• Each shard has 5 transactions per second for read, with upto 2MB per second read rate, and upto 100 records per second for write with upto 1MB per second of write rate.
• Data capacity of your stream is a function of number of sharts you specify for your stream. Total capacity of the stream is sum of capacity of its shards.
• Kinesis Firehose:
• Amazon Kinesis Data Firehose is the easiest way to reliably load streaming data into data stores and analytics tools. It can capture, transform, and load streaming data into Amazon S3, Amazon Redshift, Amazon Elasticsearch Service, and Splunk, enabling near real-time analytics with existing business intelligence tools and dashboards you’re already using today. It is a fully managed service that automatically scales to match the throughput of your data and requires no ongoing administration. It can also batch, compress, transform, and encrypt the data before loading it, minimizing the amount of storage used at the destination and increasing security.
• Kinesis Analytics:
• Amazon Kinesis Data Analytics is the easiest way to process streaming data in real time with standard SQL without having to learn new programming languages or processing frameworks. Amazon Kinesis Data Analytics enables you to query streaming data or build entire streaming applications using SQL, so that you can gain actionable insights and respond to your business and customer needs promptly.
• Kinesis is a way to consume big data.
• Kinesis is used to bring large amount of social media, new feeds, logs into the cloud, whereas Elastic Map Reduce is used to process Big Data.