Amazon Kinesis vs Google Pub/Sub

Amazon Kinesis Data Streams and Google Cloud Pub/Sub are the primary managed streaming services on their respective clouds. Both enable real-time data processing, but their scaling models differ significantly.

Scaling Philosophy

Kinesis requires you to manage shards. Each shard provides fixed throughput (1 MB/s in, 2 MB/s out). You must estimate capacity, provision shards, and reshard as traffic changes. This is similar to Kafka's partition model.

Pub/Sub abstracts away all capacity management. You create a topic and publish. Google handles partitioning, replication, and scaling transparently. There are no shards, no throughput units, and no capacity planning.

Consumer Throughput

Kinesis limits consumers to 5 GetRecords calls per shard per second. With multiple consumers, this limit is shared. Enhanced Fan-Out provides dedicated 2 MB/s per consumer per shard but adds cost.

Pub/Sub has no such per-subscription limits. Each subscription can consume at whatever rate it needs. This makes fan-out to many consumers significantly simpler.

Ordering Guarantees

Kinesis provides strong ordering within each shard, determined by partition key. All records with the same partition key go to the same shard and are ordered.

Pub/Sub offers ordering via ordering keys. Messages with the same ordering key are delivered in order. However, Pub/Sub's ordering is a newer feature and not as battle-tested as Kinesis's shard-based ordering.

Cost Comparison

Kinesis charges per shard-hour plus per PUT. Costs are predictable but scale with shard count. Pub/Sub charges per data volume (ingestion + delivery). For variable workloads, Pub/Sub's pay-per-use model avoids over-provisioning costs.

Both services integrate deeply with their cloud ecosystems. The choice typically follows your primary cloud platform. See our system design interview guide and pricing page for cross-cloud analysis.