Exam Overview
Exam Details
- Duration: 180 minutes
- Questions: 65
- Passing Score: 750/1000
- Format: Multiple choice & multiple response
- Cost: ~$150-300 USD
- Validity: 3 years
Exam Domains
| Domain | Weight |
|---|---|
| Collection | 18% |
| Storage | 22% |
| Processing | 24% |
| Analysis & Visualization | 18% |
| Security | 18% |
Amazon Kinesis Family
| Service | Purpose | Key Details |
|---|---|---|
| Kinesis Data Streams | Real-time streaming | Shards (1 MB/s in, 2 MB/s out); retention 1-365 days; consumers: Lambda, KDA, KCL |
| Kinesis Data Firehose | Load streaming data to destinations | Managed; near real-time (60s buffer); destinations: S3, Redshift, OpenSearch, Splunk |
| Kinesis Data Analytics | SQL/Flink on streaming data | Real-time analytics; output to Firehose or streams |
| MSK (Managed Kafka) | Apache Kafka managed service | Full Kafka compatibility; MSK Serverless |
⚠ Kinesis vs SQS
- Use Kinesis when: multiple consumers, replay needed, ordered per shard, real-time analytics
- Use SQS when: decoupling, one consumer, simplicity
AWS Glue
| Component | Purpose |
|---|---|
| Data Catalog | Metadata repository; tables, schemas, partitions; integrates with Athena, EMR, Redshift |
| Crawlers | Auto-discover schema from S3, RDS, DynamoDB; update Data Catalog |
| ETL Jobs | PySpark/Python scripts; serverless; transform data between sources |
| DataBrew | No-code data preparation; visual transformations; 250+ transforms |
| Streaming ETL | Continuous ETL from Kinesis/Kafka to S3/Redshift |
Amazon Redshift
- Columnar storage — optimized for OLAP aggregations
- MPP — Massively Parallel Processing; leader node + compute nodes
- Redshift Spectrum — query S3 directly without loading into Redshift
- Redshift Serverless — no cluster management; auto-scaling; pay per compute
- Distribution Styles: EVEN, KEY, ALL — control data distribution across nodes
- Sort Keys: COMPOUND or INTERLEAVED — optimize query predicates
Lake Formation
- Simplifies building secure data lakes on S3
- Fine-grained access: table, column, row, cell-level permissions
- Governed Tables — ACID transactions on S3 with automatic compaction
- Tag-Based Access Control (LF-TBAC) — scale permissions with tags
- Cross-account sharing of databases/tables
Data Lake Architecture
Bronze (Raw) → Glue Crawlers → Data Catalog
↓
Silver (Cleansed) → Glue ETL → Parquet/ORC partitioned by date
↓
Gold (Curated) → Redshift / Athena for analytics
↓
Visualization → QuickSight / Grafana dashboards📋 Study Checklist
Progress0%
- Design end-to-end analytics pipelines on AWS
- Know Kinesis: Streams, Firehose, Analytics
- Understand Kinesis shard sizing and partition keys
- Know when to use Kinesis vs SQS vs MSK
- Use Glue Crawlers and Data Catalog for schema management
- Write Glue ETL jobs in PySpark
- Design Redshift cluster: node types, distribution, sort keys
- Use Redshift Spectrum to query S3
- Design a data lake with Bronze/Silver/Gold layers
- Configure Lake Formation for fine-grained access control
- Use Athena with partitioning and columnar formats for cost efficiency
- Know OpenSearch: indexing, shards, replicas
- Design real-time dashboards with QuickSight
- Understand EMR: cluster types, Spark on EMR
- Know DynamoDB Streams → Lambda → Kinesis for CDC
- Understand S3 lifecycle + Intelligent-Tiering for data lake cost
- Configure CloudTrail data events for S3 analytics access logging
- Know encryption for data at rest and in transit
- Understand QuickSight SPICE vs direct query mode
- Design for compliance: data residency, encryption, audit