File Formats & Parquet
A retail analytics team stores 6 months of clickstream logs in S3 as gzip-compressed CSV files. Their SageMaker training jobs spend most of their time scanning full files to select only 8 of 40 columns, and storage costs are climbing. They want a format that supports column pruning and predicate pushdown for faster, cheaper reads. Which file format should they migrate to?
Parquet is a columnar format that lets query and training engines read only the required columns (column pruning) and skip row groups using min/max statistics (predicate pushdown), directly solving the "read 8 of 40 columns" problem while also compressing better than row-oriented CSV.