Tag

#s3

2 posts ·all posts

Athena's DDL, your bucket, nobody's metastore

SQE now reads Hive-style external tables over CSV, line-delimited JSON, and partitioned Parquet on any S3-compatible store, using Athena's own CREATE EXTERNAL TABLE syntax. There is no Hive Metastore, no Glue API call, and no catalog service of any kind behind the table: the definition is a JSON manifest next to the data, shaped like a Glue TableInput. This post is what emulating Athena and Glue actually requires, which parts we copied deliberately, the four Athena behaviours we refused to copy, and the credential difference that is a real change from how SQE treats Iceberg.

Read

The log bucket that finally takes KMS

S3 server access logs could only land in an AES256 bucket for years, and AES256 is exactly what Security Hub flags. The escape was never a bucket setting. It was a KMS key policy that lets the logging service write with your key.

Read