SQE now speaks Model Context Protocol over Streamable HTTP, and it does it as an OAuth resource server: no service account, no token minting, no credential proxy. The caller's bearer becomes an SQE session and travels to Polaris and S3 unchanged. Read tools are gated on one scope, the read-only SQL tool is enforced by SQE's own parser rather than a keyword regex, and the write tool needs three independent authorizations that do not substitute for one another. The runnable quickstart proves it with 14 checks against live Keycloak, Polaris and RustFS, including the one that matters most: a reader holding the OAuth write scope is still denied by the source.
A working catalogue of the access control SQE and Spark share through Apache Ranger: object grants at the catalog, five column masks, row filters, tags that span tables, four personas built from the same primitives, and the two places the engines still render differently. Everything here is a line from a run that finished 43 of 43, on an EU bank fixture rather than a toy table. Includes the reason there are two enforcement points rather than one, and what changed when we moved every privilege write onto an authenticated endpoint.
A complete, executable account of how SQE and Spark query the same Iceberg table under the same Apache Ranger policies: Polaris object grants, row filters, column masks, tags, precedence, writes, views, and the differences that remain.
Spark was connecting to our catalog as root, which meant every access-control policy we had written applied to exactly one engine. Fixing it took no engine code: Polaris already authorizes against the OIDC identity, so a per-user token was the whole change. Then we ran both enforcement tiers together for the first time and found they compose in an order nobody documents, with Spark's plugin answering before the catalog is asked. Along the way two of our own new tests passed for the wrong reason, the two engines turned out to disagree about which mask wins, and renaming a column silently removes its protection. Here is the full evaluation order, what we measured, and why Databricks and Snowflake do not have this class of problem.
Access control was the last subsystem whose end-to-end behaviour we checked by grepping CLI output. The denial test matched the string 'not found', which is also what a typo'd table name prints. We replaced that harness with twenty assertions on decoded Arrow values against a live Apache Ranger, and the exercise found a feature that had never once worked: every tag-based column mask had been restricting the column instead of masking it since the day we shipped it. Two of the new tests then passed for the wrong reason, and only mutation caught them. Here is what fine-grained access control in SQE actually does, and how we know.
SQE's founding rule is no service account: every query runs as the authenticated user, tokens passed through to Polaris and S3. Background compaction has no user to be. Here is how we gave Iceberg tables a self-healing maintenance path while keeping that rule intact: a structurally isolated principal, deny-by-default grants, and a commit authority that never leaves the coordinator, even when the rewrite itself is distributed across the worker fleet.
Tag-based masking is two systems wearing one name: a rule that says what a tag means, and an association that says which columns carry it. We put the rule in Apache Ranger, where Spark can share it, and the association in the Iceberg table metadata, where it survives rename, replication, and federated catalogs that no policy store can see. Here is the split, the precedence contract, and the identity bug the tests now pin down.
A lakehouse rarely has one query engine. Spark writes the tables, SQE serves the interactive queries, and both touch the same Iceberg data. The usual failure is policy drift: each engine has its own access-control plugin, the same masking intent gets translated twice, and the two translations disagree. We avoided that by not translating twice. SQE and Spark read the same Apache Ranger hive service, so one policy written once enforces the same way in both. The SSN reads xxx-xx-1111 whichever engine ran the query.
S3 server access logs could only land in an AES256 bucket for years, and AES256 is exactly what Security Hub flags. The escape was never a bucket setting. It was a KMS key policy that lets the logging service write with your key.
SQE got a second access-control backend: write GRANT/REVOKE as Apache Ranger policy and let Polaris enforce it. The protocol took an afternoon. The identity model took the week, because Polaris federation does not work the way the token suggests.
SQE's fine-grained Ranger backend does row filters, column masks, role-conditional masking, and tags. The kicker: the same Ranger policy on the same Polaris catalog produces byte-exact identical masked output in SQE and in standard Apache Spark. We proved it live, and the first run failed for a reason no amount of reading could have predicted.
Snowflake gives you masking policies, row access policies, object tags, and a GRANT model. SQE gives you the same primitives on Apache Ranger and open Iceberg, enforced by plan rewrite and shared across engines. Here is the mapping, the one real edge SQE has, and the gaps we have not closed yet.
Our own README promised fine-grained Lake Formation in the SQE Glue quickstart. The engine does not do it. Here is what Lake Formation actually enforces when SQE reads S3 directly, and where SQE's real column and row masking lives.
We tested Renovate's registry behaviour for every ecosystem — npm, PyPI, Go, Cargo, Maven, NuGet, Composer — and found that minimumReleaseAge works for all of them, but how each datasource resolves its registry is surprisingly different. Here's what we found and how to build a complete defence.
A three-layer defense for developers and DevOps engineers: harden your dev machine in 5 minutes, lock down project configs for every ecosystem, and enforce policy org-wide with a proxy. Covers npm, pnpm, yarn, bun, pip, uv, Go, Maven, Gradle, NuGet, Cargo, and Composer — plus escrow, JFrog Curation, and Nexus.
Every time a developer runs npm install, they're implicitly trusting hundreds of strangers. Intercepting that trust is straightforward in theory — and surprisingly subtle in practice.
Two days after the nine-PR audit pass, we ran a bigger one. 130 issues filed, 19 themed MRs merged across four waves. Same workflow, more failure modes. Watchdog stalls, a reboot mid-wave, a broken main, and config.rs as the conflict magnet. Here is what actually happened.
Supply chain attacks on package registries are surging across every ecosystem. Two controls block the majority: a quarantine window and disabled install scripts. The exact config for npm, pip, Go, Maven, NuGet, Cargo, and more — plus a one-command proxy and shell tests you can run today.
Sunday afternoon: an audit dropped eighteen issues into the tracker. Monday morning we had nine merge requests open. Two of them hit conflicts on rebase. Neither one mattered. Here is why that was deliberate, not lucky.
Modern object-store abstractions unify the filesystem and HTTP behind a single URL. That's a feature for ergonomics. It's a security trap when the URL comes from a user. SQE shipped that trap and then closed it. The IMDS pivot is the part worth telling.
We ran a full production sign-off audit against SQE and found 43 issues across security, runtime safety, logic bugs, and code quality. Then we fixed all of them in one session.
Explore how AWS's new zonal shift capabilities and Security Incident Response features are transforming traditional disaster recovery. This technical deep-dive reveals how automatic failover, self-healing systems, and integrated security responses are making DR more reliable and cost-effective than ever before.