We pointed SQE's MCP endpoint at an Okta tenant and asked Claude Code to query a Glue Data Catalog as a real person. The result is the Atlassian-plugin experience for a lakehouse: no token to paste, no client secret, no service account. Okta issues the access token in the browser, SQE verifies it, and Apache Ranger decides which tables that person's groups may read. Three Okta settings decide whether it works at all, and two configuration slips look exactly like authorization failures without being one. This is what we set, what broke, and what the System Log told us.
A working catalogue of the access control SQE and Spark share through Apache Ranger: object grants at the catalog, five column masks, row filters, tags that span tables, four personas built from the same primitives, and the two places the engines still render differently. Everything here is a line from a run that finished 43 of 43, on an EU bank fixture rather than a toy table. Includes the reason there are two enforcement points rather than one, and what changed when we moved every privilege write onto an authenticated endpoint.
A complete, executable account of how SQE and Spark query the same Iceberg table under the same Apache Ranger policies: Polaris object grants, row filters, column masks, tags, precedence, writes, views, and the differences that remain.
Spark was connecting to our catalog as root, which meant every access-control policy we had written applied to exactly one engine. Fixing it took no engine code: Polaris already authorizes against the OIDC identity, so a per-user token was the whole change. Then we ran both enforcement tiers together for the first time and found they compose in an order nobody documents, with Spark's plugin answering before the catalog is asked. Along the way two of our own new tests passed for the wrong reason, the two engines turned out to disagree about which mask wins, and renaming a column silently removes its protection. Here is the full evaluation order, what we measured, and why Databricks and Snowflake do not have this class of problem.
Access control was the last subsystem whose end-to-end behaviour we checked by grepping CLI output. The denial test matched the string 'not found', which is also what a typo'd table name prints. We replaced that harness with twenty assertions on decoded Arrow values against a live Apache Ranger, and the exercise found a feature that had never once worked: every tag-based column mask had been restricting the column instead of masking it since the day we shipped it. Two of the new tests then passed for the wrong reason, and only mutation caught them. Here is what fine-grained access control in SQE actually does, and how we know.
Tag-based masking is two systems wearing one name: a rule that says what a tag means, and an association that says which columns carry it. We put the rule in Apache Ranger, where Spark can share it, and the association in the Iceberg table metadata, where it survives rename, replication, and federated catalogs that no policy store can see. Here is the split, the precedence contract, and the identity bug the tests now pin down.
A lakehouse rarely has one query engine. Spark writes the tables, SQE serves the interactive queries, and both touch the same Iceberg data. The usual failure is policy drift: each engine has its own access-control plugin, the same masking intent gets translated twice, and the two translations disagree. We avoided that by not translating twice. SQE and Spark read the same Apache Ranger hive service, so one policy written once enforces the same way in both. The SSN reads xxx-xx-1111 whichever engine ran the query.
SQE got a second access-control backend: write GRANT/REVOKE as Apache Ranger policy and let Polaris enforce it. The protocol took an afternoon. The identity model took the week, because Polaris federation does not work the way the token suggests.
SQE's fine-grained Ranger backend does row filters, column masks, role-conditional masking, and tags. The kicker: the same Ranger policy on the same Polaris catalog produces byte-exact identical masked output in SQE and in standard Apache Spark. We proved it live, and the first run failed for a reason no amount of reading could have predicted.
Snowflake gives you masking policies, row access policies, object tags, and a GRANT model. SQE gives you the same primitives on Apache Ranger and open Iceberg, enforced by plan rewrite and shared across engines. Here is the mapping, the one real edge SQE has, and the gaps we have not closed yet.