Security & k-anonymity
Denying a field is easy. The hard part is the set of ways a caller can learn a value without reading it. Eight such channels follow, then five bypasses that are not channels, then a path the query cannot compute — not a channel either, but the one request the tier used to answer with neither an answer nor a refusal — then the requests that, until 3.2.0 and 3.3.0, carried out a denied or untransformed value the gate could not see; each has a test that reproduces the attack and goes red if the control is removed.
1. Set operations reconstruct a denied field
Segment composes UNION, INTERSECT and EXCEPT. Where a field is deny-select but allow-where:
AllEmployees EXCEPT (AllEmployees WHERE Salary > 100000)returns exactly the people earning under 100k, by name, with the salary column never selected. The protected value is reconstructed from set membership.
Closed by: policy applies to every segment independently, and in the Strict tier a deny-select field is automatically deny-where inside a Segment.
2. Aggregates over singleton groups
SUM, MAX and MIN execute in SQL against the real values, before any transform can apply. GROUP BY Department with MAX(Salary) over a department of one returns that person exact salary.
Closed by two halves, and neither works alone:
- Aggregating a transformed field is denied by default — all six transform attributes, not masks alone — and opted into with
AllowAggregate = true. MinGroupSizesuppresses any group smaller than k. Groups below the floor are removed from the result.
[DwGeneralize(GeneralizeMode.Round, Step = 5000,
AllowAggregate = true, MinGroupSize = 5)]
public decimal Salary { get; set; }DwCaps.MinGroupSize defaults to 5. Writing MinGroupSize = 1 switches it off and it is off — in production, with nothing refused and nothing warned about. A deployment that wants singleton groups is entitled to them.The setting starts unset rather than at one, which is what makes both halves possible: IsMinGroupSizeSet tells a deliberate opt-out from a deployment that never heard of the control. Without that distinction, any check strict enough to catch the second would trap the first. A per-field MinGroupSize on any transform attribute raises the floor for that field; the effective floor is the largest in play.
new DwPolicyOptions() // floor of 5
new DwPolicyOptions { Caps = { MinGroupSize = 1 } } // no floor, and meant
new DwPolicyOptions { Caps = { MinGroupSize = 10 } } // stricterThe floor suppresses rows; it does not refuse the query. A summary whose every group is a singleton returns nothing.
3. TotalCount cardinality disclosure
ToList computes Count() on the pre-pagination query. Filtering Salary > 200000 and reading TotalCount counts the high earners without selecting anything.
This is inherent to permitting WHERE on a protected field. The control is [DwOperators] restricting the field to Equal and In, so a caller can confirm a value it already knows and cannot sweep for one it does not. A documented consequence, not a defect.
4. Sort plus paging is a binary search
Sorting by a masked field ranks the real values. Paging through a known set reveals relative magnitude, and combined with range filters it converges on exact values.
Closed by: startup validation warns when a field is transformed but still orderable, and [DwNoOrder] is the explicit fix. A warning rather than an error because there are models where the ordering is the point and the transform is cosmetic — the engine names the fix rather than deciding for you. A declared default order cannot reopen the channel: a field in [DwEntity(DefaultOrder = ...)] that the caller may not order by is left out of their query, and recorded in the trace.
Employee.Email: the value is transformed on output but the field can still be
sorted on, and sorting runs against the real value. Paging through it ranks the
true order. Add [DwNoOrder] unless that is intended.5. getQueryString leaks the generated SQL
Returning raw SQL exposes injected tenant predicates and the column names of denied fields. The Strict tier throws QueryStringDenied; the Convenience tier allows it, documented.
The trace a result carries names the same things — the fields a policy dropped, what sealed each one, and every injected predicate — and an API that serializes a result hands it over. The Strict tier therefore keeps it off the result unless IncludeTraceInResult is true; it stays on PolicyQueryable<T>.LastTrace, in-process.
6. A refusal tells a missing field from a denied one
A caller who may not read a column can still ask about it. When a name that matches nothing fails validation while a denied field is refused by the policy — naming the field and the attribute that sealed it — every guess is answered: this column does not exist, that one does and is hidden. Repeated, the probe lists the schema, the columns the caller may never read included.
Closed by: under the Strict tier, outside a dry run, a name that matches nothing is gated as a field denied for every feature, at the step where a denial is raised and after the same caps a real field passes, so it receives the code a [DwDenied] field receives in that clause — FieldDeniedForWhere … FieldDeniedForSegment. All six codes carry FieldPath "*" and no RuleId or SourceOrigin, and CapExceeded names no path either, so the two refusals are identical. The trace keeps the real path, and AuditRefusals writes every refused guess to the audit — a guess at a name that does not exist included, which no [DwAudit] could record. The Convenience tier still names the field, documented. See What a strict refusal says.
The strict tier closes the side doors too. Inside a Segment every field refusal is FieldDeniedForSegment, so a field denied for every clause but not for segments cannot answer by clause while a missing name answers for taking part. A name padded with dots or blank segments is normalized the way a real path is, so it cannot trip the navigation cap that a padded real field passes. MaxQueryCost is checked only after every field has passed its gate, so a field weighted by [DwCost] is refused as denied before its weight could set it apart from a name that does not exist. And MissingContextValue names neither the scope's column nor the context key it reads, which together describe how the rows are partitioned.
7. The audit cap answers differently for a real field
An audited field records an event per use, and the query is refused rather than the record dropped when DwCaps.MaxAuditEvents is reached. Until 3.3.0 that refusal carried CapExceeded and a SourceOrigin naming the cap, where a name matching nothing carried the ordinary field refusal and no origin. One guess per request therefore told a caller which names are real and audited — which is to say, exactly the fields [DwAudit] is put on, since an unknown name is never audited and never reaches the cap.
Closed by: under Strict, outside a dry run, the cap refuses with the clause's own field refusal — same code, FieldPath "*", no origin. The request still fails, so the buffer still fails closed, and the trace still records which refusal it really was.
8. Four refusals that named a field
A strict refusal names no field, so that a denied field, a misspelling and a field that does not exist cannot be told apart. Four refusals named one anyway: an ambiguous name said the caller's guess matched more than one field, and so at least one; an ambiguous grouping key reported the column behind the caller's alias and said its values are transformed; the refusal for a clause that cannot be transformed listed every transformed column on the type — masked, generalized, truncated or formatted; and a deployment with no hash salt or token vault named the masked field it could not write.
Closed by: under Strict, outside a dry run, an ambiguous name is refused exactly as an unknown name is. The grouping key, the hash salt and the token vault name the clause and carry no origin; the clause that cannot be transformed names the clause and keeps an origin, which names the method and what to call instead rather than any field. The trace keeps the real reason for the operator. Convenience and a dry run — the posture's switch or the caller's — are unchanged.
9 to 13. The five that are not channels
| Attack | Control |
|---|---|
| An unguarded DynamicWhere call on a type that requires a policy | [DwEntity(RequirePolicy = true)] throws PolicyRequired rather than returning rows. Only this library's own extension methods run the check, so plain EF Core or LINQ against the DbSet is not intercepted — the flag closes the hole in this API, not every route to the table. |
| An empty policy store | Attributes still enforce; an empty store never resolves to Allow |
Reading an audited field by sending no Selects | A use is what the request reads, not only what it spells out. Until 3.3.0 only a field the request named was recorded, so a caller who named none received every audited member of the row with nothing written down — one token past [DwAudit]. Every audited member a projection the caller did not name hands back is recorded for Select, one event per query rather than per row. |
| Read an audited member no path of the policy names: one only a subtype of the row's type declares, or one past the four segments the attribute walk reads, inside a row or a navigation returned whole | The gate records a use by path, before the query runs, and such a member has no path it could ask about, so it came back with nothing written down. Since 3.3.0 the outbound walk's second pass reports each one it meets and the terminal records it: one event per path per query, for Select, with Effect Mask where the member is transformed as well. At MaxAuditEvents it fails closed as the gate does and the rows are withheld. |
| Close the connection as the rows arrive, so the record of what was read is never written | The audit middleware drained a request's events with the request's own abort token, so a client that hung up cancelled the write that follows the response: the sink threw, the middleware logged it, and the events went with the context — an audited read with nothing written down, for the price of a socket. Since 3.3.0 the drain has a budget of its own, thirty seconds, which the caller cannot cancel and a hung sink cannot outlast. |
A path the query cannot compute
A member of a row's type is not always a value a database can produce. A shared kernel type carrying two columns and a getter over them gives Name.Ar and Name.En, which translate, and Name.IsEmpty, which does not. The policy has nothing to say about the third — [DwNoWhere] on Name matches that path and not the ones beneath it — so until 3.3.0 every check passed and EF Core threw. The caller got a five-hundred where the strict tier promises a refusal.
Such a path is now refused as an unknown name is: the clause's own code, FieldPath "*", so it cannot be told from a misspelling or from a field the caller may not use. It applies to every clause the database has to compute, and not to Selects, which EF Core evaluates on the client when it cannot translate it. It is refused only where the whole set of members a container can produce is known — an entity's own EF Core model, and the initializers of a projection composed before ApplyPolicy, including a member that projection copies from the entity. A projection that builds its rows any other way — an anonymous type, a constructor with arguments — says nothing about which member each value sets, so no member of such a row is refused here. Rows in memory, a framework member the provider translates such as Length or Year, anything beneath a column, the convenience tier and a dry run are all unchanged: the path is left alone, and behaves exactly as it does unguarded — which for rows in memory and a framework member means it runs and returns rows, and beneath a converted column means the provider decides. So is a query a provider in front of EF Core translates — LinqKit's AsExpandable(), DelegateDecompiler's Decompile(), or a host's own registered through ReplaceService<IAsyncQueryProvider, …> — since such a provider may rewrite what EF Core cannot, and the library cannot tell one that does from one that passes straight through. The test is EF Core's own provider type, from EF Core's own assembly.
The refusal raises no [DwAudit] event, for the reason an unknown name raises none: no field was read, and the refusal names none. AuditRefusals records it, and the trace carries the real path. A simulation is handed no source, so it cannot refuse such a path at all.
This closes no leak: the query failed, it did not answer. It removes a way of telling one member from another by the shape of the failure, and it keeps the tier's promise that a guarded request is answered or refused. The trace records the reason.
Denials the gate could not see
A denied field often sits on a type the query reaches through a member: a secret on each line of an order, a code inside a nested object. The denial holds on every path that reaches it, and it has to hold whether or not the caller names the member. Each request below carried a denied or untransformed value out, until 3.2.0 or, where the row says so, until 3.3.0. All are closed.
| Attack | Control |
|---|---|
Send no Selects, on a type whose only denied fields sit beneath a member | A guarded query synthesizes a projection whenever the denied value can reach the result, and narrows the member around it or leaves the member out. Only a simple field denied at the top of T used to synthesize one, so the whole row came back with the denied value in it: in a list or nested object of a row projected before ApplyPolicy, in a row held in memory, and in an entity's included, automatically included, lazily loaded or owned member — in both tiers. A denial beneath a navigation nothing loads never leaves the database, so it asks for nothing. See A request that sends no Selects. |
Send no Selects, on a type whose denied field holds no simple value: a blob, a list, an owned object, a JSON column | A field denied at the top of T asks for the projection whatever it holds. Such a field used to be passed over, so with nothing else denied the whole row came back with it. |
Name a navigation whose key, Id, is denied | Refused with FieldDeniedForSelect in both tiers. The core's typed projection adds the key of every nested node it builds, so the convenience tier used to narrow the key away and get it back. A navigation named through another, Main.Lead, now gates the key of Main as well, which the projection adds. |
Name a member typed IReadOnlyList<T>, or another collection the core does not unwrap, with a denied field beneath it | Refused with FieldDeniedForSelect in both tiers. The projection gate now reads collections the way the attribute walker does. It used to read them through a narrower list, found nothing beneath such a member, and returned every field, the denied ones included, in both tiers. |
Name a member that carries a denied field no path reaches: deeper than four segments, inside a framework generic such as Dictionary<string, T>, or, on an entity's navigation, in its owned chain or a converted column | Refused under Strict. Under Convenience it is narrowed where the core can narrow it and refused where it cannot. What the member carries is read from the source — from the EF Core model for an entity, so only what loads counts. The gate also reads the rules themselves, so a denied property with no setter and a rule on a path reached through a cycle are found beneath a named member too. |
Include a navigation from the root, then reach the rows through it — Select(o => o.Customer), SelectMany, Join — or hide a projection behind another Select | Every navigation counts as loaded on such a chain, since EF Core still applies includes named from the root to the entities it reaches, and the library cannot read which. The includes used to be read against the wrong root, so the denied value beneath them was returned. |
Let a lazy loader fill a navigation after the query: a loader delegate or ILazyLoader the constructor takes, kept in a field or a property of any name | Counts as loading every navigation, as EF Core's proxies and an injected ILazyLoader property already did. The model keeps no record of such a loader, so the navigation it filled came back with the denied value. |
| Declare the denied field on a subtype — a derived entity, a subclass, an interface's implementation — and read it through the base type: a query over the hierarchy's root, or a member declared as the base type | The subtypes are read too: the types the EF Core model derives for an entity, and for a projected or in-memory row every loaded subtype, an open generic one and an application's subclass of a framework class such as Exception included. Such rows are projected to T and such members narrowed to the declared type; a named one is refused under Strict. A projection constructing a subtype of T is read as it, and a rule on a subtype's field through a base-typed member is enforced. The policy used to read the declared type only. |
Put the [DwDenied] on an override, on a public member a subtype hides with new, or on a class's implementation of an interface member, and read the member through the base type or the interface, a variant instantiation of it included | The denial applies to the path for every row, in every clause. The attribute walker read the declaration it walked and the attributes above it, never an override, a hiding member or an implementation below, so the base path filtered, sorted, grouped and returned the value. |
Guard a query through a provider that wraps EF Core's, as LinqKit's AsExpandable or DelegateDecompiler's Decompile do | The query runs untracked. EF Core's AsNoTracking hands such a query back unchanged, so it tracked: the context filled in navigations it already held, the denied ones included, and a masked value became a pending change the next SaveChanges would write. The call now goes into the query itself. |
Under a "*" deny with exact allows, reach a path the walk never asks about: past four segments, around a cycle, a property with no setter | Such a path is denied, so a member holding one is narrowed, left out or refused. It used to resolve as allowed, so the member was returned whole, named or not. |
Name a path one segment beneath a denied member whose type the framework declares: Salary.Value on a decimal?, Secret.Length, Born.Year, Bag.Count, Lines.Count on an application's own collection class | Such a path takes the policy of the member it reads since 3.3.0: the deny effects per feature, the [DwOperators] restriction, the [DwCost] weight and the audited features, from whichever provider supplied them. No attribute can be placed there and no fragment named it, so it resolved as allowed: a [DwDenied] decimal? was filtered on, sorted by, grouped by with its values as the group keys, aggregated and handed back by a dynamic projection, under Strict. A transformed member gave its stored value the same way, an audited one was read with nothing recorded, and a weighted one cost the default. What is said to the caller about the member stays the member's: the alias, the required filter, the forced scope and the description. |
Raise Caps.MaxNavigationDepth above 4 and name a denied member five or more segments out | The attributes of the member at the end of such a path are read directly since 3.3.0. No fragment of the attribute walk, which stops at four segments, reached it, so the member was filtered on, grouped by and returned under Strict. Default configuration was never exposed to this one. |
| Read a masked member the policy names no path to: five segments down an included or in-memory graph, one only a subtype of the row's type declares, one on an object a dictionary holds, one on the far side of a cycle | The rows are walked by run-time type as well since 3.3.0, and a member that declares a transform and was not transformed along a named path is transformed by its own attributes, once. The outbound walk transformed along the named paths only, so each of these came back exactly as stored — at the default caps, under Strict, with no Selects, with the navigation named whole in Selects, and in a dynamic projection holding a real object. |
Compose SelectDynamic, Group, FilterDynamic or Summary on a type whose only transformed member sits where the policy names no path | Refused with TransformRequiresMaterialization since 3.3.0. The four hand back a query the library never sees materialized, and whether the type is transformed was read from the named paths alone, so such a type got its query and its rows exactly as stored — the same gap, one method call away from the terminals. The refusal asks what a row of the type can hold as well, and names the clause when it has no column to list. |
Declare a [DwForceWhere] on a type first met at the walk's fourth segment, then reach that type by a shorter path | The walk returned at its depth limit with the type still marked as being inside it, so the type read as a cycle wherever it was met again — and what a cycle leaves out, the forced scope, the [DwRequireWhere] and the [DwAlias], was left out of the shorter path. Which of two members was declared first decided whether a tenant scope applied. Since 3.3.0 all three apply on every path within four segments that is not around a cycle. |
Put the policed type in an application namespace that starts with System, such as SystemsCorp.Payroll | Policed. The walker read any namespace starting with System as the framework's and put no policy beneath its types, so a [DwDenied] field there was returned, filterable and sortable. Only System and the namespaces beneath it are the framework's now. |
object, a framework interface or a collection that is not generic, such as IEnumerable, ArrayList or an application's own, is opaque to the policy: it never asks for a projection, a synthesized projection over a projected row or rows in memory leaves it out, and naming it returns whatever it holds. A framework generic holding a policed type, such as Dictionary<string, LineDto>, has no paths beneath it: naming it is refused in both tiers where the core cannot narrow it, narrowed away under Convenience beneath a navigation, and a synthesized projection leaves it out. Hold such values in a list of the policed type instead. A member EF Core does not map is read as its type, since its getter can hand out what EF Core loaded; a getter that copies a denied column into a type with no denial is the application's to withhold.Selects naming the list returns every element, those the scope excludes included, as in every release; a synthesized projection leaves such a list out. Scope the elements where the row is built.Getting the posture right
- Use
DwTier.Strictunless you needgetQueryString. - Leave
IncludeTraceInResultunset underStrict. A serialized result carries the trace to the caller; read it fromLastTraceinstead. - Turn on
AuditRefusalsonce anIDwAuditSinkis registered, so a probe for hidden columns leaves a record. - Leave
MinGroupSizealone unless you have a reason; setting it to 1 is a decision, not a default. - Prefer
TokenizeoverHashwhere you can run a durable vault: neither hides equality, but only one of them can be undone by a leaked constant. - Give a durable token vault a key (3.3.0), and hold it where the store is not. Unkeyed, a mapping is stored under a plain digest of the value, and a tokenized column is nearly always drawn from a space small enough to hash whole — so a backup, a replica or a dump of the vault gives back every value in it, and with them the value behind every token ever issued. See Transforms.
- Run
DwPolicy.ValidateModel(...)at startup and treat its warnings as a checklist. - Put
[DwEntity(RequirePolicy = true)]on anything sensitive, so a DynamicWhere call that forgetsApplyPolicyfails loudly. - Prefer
[DwOperators]over allowing free filtering on a protected field. - Hold a policed type in a list, never in a dictionary, another framework generic or a member typed
object. The policy has no paths into any of them. - Scope a list's elements where the row is built. A forced scope on the element type filters the rows that hold the list, never the elements.
- Set
DwCaps.DefaultPageSizeif the API does not page for itself. It ships off, and the requestMaxPageSizenever bounded is the one that sent no page at all. - Keep
DwCaps.MaxConditionSetsnear the number of sets your clients really send. A set with no conditions passes every other cap, and every set adds a condition or a subquery to the statement a segment becomes.