Three neutral criteria and eleven years of home visits

Case in one sentence

For eleven years the Dutch student finance agency picked which students to investigate for grant fraud using an algorithm scoring age, education type, and distance to the parental home, and the three criteria — none of which records anyone’s origin — sent inspectors to the doors of students with a non-European migration background three times as often as to anyone else’s.

Case scope note

The system at the centre of this case is a hand-specified scoring rule, not a machine-learning model: three risk factors, combined into a risk figure and then a risk code. It is included in this library deliberately. Everything that made it fail — unjustified features, proxy variables standing in for a protected characteristic, no pre-deployment impact assessment, no evaluation after go-live, human reviewers amplifying rather than correcting the model — is the standard failure catalogue for deployed machine learning, arriving here without any machine learning at all. The regulator’s finding is written to apply to any government algorithm that selects people. Readers looking for a model architecture will not find one; readers looking for what an unaudited scoring rule does to a population over a decade will.

Executive summary

Students in the Netherlands who live away from their parents receive a larger grant than those who do not. From 2012, after a 2009 misuse scandal and parliamentary pressure, the Dienst Uitvoering Onderwijs (DUO) checked entitlement through a three-step process: an automated risk classification, a desk study applying exclusion grounds, and a home visit.¹

The risk profile used three factors. A vocational (mbo) course scored higher risk than a university one. A shorter distance between the student’s registered address and the parents’ address scored higher risk. A lower age scored higher risk.²

In the summer of 2023 the investigative outlet Investico, with NOS op 3, the Hoger Onderwijs Persbureau, Trouw, and De Groene Amsterdammer, reported that students with a migration background were being accused of fraud far more often than others. DUO suspended risk-based selection within days and moved to random sampling; the ministry commissioned PwC.³¹

PwC reported in January 2024, published on 1 March. Its finding was indirect discrimination, and its root cause was that neither the risk profile nor the exclusion grounds rested on any analysis: they had been built from practical experience and never validated. The government and DUO apologised.¹⁴

Algorithm Audit then repeated the analysis with origin statistics that PwC had never had, supplied by Statistics Netherlands for more than 300,000 grant recipients. Students with a non-European migration background were classified high risk 2.0 times as often, manually selected for a home visit 6.2 times as often, and were 3.0 times as likely to receive a home visit that turned up nothing.⁵

In November 2024 the data protection regulator went further than either. Having inspected the algorithm itself on site, the Autoriteit Persoonsgegevens found that the three criteria had been set on the basis of “experience and common sense” with no objective justification, that distinguishing between students on that basis was itself unlawful discrimination, and that the Minister of Education had breached the GDPR’s lawfulness principle.²

In April 2026 the state set aside €80 million to compensate roughly 25,000 people, including those who had in fact misused the grant — because the evidence against them had been unlawfully obtained.⁶⁷

Research question

What does an unjustified selection algorithm do to a population when nobody checks it, and how long does it take anyone to notice?

Organization and operating context

DUO is the executive agency of the Dutch Ministry of Education, Culture and Science responsible for funding institutions, paying student finance, and collecting tuition fees, and it holds supervisory duties alongside those.¹ The Minister of Education is the data controller; DUO operates.² That split matters to the outcome: the regulator’s finding of breach lands on the minister, not the agency.

The grant in question, the uitwonendenbeurs, pays more to students living away from the parental home. Misuse means claiming it while living at home. In 2015 the introduction of the Dutch loan system abolished the grant for higher-education students, so from 2015 to 2023 essentially only vocational students were eligible — and therefore only vocational students were checked.⁴⁵

The situation before AI

The control process was a political product before it was a technical one. Media coverage and parliamentary questions about grant misuse in the summer of 2009 produced a ministerial action plan; the tightened control process began in 2012.¹⁴

PwC found no risk analysis underlying it. There is no document identifying, analysing, and prioritising the risks of grant misuse, and none translating any such risk into the profile that was built. The process rested on practical experience and was never formally validated by an internal or external expert.¹ The regulator, having asked the same question of the people who built it, recorded the answer as “experience and common sense”.²

Nor was there an assessment of its effect on the people it would be pointed at. No Data Protection Impact Assessment was carried out, though the process carried a high privacy risk, and the government audit service found no retention activity — no deletion, destruction, or anonymisation of personal data — since 2011.¹

The AI intervention or event

Step one, the algorithm. Student data was loaded, the profile computed a risk figure from the three factors, and the figure became a risk code.² Education type, distance, and age. Nothing about origin: DUO does not process nationality or background data at all in this process, a fact both PwC and the regulator record.¹²

Step two, the desk study. Officials narrowed the scored population using exclusion grounds — students in student houses, students with a child, married or cohabiting students, students in care institutions.¹ This step is where the disparity multiplied, and it is where a small number of staff performed the analysis, the selection, and the decision with at most limited internal checking, no standard quality control, and, as PwC put it, no safeguards to guarantee objectivity.¹ Selection here was not reproducible.¹

Step three, the home visit. Roughly 26,800 students received one between 2012 and 2023.⁴ Where misuse was established, the student repaid up to three years of grant and often received a fine on top — thousands of euros. Nearly 10,000 students were labelled fraudsters from 2012 onward.³

The mechanism connecting three neutral-looking criteria to a discriminatory outcome is documented rather than hypothesised. Children of Turkish-Dutch and Moroccan-Dutch parents live on average much closer to the parental home than children of parents without a migration background — a CBS finding PwC cites.¹ Vocational courses are typically nearer home than university ones, which is precisely why mbo scored higher.⁴ Algorithm Audit’s proxy analysis confirmed the pair directly: mbo 1–2 and short distance to parents correlate strongly with a non-European migration background.⁵ Excluding student houses from visits pushed the other way, because student houses are disproportionately occupied by students without a migration background.¹

The stop. Since 21 June 2023, selection has been by random sample only.⁴

Outcomes and economics

Measured by Algorithm Audit against CBS origin data, comparing students with a non-European migration background to students of Dutch origin:⁵

The two within-band rows are the finding that matters most and is quoted least. They isolate the human step: among students the algorithm had scored identically, officials still selected those with a non-European migration background 5.5 times as often in the high-risk band and 1.8 times as often in the low-risk band. The algorithm supplied a skew; the people acting on it more than doubled it.

On accuracy, 69% of students taken forward for further investigation turned out not to have misused the grant, a proportion PwC found roughly constant from 2012 to 2023 — and which never prompted a change to how DUO searched and selected.¹ DUO’s own figure is consistent: of about 26,800 home visits, misuse was established in roughly a third.⁴

Two findings cut against the harshest reading. Algorithm Audit found no bias in the appeal process itself: appeals succeeded at the same rate regardless of origin.⁵ And DUO reports PwC finding that a higher chance of a home visit did not generally translate into a disproportionately higher chance of the grant being stopped.⁴

On money: €80 million set aside in April 2026. About 12,000 students receive €500 each, having been visited at home with no consequence; about 10,000 receive €2,000 each, having had a grant withdrawn or adjusted or another measure imposed. A further 4,000 approached during a pilot phase must come forward themselves, because DUO cannot identify them. Those who believe they are owed more can seek a bespoke assessment, at the risk of receiving less.⁶

No source gives the cost of building or running the control process, or the value of the misuse it recovered.

Causal assessment and competing explanations

This case is labelled plausibly-causal, not causal, and the distinction is worth defending because the temptation to write “the algorithm caused this” is strong.

What is established well past dispute is the disparity: measured at every step, against actual origin statistics rather than proxies, across five reference years and more than 300,000 people. What is established as mechanism is also strong: the proxy analysis names which criteria carry the correlation, and an unsupervised bias detection method reaches the same result.⁵

What is missing is a counterfactual. Nobody ran the process without the profile and compared. The 2023 switch to random sampling could in principle supply one, and Algorithm Audit says explicitly that the random sample is too small to measure the underlying rate independently of the process.⁵ So the honest formulation is that the profile is a well-evidenced and mechanistically explained cause of the skew, not that its contribution has been isolated from everything else.

Two competing explanations deserve to be taken seriously and then set aside.

“Students with a migration background really do misuse the grant more often, so the profile was finding real fraud.” This is the explanation the process itself was built to be unable to test. Because the selection was biased, the discovered-misuse population is biased too; Algorithm Audit states that the underlying rate cannot be determined from the available data and that the random sample is too small to settle it.⁵ The claim is not refuted — it is unmeasurable, which is itself a consequence of the design. The regulator’s position renders it beside the point in law: the criteria needed justification before use, and had none.²

“It was the officials, not the algorithm.” The within-band ratios show the manual step contributed a great deal. But Algorithm Audit found no indication that this came from individual officials’ personal prejudice, pointing instead to the work instructions and exclusion criteria — the exclusion of student housing and care facilities among them.⁵ PwC reaches the same place from the other direction, identifying self-confirmation risk in a manual selection layered on an unjustified profile.¹ The two steps are not rival explanations; the second amplified the first, and both came from the same absent analysis.

Failures, limitations, and governance

• Nothing justified the criteria. No risk analysis, no scientific or legal grounding, no statistical validation. The regulator’s phrase for their origin is “experience and common sense”.²
• Nothing checked the outputs. DUO never periodically evaluated the profile, so the disparity surfaced only after media reports, eleven years in.²
• The warnings existed and were removed. By February 2010, before the process went live, documentation showed the pre-selection mainly hit mbo students with non-Dutch-sounding surnames. It changed nothing. A handover document of 2 June 2010 listed as a risk that misuse “appears to occur more among minority groups. E.g. mbo/hbo students of Moroccan origin”; in the next version of the same document, circulated three weeks later, those sentences were gone. Neither observation reached the progress reports or the final report to parliament.¹
• A second warning in 2013 also went nowhere. Team minutes that year record “a certain prejudice” in the reports of external inspectors feeding into the process, and signals that the process was too strict.¹
• A 69% miss rate was stable for eleven years and changed nothing.¹ One reason officials gave in interviews for continuing to search the minority population was a desired preventive effect; PwC found no documentation prescribing that.¹
• The complaints channel could not surface the pattern. Objections could not be filed online or by email, only on paper, and were never digitised, so pattern analysis had to be done by hand and therefore mostly was not. A manual review after the 2023 coverage found two cases in eleven years that flagged profiling.¹
• Data protection basics were absent. No DPIA on a high-risk process, and no data retention activity since 2011 — a GDPR breach on its own terms.¹
• The people who could see it had no standing. 79–85% of appellants had a non-European migration background, and their appeals succeeded at the same rate as anyone else’s.⁵ The system was fair at the one step where the affected population was concentrated, and that fairness told nobody anything.

What this case demonstrates

1. A selection rule needs no protected characteristic to discriminate by one. Three criteria that record nobody’s origin produced a threefold disparity in unjustified home visits, through documented correlations any prior analysis would have surfaced.
2. Under the Dutch regulator’s reading, the failure to justify the criteria is itself the violation — independent of the disparity it produced. Selecting on unjustified grounds is unlawful whether or not anyone can show downstream harm.²
3. Human review amplified rather than corrected the model. Given students the algorithm scored identically, officials selected those with a non-European migration background 5.5 times as often.⁵ A human in the loop is a governance control only if someone measures what the human does.
4. Bias arrives through exclusions as much as through inclusions. Excluding student housing from home visits shaped the population as surely as the distance criterion did.¹
5. Precision failure and fairness failure are the same failure here. A process wrong 69% of the time was concentrating those errors on one group, and the stable miss rate was a visible signal that went unread for eleven years.¹
6. Internal warnings need a route out. Two clear signals, in 2010 and 2013, existed inside the organisation and reached neither the ministry’s reports to parliament nor any change to the model.¹
7. An appeals process can be perfectly fair and still useless as a detector. Equal success rates across origins told nobody that 79–85% of appellants shared one.⁵
8. Unlawful selection contaminates true positives. The Dutch state extended compensation to students who had in fact misused the grant, on the ground that the evidence against them was unlawfully obtained.⁷
9. Discovery came from journalism, not from governance. Every investigation in this case was commissioned after the story broke.³

What this case does not demonstrate

• It does not establish whether students with a migration background misuse the grant at a different rate. The data cannot answer it and the random sample is too small.⁵
• It does not show that any official acted with intent. Algorithm Audit found no indication of personal prejudice and attributes the manual-selection skew to work instructions and procedure.⁵
• It does not isolate the profile’s contribution from the rest of the process. There is no counterfactual arm; hence plausibly-causal.
• It does not quantify harm to individuals. PwC declines to judge damage, saying that would require a file-by-file legal analysis outside its scope.¹
• It does not show that the disparity in visits produced a matching disparity in grants being stopped. DUO reports PwC finding it generally did not.⁴
• It does not describe machine learning, and nothing here should be read as evidence about what learned models do.
• It does not tell you whether the replacement works. Selection has been random since June 2023 and a new system is being designed; no evaluation of either is in evidence.⁴
• It does not confirm that compensation has been paid. The April 2026 sources describe a scheme and an allocation, not disbursement.⁶

Evidence assessment

Grade A — four independent chains, three of them with direct access to the system or its records.

The regulator inspected the algorithm itself, over three days on site, and questioned both the staff who ran the checks and the staff who chose who was visited.² PwC read eleven years of internal documentation and minutes and interviewed the people who built the process.¹ Algorithm Audit measured the outcome against origin statistics from the national statistics office covering more than 300,000 people, and published its code.⁵ Investico’s journalism is the reason any of the others exist.³ These are genuinely separate chains: different access, different methods, different institutional interests, converging on the same account of what the profile was and what it did.

The convergence is not total, and the disagreements are the most interesting thing in the record.

PwC and the regulator disagree about what kind of discrimination this was. PwC found indirect discrimination and stated explicitly that it was not ruling on lawfulness.¹ The regulator held that distinguishing between students on three criteria with no objective justification qualifies as direct discrimination, and that the resulting overrepresentation of students with a non-European migration background is the indirect discrimination on top.² Both are recorded here; this case does not pick a winner, because the disagreement is legal rather than factual and no court has resolved it.

The regulator’s report is internally inconsistent twice. Its summary calls the distinction “direct discrimination” while its conclusions section says “(direct or indirect)”. Its summary cites GDPR Article 5(2)(a) — accountability — while its conclusions cite Article 5(1)(a) — lawfulness. The substance of the finding is a lawfulness breach and this case reports it as such, but the citation should not be relied on to the sub-paragraph.

The 86% that is not in the report. DUO’s own statement gives the share of appellants with a non-European migration background as 86%.⁸ The report it is summarising gives 79–85% across its four reference years.⁵ This case uses the report’s range. A press relay then compounded the error into “86% of the students screened”, which describes a different population entirely.⁶

Press restatement drifts from the report. Investico gives students with a migration background “three times the chance of a home check”; the report’s 3.0x figure is specifically the chance of a home visit that found nothing.³⁵ Every ratio in this case is cited to the report.

Conflicts are real and disclosed on every side. PwC was paid by the responsible ministry and disclaims liability to anyone else, and states on its own front page that it accepted the supplied information without auditing it.¹ Algorithm Audit was paid by DUO, and produced findings worse for DUO than the ministry’s own investigation had. DUO’s two statements are an organisation describing its own failure and are used here only for operational totals and for its own words. The regulator has an institutional interest in establishing a justification requirement for risk profiling and says so in the report.

Two gaps hold the record short of complete. The scoring weights and thresholds — what a risk figure actually was, and where the high-risk cut fell — appear in no published source; the regulator saw them and describes their direction, not their values. And the April 2026 compensation scheme reaches this case only through press reporting of a ministerial letter that could not be retrieved from this sandbox, which is why both 2026 sources are marked as relays and why every claim resting on them is labelled Supported rather than Verified.

Material claims

Direct quotations

Quotations from Dutch sources are given in the original, with a working English rendering in the attribution line.

“Elke verwerking die een onderscheid maakt dat niet objectief gerechtvaardigd kan worden, is discriminatoir en daarmee onrechtmatig.”

— Autoriteit Persoonsgegevens, stating the standard it then applies; in translation, “Every processing operation that makes a distinction which cannot be objectively justified is discriminatory and therefore unlawful.”² · locator: Samenvatting, “Inzet algoritmes door de overheid”, page 3

“De in het algoritme gehanteerde risicofactoren waren tot stand gekomen op basis van ‘ervaring en gezond verstand’.”

— Autoriteit Persoonsgegevens; in translation, “The risk factors used in the algorithm had come about on the basis of ‘experience and common sense’.”² · locator: Samenvatting, “Discriminatoire, onrechtmatige verwerking van persoonsgegevens”, page 4

“misbruik meer blijkt voor te komen bij minderheidsgroepen. Bijv. MBO/HBO-studenten met een Marokkaanse afkomst”

— A DUO handover document of 2 June 2010, quoted by PwC; the sentences are absent from the version circulated on 28 June 2010. In translation, “misuse appears to occur more among minority groups. E.g. mbo/hbo students of Moroccan origin”¹ · locator: management summary, footnote 34, page 12

“Deze kennis heeft noch tot nadere verkenning van het risicoprofiel noch tot aanpassing van het model geleid.”

— PwC, on what happened after February 2010; in translation, “This knowledge led neither to further examination of the risk profile nor to adjustment of the model.”¹ · locator: management summary, section 3, page 12

“Maar we zijn onvoldoende zorgvuldig geweest bij het opzetten en het onderhouden van het controleproces.”

— Harmen Harmsma, director-general of DUO, on the day of the apology; in translation, “But we were insufficiently careful in setting up and maintaining the control process.”⁴ · locator: press statement, opening quotation

“Whether students with a migration background also make unduly use of the college grant more frequently cannot be determined based on the available data.”

— Algorithm Audit, on the question the process made unanswerable⁵ · locator: Executive summary, Finding 4, page 9

“Dat zijn betreurenswaardige constateringen, die nogmaals duidelijk maken dat we het controleproces anders moeten doen en anders gaan doen.”

— DUO, on the follow-up research; in translation, “Those are regrettable findings, which once again make clear that we must do the control process differently and are going to do it differently.”⁸ · locator: press statement, “Controleproces anders”

Revision notes

• 2026-09-13 — Initial publication at Grade A, the library’s second. Four independent evidentiary chains with direct access: a regulator that inspected the algorithm on site, a government-commissioned document study, an independent bias analysis against national origin statistics, and the journalism that triggered all three. Labelled plausibly-causal rather than causal — the disparity and its mechanism are measured across five reference years, but no counterfactual arm exists and the post-2023 random sample is too small to supply one. Carries a scope note because the system is a hand-specified scoring rule, not machine learning; it is included because its failure catalogue is the standard one. Records four discrepancies rather than resolving them: PwC’s “indirect discrimination” against the regulator’s “direct”, two internal inconsistencies in the regulator’s own report, DUO’s 86% against the underlying report’s 79–85%, and press restatements that drift from the audited ratios. The April 2026 compensation scheme is cited to press relays and labelled Supported, because the ministerial letter behind it could not be retrieved from this sandbox. First public-sector case and first case outside the US, UK, and Sweden.