Social media ban trial data reveals racial bias in age checking software: just how inaccurate is it?
A Guardian review of trial data for Australia’s upcoming teen social media ban finds age‑verification tools are less accurate for Indigenous and south‑east Asian users, often misclassifying them. The findings highlight potential discrimination and raise questions about the trial’s methodology and t…
Australia is set to enforce a ban that prevents users under 16 from accessing most social media platforms from December 2024. The ban relies on age‑assurance technology – a mix of facial‑age estimation, document verification and inference tools – to verify a user’s age before they can create an account. A recent analysis of the trial data, released by The Guardian, shows that the technology is likely to disadvantage Indigenous Australians and people of south‑east Asian heritage, misclassifying many young users as over the age limit and, conversely, older adults as under‑aged.
What the trial data reveals
The $6.5 million trial was run by the Age Check Certification Scheme (ACCS), a UK‑based body that tested three main approaches: age inference (using behavioural cues), age estimation (face‑scanning software) and age verification (document upload). Trials involved school students across the country, mystery‑shopper scenarios and automated image‑dataset testing. Publicly released data allowed independent researchers to compare accuracy across demographic groups.
- For participants identified as Indigenous, the age‑estimation software was seven percentage points less accurate than for respondents who selected “Australian” or “Oceania and Antarctica”.
- People with a south‑east Asian background saw a five‑point accuracy gap.
- Indigenous participants also experienced longer processing times before a result was returned.
- When document‑verification was tested on 328 mystery shoppers, three of the six Indigenous users received a false‑negative result – a 50 % error rate, albeit from a sample too small for statistical confidence.
Overall, the trial reported a 92 % accuracy rate for age estimation, lower than the 97 % figure quoted in the ACCS final report. The discrepancy stems from the report’s inclusion of unpublished lab tests that were not part of the anonymised public dataset.
Why the technology struggles with certain groups
Age‑estimation algorithms are trained on large image libraries that often under‑represent darker skin tones and facial features common among Indigenous peoples and many Asian communities. This lack of representative data leads to higher error rates for those groups. The ACCS acknowledged “known challenges in accuracy for under‑represented skin tones or facial features”, but the executive summary claimed the systems performed “broadly consistently across demographic groups”. The Guardian’s deeper dive contradicts that claim, showing measurable disparities.
Recent research from the United States and from leading age‑estimation vendors corroborates these findings, indicating higher error rates for women and people with darker skin. The ACCS did not publish gender‑specific accuracy data, citing advice from an independent ethics committee that detailed gender analysis could be misinterpreted.
Impact on Australian youths
False‑positive rates – where a user is incorrectly deemed over the age limit – ranged from 25 % to 73 % for children under 16. By contrast, adults aged 18 and over saw error rates under 5 %. Because the software struggles to pinpoint exact ages during puberty, the trial recommends a “buffer” of two to three years around the 16‑year threshold. This means that many 16‑ to 19‑year‑olds – an estimated 1.3 million Australians – will likely be asked for additional proof, such as a driver’s licence or passport, before they can join a platform.
Professor Tama Leaver of Curtin University warned that “the tools might be good enough to distinguish a 16‑year‑old from a 30‑year‑old, but they certainly are not good enough to distinguish between a 16‑year‑old and a 15‑year‑old”. The uncertainty could push schools, parents and community groups to seek alternative ways of verifying age, potentially widening the digital divide in remote and First Nations communities that already face limited internet access and credential availability.
Methodological concerns and transparency issues
The Guardian raised several questions about the trial’s methodology. Age‑verification testing involved only 328 participants, a sample size too small to draw robust conclusions about demographic performance. Two providers that consistently failed to meet the 16‑year threshold were excluded from the final results without explanation, raising concerns about selective reporting.
When pressed about the discrepancy between the 92 % public accuracy figure and the 97 % figure in the ACCS report, the scheme explained that the higher number incorporated “repeated lab tests” stored in the internal system of KJR, a consultancy contracted for part of the testing. Those lab results were not released publicly, limiting independent verification.
Professor Toby Walsh, an AI expert from the University of New South Wales who advised on the trial, noted that most adults will likely avoid inconvenience because the cascade of solutions – starting with inference, then estimation, and finally verification – will allow older users to rely on less intrusive methods. However, he cautioned that the cascade still depends on the underlying accuracy of each layer, which the current data suggests is uneven.
What comes next?
The eSafety Commissioner, the agency overseeing the ban, acknowledged the need for ongoing improvement. A spokesperson said that “consistent training and retraining of classifiers and facial age estimation are essential to better identify the broad range of ethnicities reflected in Australia”. The government has not yet announced whether the trial’s findings will trigger adjustments to the rollout schedule or prompt additional testing with more diverse datasets.
Minister for Communications Anika Wells was referred to the eSafety Commissioner for comment. As the December deadline approaches, stakeholders – including Indigenous advocacy groups, privacy organisations and tech companies – are urging the government to address the bias before the ban becomes law.
In the meantime, users and platform operators are advised to prepare for a multi‑step verification process. Platforms may need to integrate supplementary checks, such as parental consent mechanisms or community‑based verification, to ensure that the ban does not unintentionally marginalise already vulnerable groups.
Why it matters
The bias in age‑verification tools could disproportionately block Indigenous and Asian youths from social media, deepening digital inequality as Australia enforces its teen ban.
Key points
- Age‑estimation software is 5‑7 % less accurate for Indigenous and south‑east Asian users
- False‑positive rates for under‑16s range from 25 % to 73 %
- Document verification showed a 50 % error rate for Indigenous participants in a small sample
- Methodological gaps and undisclosed lab data limit confidence in the trial’s conclusions
- More than 1.3 million 16‑19‑year‑olds may need additional proof to access social media
Frequently asked questions
What is the Australian teen social media ban?
Starting in December 2024, Australian law will prohibit anyone under 16 from creating accounts on most mainstream social media platforms unless they can prove their age through approved verification tools.
Why are Indigenous and Asian youths more likely to be misidentified?
The age‑estimation algorithms were trained on datasets that under‑represent darker skin tones and certain facial features, leading to higher error rates for those groups.
Will adults be affected by the new age‑check system?
Adults are far less likely to be inconvenienced because the error rate for users aged 18 and over falls below 5 %, and many platforms can infer age from existing account data.
What steps can the government take to fix the bias?
Increasing the diversity of training data, publishing full methodological details, and conducting larger, statistically robust tests across demographic groups are recommended.
How reliable is document‑verification technology?
In the trial, it correctly identified 97 % of users overall, but the small sample showed a 50 % failure rate for Indigenous participants, indicating reliability concerns for certain groups.
