Public Trust in Artificial Intelligence Starts With Institutional Reform
October 2022 Update: We recently commented on NIST’s latest draft of a framework to manage the risks of AI. To improve the framework before it is finalized in January 2023, we think NIST should adjust the framework in several ways, including to better center communities impacted by AI systems, interrogate the broader structures around AI systems, and provide guidance on decommissioning AI systems or considering non-AI alternatives when appropriate.
Gaining the public’s trust in artificial intelligence (AI) will take more than just setting technical standards. It will require that institutions using this technology first prove they are worthy of our trust. Earlier this year, Congress passed its most significant law to date on AI. As part of this law, Congress tasked the National Institute of Standards and Technology (NIST) with creating a framework to manage risks associated with the use of AI. While this is a critical first step towards AI accountability, technical standards alone can’t guarantee trustworthiness in AI development and use. That’s why last week we sent a letter responding to NIST’s draft proposal on methods to manage bias in AI.
AI might evoke science fiction to some, but it is already being deployed throughout our society in ways that directly impact rights and liberties. For example, AI is used to determine where police resources will be deployed and who will stay incarcerated or be released on bail. Oftentimes, it is introduced without anything close to a sufficient evaluation of whether it is replicating or even super-charging existing biases in our society. That’s why we’re glad to see that NIST is soliciting feedback on the issue of bias in AI. Still, we’re concerned that their focus might be too narrow.
AI bias takes many forms, as NIST’s own proposal makes clear by identifying more than 35 types of biases. While some AI bias is due to technical factors such as the choice of variables that are closely related to demographic characteristics — such as when a tool can encode race based on zip code — many are not technical. These non-technical biases include population bias, which occurs when the people contributing data differ from the general population; historical bias, which occurs when models are trained on past biased data; and systemic biases, like institutional racism and sexism. Given this array of biases, it makes sense that bias can’t be mitigated only through technological changes, especially when technical fields themselves still struggle with a severe lack of diversity.
NIST’s draft proposal places a lot of emphasis on technical techniques and not enough on the societal context in which AI tools are designed and used. For example, the publication recommends engaging with the community when developing AI, but this engagement appears limited to “experts” and “end users.” Notably absent from the discussion is outreach to those who will be directly impacted by the use of AI systems, such as people in pretrial detention and families in the child welfare system. Impacted community members may hold very different views on how AI should be designed in a particular context — and whether it should be used at all.
Further, the draft proposal frames managing bias as a way to cultivate public trust. In doing so, it puts the burden on the public to increase its trust rather than on institutions to prove they are creating trustworthy technologies. AI with well-documented bias is being used in some of this country’s least trusted institutions, from the criminal legal system to the banking system. While we commend NIST for seeking public input, the other institutions where AI is already in use have historically failed to respond to public opinion and critique. That needs to be fixed before the public can begin to trust the use of even more AI by these institutions.
It's also important to remember that algorithmic design choices aren’t just technical choices. The choices made when designing algorithms are essentially policy decisions, and they often go unexamined. For example, in the context of bail decisions, a practitioner designing a tool to predict who is at risk of failing to reappear in court is essentially acting as a member of the legal system when they choose the variables that will or won’t go into the model and set the boundaries between high and low risk. In doing so, they enforce policies in the coding choices they make.
Transparency is vital in algorithmic decision-making systems. Making this process truly transparent may require regular audits and ongoing impact assessments by neutral third parties, with the results made public. NIST should take an active role in setting clear standards for these audits and impact assessments. They should also articulate the safeguards entities must put in place for those affected by algorithmic decision-making, which could include effective appeals processes for impacted individuals and compensation when people have been misjudged by AI tools.
NIST has proposed mitigating AI bias through a process of gradual refinement at the pre-design, development, and deployment stages. However, this approach seems to accept the deployment of AI as inevitable and always the best solution to a problem. We encourage NIST to reframe its approach to reflect the possibility that an algorithm might need to be terminated at each of these stages if unacceptable biases, harms, or impacts arise. Moreover, we suggest NIST set clear standards for instances in which AI should either be dropped or not developed in the first place.
Finally, NIST should emphasize that data privacy violations, flawed or unethical collection methods, and data inaccuracy can all contribute to algorithmic bias. Examples of each abound, from the creation of facial recognition databases without people’s consent, to problematic predictive policing technologies based on historically biased data.
Reducing AI bias is a noble goal, but the public’s trust in AI and other technology-enabled decision-making tools can’t be earned through the release of technical standards alone. Earning the public’s trust will require the meaningful engagement of impacted communities, increased transparency, improvements in data quality, and, most importantly, institutional reform.
Learn More About the Issues on This Page
-
Press ReleaseDec 2024
Free Speech
National Security
ACLU and Partners Urge Supreme Court to Block TikTok BanACLU and Partners Urge Supreme Court to Block TikTok Ban
WASHINGTON — Today, the American Civil Liberties Union, the Electronic Frontier Foundation (EFF), and the Knight First Amendment Institute at Columbia University filed an amicus brief urging the Supreme Court to block the enforcement of a law that would effectively ban people in America from using TikTok as soon as January 19, 2025. Earlier this month, the U.S. Court of Appeals for the D.C. Circuit rejected TikTok’s challenge to the law. TikTok has asked the Supreme Court for a temporary injunction to prevent the app from being banned while the court considers whether to take the case, saying that unless the justices intervene, the law will “shutter one of America’s most popular speech platforms the day before a presidential inauguration.” “The Constitution imposes an extraordinarily high bar on this kind of mass censorship,”said Patrick Toomey, deputy director of ACLU’s National Security Project. “The Supreme Court should take up this important case and protect the rights of millions of Americans to freely express themselves and engage with others around the world.” The brief argues that the D.C. Circuit failed to fully address the law’s profound implications for the First Amendment rights of the 170 million Americans who use TikTok. While the lower court’s decision correctly recognized that the statute triggers First Amendment scrutiny, it barely addressed users’ First Amendment interests in speaking, sharing, and receiving information on the platform. The court also perplexingly attempted to cast the government’s ban on TikTok as a vindication of users’ First Amendment rights, which it is not. The rights groups also explain that the law was intended to suppress certain content and viewpoints that many legislators believe could be amplified on TikTok, including the risk of foreign “propaganda.” But under the First Amendment, the government must meet a very high bar to restrict speech based on concerns about its “motivating ideology” or “perspective,” and the government has not come close to meeting that bar here. “The government should not be able to restrict speech, especially to the extent here, based on guessing about the mere possibility of uncertain future harm,” said David Greene, civil liberties director at EFF. “The Supreme Court should put the TikTok ban on hold while it considers the DC Circuit’s erroneous ruling.” Finally, the brief underscores that the government can’t impose this type of sweeping ban unless it’s necessary to prevent extremely serious and imminent harm to national security. But the government has not provided evidence of impending harm, or evidence that banning TikTok is the only available way to address its concerns. As the brief explains, the D.C. Circuit improperly treated the government’s invocation of “national security” as a trump card and failed to hold the government to its burden. “Restricting citizens’ access to foreign media is a practice that has long been associated with repressive regimes, and we should be very wary of letting the practice take root here,” said Jameel Jaffer, executive director at the Knight First Amendment Institute at Columbia University. “It would do lasting damage to the First Amendment and our democracy if the Supreme Court let this ban go into effect even temporarily.” You can find the brief online here: https://www.supremecourt.gov/DocketPDF/24/24A587/335380/20241217144322392_24A587%20TikTok%20v%20Garland%20Amicus%20Brief%20pdfa.pdf -
Press ReleaseDec 2024
National Security
Religious Liberty
ACLU Statement on New White House Strategy to Counter IslamophobiaACLU Statement on New White House Strategy to Counter Islamophobia
WASHINGTON – The Biden administration today released the first-ever national strategy to counter Islamophobia and related forms of bias and discrimination, including hate against Arab, Sikh, and South Asian Americans. In advance of the strategy, American Civil Liberties Union and its partners had urged the administration to overhaul government programs that reflect anti-Muslim discrimination. In particular, we have called for urgent action to constrain governmental agencies from continuing to exercise their authorities and technology to wrongly surveil and investigate, watchlist, and question and detain Muslims at the border, as well as deny immigration benefits to people from Muslim-majority countries. While the White House raised expectations that many of these issues would be addressed, the final strategy ended up falling far short. The following is a statement from Hina Shamsi, director of the ACLU’s National Security Project: “While this strategy acknowledges discrimination and its harms, it does little to end them and is a squandered opportunity. For decades, American officials have invoked national security to pass laws and implement programs that disproportionately harm Muslims and people perceived to be Muslim. A serious anti-discrimination strategy would concretely address multiple bias-infused government practices that deny our communities equal participation in civic life and our democracy, like federal watchlisting, surveillance, and investigation. We’re profoundly frustrated that the administration didn’t take even the basic, overdue step of recognizing that anti-Muslim discrimination is uniquely normalized and embedded in government policies.” -
VirginiaDec 2024
National Security
Trabelsi v. Crawford, et al. – Lawsuit Challenging Unlawful Detention and Inhumane Treatment of Acquitted ManTrabelsi v. Crawford, et al. – Lawsuit Challenging Unlawful Detention and Inhumane Treatment of Acquitted Man
Our client, Nizar Trabelsi, is in the United States against his will. The federal government brought him here from Belgium more than a decade ago and charged him with terrorism-related crimes. At trial, the government’s case failed: a federal jury found Mr. Trabelsi not guilty. But instead of allowing Mr. Trabelsi to return to Belgium after his acquittal, the United States placed him in highly restrictive immigration detention and began an ongoing effort to force him to Tunisia, where he was born and where he will very likely be tortured. Mr. Trabelsi’s detention violates the Constitution, immigration law, and the extradition treaty between the United States and Belgium. Through this lawsuit, he seeks to return to Belgium, and he demands an immediate improvement of his detention conditions. -
Press ReleaseDec 2024
National Security
Immigrants' Rights
Federal Judge Rejects Acquitted Man’s Argument That His Immigration Detention Is UnlawfulFederal Judge Rejects Acquitted Man’s Argument That His Immigration Detention Is Unlawful
ALEXANDRIA, Va. — This week, the U.S. District Court for the Eastern District of Virginia ruled that the court does not have jurisdiction to decide whether the government can continue to detain Nizar Trabelsi, a Tunisian national who was brought to the United States against his will for prosecution and was cleared of all charges last year. The ruling means the United States will be able to continue to detain Mr. Trabelsi while it attempts to deport him to Tunisia, where he was born and where, as an immigration judge ruled earlier this year, he will likely face torture. The court also concluded that even if it did have jurisdiction over Mr. Trabelsi’s case, his claims challenging the government’s detention of him would fail. “Reviewing claims that the government is holding someone unlawfully is at the core of the judicial function, and we’re heartbroken for our client that the court got this so wrong,” said Brett Max Kaufman, senior staff attorney with the ACLU’s Center for Democracy. “The government’s arguments in defense of Mr. Trabelsi’s detention makes a hash of the Constitution, immigration laws, and its own extradition treaty.” In 2013, the United States forcibly extradited Mr. Trabelsi from Belgium to face criminal charges in the United States. In July 2023, after almost 10 years of highly restrictive pretrial detention, a federal jury cleared Mr. Trabelsi of all charges. But instead of releasing Mr. Trabelsi or returning him to Belgium, the U.S. transferred him to immigration detention, wrongly treating him as an applicant for admission and placing him in the deportation process. Over the years, the Belgian government has issued multiple formal diplomatic requests asking the U.S. to facilitate his return, but the U.S. has refused to send him back. Mr. Trabelsi filed a lawsuit challenging the government's authority to detain him. He also sought immediate improvements to his detention conditions. Those claims were not part of the court’s ruling today, and will continue to be litigated on a separate track. “Mr. Trabelsi only wants to return to Belgium after being illegally extradited to the United States, held for ten years, and then acquitted by a jury of any crimes,” said Nicole Hallett, clinical professor of law and director of the Immigrants’ Rights Clinic at the University of Chicago Law School. “He cannot be held indefinitely and we will continue to fight to make sure that justice ultimately prevails.” Mr. Trabelsi is represented by the American Civil Liberties Union, the University of Chicago Law School’s Immigrants’ Rights Clinic, the ACLU of Virginia, and Professor Jonathan Hafetz of Seton Hall Law School. The complaint in Trabelsi v. Crawford was filed against Jeffrey Crawford, warden of the Farmville Detention Center where Mr. Trabelsi is being held; Liana Castano, ICE field office director for the Washington Field Office; Alejandro Mayorkas, secretary of the Department of Homeland Security; and Merrick Garland, U.S. attorney general.