IDEA Lab

We are the IDEA (Impact-Driven Evaluation of AI) Lab in NYU Computer Science, led by Dr. Emily Black. Our research develops methods to measure and mitigate harm AI systems, with a special focus on preventing harm in real-world.

Our research

Our work spans three connected areas:

1. Algorithmic fairness and AI safety
   We build tools, methodologies, and frameworks to measure and prevent harms from AI systems that could negatively impact society. This includes developing evaluation paradigms to identify harms, explainability tools to diagnose why harms arise, and methods for training and selecting models that are both accurate and minimize harms. While our techniques are general and can be applied to a wide range of application areas, we are often inspired by issues such as protecting fair access to high-stakes opportunities like jobs, credit, housing, and health in the age of AI; promoting high-quality and diverse LLM output; and understanding and mitigating instability and arbitrariness in AI systems.

   • Multilingual Language Models and Representation Harms
   • Generative Monoculture in Large Language Models
   • Towards Effective Discrimination Testing for Generative AI
   • Toward Operationalizing Pipeline-aware ML Fairness
   • FlipTest: Fairness Testing via Optimal Transport

2. AI and the law / AI governance
   Beyond creating methods to find and prevent harm technically, we study how effective incentive structures, or AI governance strategies, to prevent harm in practice. Towards this goal, we study how to interpret the law in the context of AI systems to understand what requirements companies and other institutions using AI systems in high-stakes may be subject to, how companies can comply with those requirements, and to what extent various AI policy and governance strategies have worked in practice.

   • Algorithmic UDAP
   • The Legal Duty to Search for Less Discriminatory Algorithms
   • D-Hacking

3. Learning from and auditing AI deployments on the ground
   We study how AI systems are actually used in practice and build methods to audit real deployments and their impacts. This work helps us understand what problems and harms are happening in the real world, which helps guide our technical and legal/governance research.

   • Resume-ing Control: (Mis)Perceptions of Agency Around GenAI Use in Recruiting Workflows
   • Measuring Fairness in the U.S. Tax System

Team

Emily Black	Falaah Arif Khan	Eliza Berman	Sajel Surati
Principal Investigator	PhD Student	PhD Student	PhD Student

Qihan Wang	Bella Chang	Hoon Cho	Chuhan Ku
M.S. Student	M.S. Student	M.S. Student	M.S. Student

Gordon Dai	Riley Stacy
B.A. Student	B.S. Student (Barnard)