OpenAI introduces IndQA, designed to evaluate AI systems on Indian culture, languages

Bengaluru, Nov 4 (PTI) OpenAI on Tuesday introduced IndQA, a benchmark dataset designed to measure how well AI models understand and reason about questions on Indian culture, languages and context.

IndQA evaluates knowledge and reasoning about Indian culture and everyday life in Indian languages, it said.

According to OpenAI, IndQA includes 2,278 questions across 12 languages and 10 cultural domains, created in partnership with 261 domain experts from across India.

Unlike traditional benchmarks, IndQA's questions are natively written, not translated, reflecting the nuances of how people in India actually think, speak, and ask questions, it said.

"At OpenAI, we believe AI should be useful for all of humanity. That means AI must understand local cultures, languages, histories and contexts -- not just the Western world. India is a country of immense diversity, with many languages, traditions, and cultural nuances. For AI to be truly valuable here, it must understand that richness," Srinivas Narayanan, CTO, B2B Applications, OpenAI told reporters here.

Announcing the launch of IndQA, Narayanan said, it has been created with a curated dataset that captures India's cultural and historical context.

"This dataset helps our models understand Indian nuances more deeply. The experts also provide evaluation rubrics, so we can measure how well the AI performs on culturally grounded questions. Our goal is to take this as a playbook and use it in other countries too. We have been working with India in this way," he said.

Narayanan added that IndQA also underscores OpenAI's growing commitment to the Indian ecosystem, where local developers, educators, and creators are shaping how AI is being adopted and built for the world.

According to OpenAI, IndQA covers a broad range of culturally relevant topics, such as architecture and design, arts and culture, everyday life, food and cuisine, history, law and ethics, literature and linguistics, media and entertainment, religion and spirituality, and sports and recreation--with items written natively in Bengali, English, Hindi, Hinglish, Kannada, Malayalam, Marathi, Odia, Tamil, Telugu, Gujarati, and Punjabi.

Each datapoint includes a culturally grounded prompt in an Indian language, an English translation for auditability, rubric criteria for grading, and an ideal answer that reflects expert expectations, it added. PTI AMP ADB