New Dataset Shows AI Stumbles on Religious Heritage Without Sources

Nearly 100,000 questions about Indian temples and holy sites exposed a weak spot: AI answers poorly from memory, but much better when handed the source.

The facts

The authors built a set of nearly 100,000 questions about Indian temples and related religious sites, covering history, architecture and practice. They started from heritage sources, pulled related Wikipedia articles, and used AI to generate yes/no, factual and explanatory questions, then filtered them. Current AI models did poorly when they had to rely on their own knowledge, and much better when given the source passage.

Limit to keep in mind: the questions and answers were largely generated and filtered by AI from Wikipedia, not primary sources or full expert review. The set is English-only and centers on Hindu temples, so other traditions and languages are thinly covered.

Why it matters biblically

The pattern is the same for biblical history: an answer is only as good as the source behind it. “Prove all things; hold fast that which is good” (1 Thessalonians 5:21). When YahCast covers temples, holy sites or ancient religion, the source link comes first.

Full story

BharatSynthQA: A Context-Grounded Question Answering Dataset on Indian Temples and Religious Heritage
Riya Naik, Shivam Sharma, Swati Agarwal

Similar Posts