Catholic magazine, the Pillar, recently put a new Catholic artificial intelligence through the wringer to determine whether the chatbot could accurately answer theological questions. The staff of the Pillar pitted the chatbot Magisterium AI against a professor of theology and chancellor of the Fargo diocese Turing-Test-style to see whether the bot could accurately answer an array of questions ranging from the nature of baptism to the validity of non-Church marriages. The final score tally was not kind to the computer, and neither was the Pillar’s takeaway:
Look, the Magisterium AI is clearly “well-read,” — it has a lot of resources at its disposal, and is able to access them quickly. It sometimes does pretty well at regurgitating information from those resources.
But theology and canon law are complicated — and the machine makes a few rookie mistakes. Sometimes it confuses related concepts. Sometimes it says too much, and says it wrong. The machine is sometimes completely wrong, and sometimes its answers are decent — but if you ask us, it’s not ready for prime time.
That doesn’t mean it won’t be. AI technology is evolving fast. But for now, our recommendation is that when the questions matter — and on matters of faith, they usually matter — you’d be better off asking the human experts.
You can read the questions and answers here. I recommend reading through it if only so you’re that much more convinced by my argument that the supposed benefits of these digitized religious tools are far outweighed by their flaws. Religion is the purview of God and man alone; trusting code to answer complex theological questions with real implications for the human soul is a recipe for spiritual disaster.