Sycophancy, which refers to when an AI chatbot gives overly agreeable responses that closely mirror the user’s views

Sycophancy stems from the way many AI models are trained; being seen as helpful or friendly is often rewarded and disagreement is avoided, even if that means generating harmful outputs. While this can make the chatbot seem more pleasant, it may also reduce accuracy, increase risk and limit the chatbot’s ability to challenge unsafe or incorrect ideas.