How Good Are We at Assessing the Trustworthiness of LLMs?

Chain-of-thought prompting has been proposed as a technique that would increase people's trust in instruction-tuned large language models such as ChatGPT-3.5 or LLaMA-2. We find, somewhat surprisingly, that while people prefer chain-of-thought explanations, such explanations only increase trust when they are not read, but decrease trust when they are read. Moreover, the question type is predictive of trust. Across these two psychological biases, much of people's trust in instruction-tuned large language models seems independent of the content of their responses.