Theory of Mind 🧠

Almost every time I get into a discussion about AI outside of the industry, it seems that most have not experienced the theory-of-mind (ToM) abilities of these language models. Many presume that AI has no abilities in that area. ChatGPT only explores a one-on-one scenario of user and assistant, and only when you start using roles and context do its ToM capabilities become more apparent.

I discovered that GPT-4 had this capability through the 'Sparks of AGI' paper. It specifically had a test for this. It was one of the dozen things in that paper that amazed me - and the rest of the industry and computer scientists who realize how different this technology is. This test goes beyond simple next-word prediction.

Prompt Engineering Notes:

Read the part of the full research paper about faux pas. This type of information - gaps in LLM cognitive abilities - is important to know when constructing prompts.

I’m curious if the latest frontier model - as of this writing - Claude Sonnet 3.5 - would do at these tests.

Note that at some point the models seem to fill in these gaps through emergent behaviors that are filled in through an ever expanding and higher fidelity world model.