Things the New ChatGPT ChatBot Is Good At and Terrible At

Getting behind the hype

Any of you who have been following Data Science and AI on Social Media over the past week will have seen no shortage of examples and comments about OpenAI’s new ChatGPT chatbot. Once you play around with it (which as a research release you can do for free here), you’ll see that there is some truth to the hype. The model has certainly moved up a step in its ability to mimic general conversation, provide useful information on many topics, put together a coherent argument and operate in many languages (including programming languages).

The hype takes two forms. First, we see people showing examples of what the chatbot can do, and some of these look genuinely impressive for sure. For example, this mind-bending article shows how the author managed to to get the bot to pretend to be a Linux terminal and ‘build a virtual machine, inside the Assistant chatbot, on the alt-internet, from a virtual machine, within ChatGPT’s imagination’.

Second we see people genuinely worried about the impact of this new technology on erstwhile human functions. Which jobs will this replace? How are teachers going to know if essays are written by a chatbot?

However, it’s important not to get too carried away with the hype, because in the end this is is not a…