Web Picks (week of 20 February 2023)

Every so often, we find the most interesting data science links from around the web and collect them in Data Science Briefings, the DataMiningApps newsletter. Subscribe now for free if you want to be the first to get up to speed on interesting resources.

ChatGPT vs. Bing Chat vs. Google Bard

Further understanding GPT

  • We Found *An* Neuron in GPT-2
    “We started out with the question: How does GPT-2 know when to use the word an over a? The choice depends on whether the word that comes after starts with a vowel or not, but GPT-2 is only capable of predicting one word at a time.”
  • GPT in 60 Lines of NumPy
    “In this post, we’ll implement a GPT from scratch in just 60 lines of numpy. We’ll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text.”
  • Anomalous tokens: a mysterious failure mode for GPT
    “We have found a set of anomalous tokens which result in a previously undocumented failure mode for GPT-2 and GPT-3 models”
  • What Is ChatGPT Doing … and Why Does It Work?
    … as explained by Stephen Wolfram. Very long but very solid!

Are we in a bubble?

  • Are we racing toward AI catastrophe?
    “As tech giants like Microsoft and Google compete to capture the AI market, safety could be an afterthought.”
  • AI Looks Like a Bubble
    “Investors need to take a cold shower”
  • Big Data is Dead
    “The world in 2023 looks different from when the Big Data alarm bells started going off. The data cataclysm that had been predicted hasn’t come to pass. Data sizes may have gotten marginally larger, but hardware has gotten bigger at an even faster rate.”

And much more…