The only podcast our Chief Data Scientist, Jon Krohn, listens to is “Last Week in A.I.”, and on episode #130, he had the pleasure to co-host the show! Every episode surveys the previous week’s biggest A.I. news; the biggest story in this episode being (of course) LLaMA 2.
LLaMA 2 was just released by Meta. Here are the key details:
• Open-source
• Unlike the original LLaMA, can be used commercially (as long as you have fewer than 700 million monthly active users 😂)
• Comes in a range of sizes, from 7 billion (fits on a single GPU) to 70 billion model parameters.
• Offers ChatGPT-level performance on a broad range of natural-language benchmarks and is generally now the leading open-source LLM (except on tasks involving code or math).
• Has double the context window (4k tokens) of the original LLaMA.
• Estimated $25m was invested in it, including extensive safety and alignment testing (probably more extensive than any other open-source LLM).
• Uses a two-stage RLHF (reinforcement learning from human feedback) approach that is key to its outstanding generative capacity.
• A new method called “Ghost Attention” allows it to perform especially well in “multi-turn” (ongoing back and forth) conversation.
Other big news from the past week that we covered includes:
• Elon Musk’s launch of his own A.I. company, xAI
• WormGPT is the LLM of choice for cybercriminals
• LongLLaMA can handle contexts of 256k tokens
• A.I. mapping the “connectome” of the fruit fly’s brain
• A.I. designing new proteins that could transform medicine
Watch here: lastweekinai.com/e/130th/
