2023 was peak AI; UK Supreme Court rules AI cannot be an inventor; Harvard says AI is changing oral health; Big Tech goes all in on AI startups; AI Santas are everywhere
America's AI safety plan faces a budget crunch; AI might require more energy than all EVs combined; OpenAI's approach to the superalignment problem; Marie Claire profiles Rumman Chowdhury
For those of us following the generative AI space, Christmas came early this year. Just in the last month, we’ve seen five companies announcing more powerful large language models, each interesting in their own way:
Anthropic with Claude 2.1
Inflection AI with Inflection-2
Google DeepMind with Gemini Ultra
Mistral AI with Mistral-8x7b
01.AI with Yi-34B
As these LLMs match GPT-4 in certain benchmarks, Hasan Chowdhury wanted to explore two questions after a very busy 2023 for generative AI:
Are there any methods to increase performance beyond GPT-4's capabilities
What are some of the interesting research areas that could complement today's traditional transformer architecture?
One common approach to boost performance for LLMs is to increase the number of parameters of the model. This means adding more neurons and connections to the network, which allows the model to capture more complex patterns in the data.
However, this comes at a computational cost, as larger models require more powerful hardware to train and run. This leads us to the second approach: adding more compute power to the training process. This can be done by upgrading to more capable hardware, such as AMD’s MI300X or NVIDIA’s H200 GPUs which have larger and faster memories, allowing the model to train faster and on larger datasets.
Thankfully, LLM performance has so far scaled well with increases in parameters and compute. But as the compute costs associated with training and inference become more prohibitive, there is a renewed effort to squeeze more performance out of smaller models. One promising approach is called Mixture of Experts (MoE). MoE is a technique that divides a large model into smaller, more specialized models. Think of MoEs as a Swiss-army knife AI - in a single system, you get a Nobel scientist, a poet laureate, a top financial analyst or a lawyer. By working together, the experts can help to improve the model's overall performance and robustness.
These approaches are not mutually exclusive, and researchers are often combining them to achieve even better performance.
But even with these advances, today's LLMs are still held back by their inability to plan or reason the same way animals can. So one particular area I’ll be following very closely in 2024 is the work on developing world models. World models are essentially internal representations of the world that allow an AI system to reason about and interact with its environment. When combined with world models, transformers could generate more realistic and coherent text, translate languages more accurately, perform complicated math problems, and even compose creative content like poems, code, and musical pieces.
Read more in this piece titled How close are we to peak AI? from Business Insider, which includes a summary of the points above from yours truly.
And speaking of Christmas, I want to introduce you to a very special friend: say hello to AI Santa from Synthesia!
And now, here are this week’s news!
❤️Computer loves
Our top news picks for the week - your essential reading from the world of AI
These six questions will dictate the future of generative AI [MIT Technology Review]
How AI Is Changing Oral Health [Harvard School of Dental Medicine]
Deloitte Is Looking to AI to Help Avoid Mass Layoffs in Future [Bloomberg]
How Microsoft’s multibillion-dollar alliance with OpenAI really works [FT]
From Microsoft to MIT MBA, the AI reeducation boot camp is coming for every worker and executive [CNBC]
Big Tech’s Year of Partnering Up With AI Startups [Bloomberg]
OpenAI’s Ilya Sutskever Has a Plan for Keeping Super-Intelligent AI in Check [Wired]
Can AI Help You Die? [Bloomberg]
AI cannot patent inventions, UK Supreme Court confirms [BBC]
Meet the AI Whisperer [Marie Claire]
⚙️Computer does
AI in the wild: how artificial intelligence is used across industry, from the internet, social media, and retail to transportation, healthcare, banking, and more
The US and its allies are tinkering with AI to help do some of the most complex naval operations [Business Insider]
Allen & Overy rolls out AI contract negotiation tool in challenge to legal industry [FT]
Dutch Fintech Bunq Offers AI Chat Tool After Data Hiring Spree [Bloomberg]
New Class of Antibiotics Discovered Using AI [Scientific American]
AI has helped us work at ‘unparalleled’ speed, says Randstad [CNBC]
Chatty robot helps seniors fight loneliness through AI companionship [Bloomberg]
Mystery of Raphael masterpiece may have been solved by Bradford-made AI [The Guardian]
AI used to deter deer from railway tracks over festive period [BBC]
Artificial intelligence is accelerating the recruitment process — but tech leaders warn of bias and other risks [Business Insider]
Autonomous trucks prove their mettle on historic Route 66 [Axios]
TomTom creates AI-based conversational assistant for vehicles with Microsoft [Reuters]
Hitch integrates generative AI into Immutable Miniverse NFTs [VentureBeat]
Ludo.ai introduces text-to-video generator tool for game devs [VentureBeat]
Patronus AI finds ‘alarming’ safety gaps in leading AI systems [VentureBeat]
Microsoft Copilot users can now turn any idea into a song using AI [VentureBeat]
AI beats human at a game of physical skill for the first time [The Telegraph]
GitHub execs say coders are rushing to use AI coding tool Copilot despite copyright concerns [Business Insider]
AI's reality road [Axios]
How Rizz Assistants and AI Matchmakers Are Transforming Dating [Time]
Breathing new life into fight to save the seas with artificial intelligence [Reuters]
Forvia aims to halve its R&D costs by 2028 thanks to AI [Reuters]
Expedia wants to use AI to cut Google out of its trip-planning business [The Verge]
AI is powering a revolution in policing, at the Olympics and beyond [Washington Post]
Why this news network is turning to AI-generated anchors [CNBC]
UC Berkeley’s transformer-based robot control system generalizes to unseen environments [VentureBeat]
Boots to introduce AI ‘personal shopper’ to boost web sales [The Telegraph]
Jaxon AI teams up with IBM watsonx in battle against AI hallucination [VentureBeat]
🧑🎓Computer learns
Interesting trends and developments from various AI fields, companies and people
Where Meta is going next with AI and AR [The Verge]
Stable Video Diffusion is now available through Stability AI API [VentureBeat]
Humanoid Robots With Artificial Intelligence Have a Long Way to Go [Bloomberg]
Midjourney V6 is here with in-image text and completely overhauled prompting [VentureBeat]
Seeking a Big Edge in A.I., South Korean Firms Think Smaller [New York Times]
Generation GPT: What Gen Z really thinks about ‘world-changing’ AI [Washington Post]
Google's new multimodal AI video generator VideoPoet looks incredible [VentureBeat]
An Anticipated Wave of AI Specialist Jobs Has Yet to Arrive [WSJ]
A.I. Chatbots, Hens and Humans Can All ‘Hallucinate’ [New York Times]
Apple's latest AI research could completely transform your iPhone [VentureBeat]
Stability AI announces paid membership for commercial use of its models [The Verge]
The Simulation by Fable open sources AI tool to power Westworlds of the future [VentureBeat]
Creators, porn stars turn to AI doppelgangers to keep fans entertained [Washington Post]
Musk’s AI Chatbot ‘Grok’ Is Weirdly Good After All [Bloomberg]
Google Gemini is not even as good as GPT-3.5 Turbo, researchers find [VentureBeat]
A skills-based talent strategy will be critical in an AI-driven world [Fortune]
AI’s top research conference morphed into a recruiting extravaganza, summing up a wild 2023 [Fortune]
Accenture chief says most companies not ready for AI rollout [FT]
Nvidia Staffers Warned CEO of Threat AI Would Pose to Minorities [Bloomberg]
New Jersey touts new East Coast AI hub at Princeton University [VentureBeat]
Salesforce markets itself as an AI company customers can trust. Customers aren't buying it yet. [Business Insider]
Keep reading with a 7-day free trial
Subscribe to Computerspeak by Alexandru Voica to keep reading this post and get 7 days of free access to the full post archives.