Happy New Year!
As we embark on a new year with renewed hope and vigour, I wanted to share my thoughts on two seminal accomplishments in the field of Artificial Intelligence in 2020 and what that means for businesses and startups.
GPT 3 (Generative Pre-trained Transformer Ver. 3) is by far the largest neural network ever built with 175 billion parameters. Building such a large model even though difficult and expensive establishes a new benchmark in the state of the art. Moreover, the results seem to bear out - GPT3 is extremely versatile. It is adept at creative writing, auto-completing pictures, answering questions on diverse fields and even simulating dialogue with fictional or famous characters (both alive and dead). Most AI that has come before has been narrow AI – i.e. good at performing just one task. Thus GPT3 has created a new high water mark on what AI is capable of and is in many ways progress towards more general AI.
However, as you might imagine, GPT3 is quite resource intensive from preparing the data to training the model (which was an estimated $12 million in just compute cost). Putting such a large model into production requires large servers with 350GB of RAM. If more businesses are to adopt AI they need a simpler and less expensive path to achieving it. For example if we can train AI models with a small dataset and still achieve high accuracy / efficacy, that would indeed democratize AI. Firms of all sizes would be able to build their custom models for their specific use cases. AI on the edge (e.g. in your phone) too needs to be small and efficient. Many startups are already attacking these problems. Thus, I am hopeful Efficient AI will become a practical reality in 2021.
AlphaFold 2 is an AI model built for the specific purpose of predicting how a protein would fold i.e. its physical shape. While biologists know the chemical formula of proteins, the physical structure of only a few thousand proteins is known. Given that the human body has over a 100K proteins, this is quite few. In biology, like in many other fields, shape determines function and thus understanding protein structure is critical. AlphaFold 2 predicts protein structure with over 90% accuracy. This opens up a whole new realm in medical science and offers the promise of more personalized and better treatments.
AlphaFold 2 is built by the DeepMind team that has been at the cutting edge of AI innovation for some time. Their innovations in RL (reinforcement learning) and GANS (generational adversarial networks) helped create AlphaGo Zero the world’s best Go player. While Alphafold 2 like AlphaGo Zero and unlike GPT3 is narrow i.e. for just one purpose, its superhuman abilities in that narrow arena gives me hope that we shall see solutions to seemingly intractable problems with AI. I urge founders of AI startups to attack problems which cannot be solved without AI. While such innovation is never easy, it is how significant progress is made.
2020 in many ways was a watershed year for not just AI but technology and digital transformation in every sector. This makes excited about the year ahead and all the smart founders we shall meet and have the opportunity to work with. Do reach out to me if you need a partner to support your quest of tackling an unsolved problem or making something new or exponentially better with AI.
Wishing you joy, success and vaccination in the year ahead!