The Bias Variance Tradeoff and How it Shapes The LLMs of Today
Is low inductive bias essential for building general-purpose AI? Michael Zakhary · Follow Published in Towards Data Science · 6 min read · 6 days ago — Photo by BoliviaInteligente on Unsplash In today’s ML space, we find ourselves surrounded by these massive transformer models like chatGPT and BERT that give us unbeatable performance on just about any downstream task, with the caveat being the requirement of huge amounts of pre-training on upstream tasks first.