Anish Dubey

Optimization methods for LLM alignment Anish Dubey · Follow Published in Towards Data Science · 10 min read · 19 hours ago — Context Language models have demonstrated remarkable abilities in producing a wide range of compelling text based on prompts provided by users. However, defining what constitutes “good” text is challenging, as it often depends on personal preferences and the specific context. For instance, in storytelling, creativity is key; in crafting informative content, accuracy

Flash attention is power optimization transformer attention mechanism that provides 15% efficiency Anish Dubey · Follow Published in Towards Data Science · 7 min read · 13 hours ago — Photo by sander traa on Unsplash Flash attention is a power optimization transformer attention mechanism which provides 15% efficiency in terms of wall-clock speed with no approximation. Context Given transformer models are slow and memory hungry on long sequences (time and memory complexity is quadratic