Shuyang Xiang

Disentangle features in complex Neural Network with superpositions Shuyang Xiang · Follow Published in Towards Data Science · 5 min read · Just now — Complex neural networks, such as Large Language Models (LLMs), suffer quite often from interpretability challenges. One of the most important reasons for such difficulty is superposition — a phenomenon of the neural network having fewer dimensions than the number of features it has to represent. For example, a toy LLM

When there are more features than model dimensions Shuyang Xiang · Follow Published in Towards Data Science · 6 min read · 13 hours ago — Introduction It would be ideal if the world of neural network represented a one-to-one relationship: each neuron activates on one and only one feature. In such a world, interpreting the model would be straightforward: this neuron fires for the dog ear feature, and that neuron fires for the wheel

A concrete case study Shuyang Xiang · Follow Published in Towards Data Science · 7 min read · 12 hours ago — “Like many other permutation-based interpretation methods, the Shapley value method suffers from inclusion of unrealistic data instances when features are correlated. To simulate that a feature value is missing from a coalition, we marginalize the feature. ..When features are dependent, then we might sample feature values that do not make sense for this

A step-by-step guide Shuyang Xiang · Follow Published in Towards Data Science · 6 min read · 7 hours ago — Image by author: assistant done with assistant api and streamlit OpenAI Assistant API OpenAI has recently introduced new features that showcase an agent-like architecture, such as the Assistant API. According to OpenAI: The Assistants API allows you to build AI assistants within your own applications. An Assistant has instructions and can leverage models, tools,

How we build a PINN for inviscid Burgers Equation with shock formulation Shuyang Xiang · Follow Published in Towards Data Science · 6 min read · 9 hours ago — PINN on Shock Waves Physics-informed neural networks (PINNs) are a special type of neural networks. They estimate solutions to partial differential equations by incorporating the governing physical laws of a given dataset into the learning process. An example of such an equation is the inviscid

Mechanistic Interpretability on prediction of repeated tokens Shuyang Xiang · Follow Published in Towards Data Science · 7 min read · 2 hours ago — The development of large-scale language models, especially ChatGPT, has left those who have experimented with it, myself included, astonished by its remarkable linguistic prowess and its ability to accomplish diverse tasks. However, many researchers, including myself, while marveling at its capabilities, also find themselves perplexed. Despite knowing the model’s architecture