The role of AI Engineer and Applied Data Scientist has undergone a remarkable transformation in the last year. As someone who’s been in the thick of it, I’d like to share my observations on how it has evolved and where it might be heading.

The Era of Prompt Engineering and Chatbots

In 2023, the focus was primarily on developing chat-based solutions. The typical interaction between human and AI was straightforward: question and answer, or call and response. This interaction pattern typically looked like this:

1. User task
2. Assistant answer
3. User task
4. Assistant answer
5. (and so on)

Applied Data Scientists and AI Engineers alike spent a lot of time learning the fickle art of prompt engineering, monitoring for hallucinations, and adjusting parameters like temperature for optimal performance.

Companies felt the immediate need to adopt AI, either from pure excitement on the competitive advantage it could yield or from a healthy level of encouragement from their executives and investors. But out-of-the-box models lacked the nuance and understanding of a company’s processes, domain knowledge, business rules, and documentation. Retrieval augmented generation (RAG) was introduced to solve for this gap and provide a way to keep information that the language model could use as context from going stale.

The role of an applied data scientist working with generative AI shifted from being focused on building custom models to learning how to extract the best performance from the newest state of the art technology.

The impact of Open-Source shift and the rise of action-driven AI

When competitive open-source models that could rival OpenAI’s GPT-3.5 started to emerge, it opened the floodgates for a flurry of possible technical developments. Suddenly, there was more flexibility and visibility for building tools that could advance the capabilities of the types of tasks language models were able to complete.

Model orchestration libraries like Semantic Kernel, Autogen, and LangChain started to catch on, and the role of the AI engineer expanded. More development skills, proficiency with object-oriented programming, and familiarity with how to scale AI solutions into business processes were necessary to take full advantage of using these developer tools.

The game really changed when AI started interacting with external systems. In 2022, the Modular Reasoning, Knowledge and Language (MRKL) system was introduced. This system was the first to combine language models, external knowledge sources, and discrete reasoning — giving way to more opportunities to build AI systems that can take action to affect the outside world.

But by 2023, we had more formalized tools like ChatGPT plugins, semantic functions, and other tools that could be called and utilized by language models. This opened up a whole new dimension of possibilities and shifted the role of the applied data scientist and AI Engineer to lean more development heavy. This meant that now they were responsible not only for the inner workings of the AI model, but also writing the code that enabled the model to interact with internal systems and perform real-world actions.

The Rise of Agentic AI

The inclusion of tool calling marked the lead from chat based systems to action based systems, which quickly developed into the rise of Agentic AI.

Agentic AI has provided new possibilities that extended the capabilities of language models, which in turn expanded the role of the AI Engineer. The complexity increased dramatically, moving from extracting the best outputs from one model to a team of models working together. With more options came more variability in design choices such as:

• How to construct agent teams (how many agents, what tasks are they responsible for, etc.)
• What conversation patterns they should follow
• How to enable them with the correct set of tools to effectively complete their tasks
• How to break down the tasks so that the agents are accurate and consistent in response

Usually, design choices of this caliber (how a system is architected) occur above the AI Engineer/Data Scientist level, and the complex design choices are handled by management or even senior management. But the amount of creative freedom necessary for creating a successful agent system has caused a downward shift in the amount of design liberties and responsibility for the engineer.

“The role of an Applied Data Scientist / AI Engineer is morphing into a unique blend of development and creative problem-solving. The creative thinking necessary to experiment, think critically, and engineer a scalable AI system team should change the way that companies look at hiring for their AI practice.“

Speculating on the future skills for Applied Data Scientists and AI Engineers

In my view, the future of AI engineering lies in our ability to adapt quickly and solve problems creatively. The most successful AI engineers will not be the ones that are best at development but those who can:

• Quickly grasp new topics and apply them
• Solve problems without a pre-defined roadmap
• Pivot quickly as new technologies arise
• Manage their time effectively

It’s an exciting time to be in this quickly evolving field — but the personal investment it will take to stay on top will not be for the faint of heart.