Building a real AI agent without frameworks

There are more agent frameworks that there are LLMs to choose from. Some of the frameworks help, while others just add clutter. The one thing they all have in common is that they abstract away the fundamentals of building agents behind their own implementations and interfaces.

Building with an agent framework can give you speed, power, reliability and more. It is in fact advised if you have enough experience and know what you're doing. The problem comes when the framework becomes your crutch. When you forget, or never even learn, the basic building blocks and core principles behind it. That's when you go from someone who knows how agents work to a one-trick pony who only knows how to use a framework.

So let's look behind the curtain, get to know the fundamentals and build ourselves a helpful agent.

The basics

An agent is just a wrapper around an LLM that allows the underlying model to make decisions and take actions (access info, manipulate data, control external systems, etc). There's not much more to it than that.

Everything else is additional layers for convenience and performance:

• Context engineering turns your agent from a jack-of-all-trades to a master-of-some
• Memory makes your agent more relevant and helpful
• Visibility and traceability allow you to monitor performance and spot issues
• Evals let you measure and track quality

These are necessary considerations if you're shipping agents in production and we will get to them in future posts. For now, we're focusing on the fundamentals of building our first agent.

Let's build a real support agent

We'll build an online support chat agent that can answer general queries, identify orders and escalate refund requests to an internal team via email.

We'll build it in Typescript and use Vercel's AI SDK to allow for easy swapping of the LLM. I'll use Gemini Flash for the code samples but can you slot in whatever provider and model you prefer.

import nodemailer from 'nodemailer';
import { render } from '@react-email/render';

export type EmailOptions = {
  to: string;
  subject: string;
  react: React.ReactElement;
};

const transporter = nodemailer.createTransport({
  host: process.env.SMTP_HOST,
  port: Number(process.env.SMTP_PORT) || 587,
  secure: false,
  requireTLS: true,
  auth: {
    user: process.env.SMTP_USER,
    pass: process.env.SMTP_PASS,
  },
});

export const sendEmail = async ({ to, subject, react }: EmailOptions) => {
  const html = await render(react);

  const options = {
    from: process.env.SMTP_USER,
    to,
    subject,
    html,
  };

  try {
    const info = await transporter.sendMail(options);
    console.log(`Email sent: ${info.messageId}`);
    return info;
  } catch (error) {
    console.error('Error sending email:', error);
    throw error;
  }
};

Adding an FAQ

Realtime information lookup

Escalation process

Wrapping it all together

Summary