AI for engineers
From writing software to building software that can think, reason and act: LLM basics, RAG, tools, evaluation and agents, with four projects and a 90-day plan.
12 min readBy @thebrokenapp (opens in a new tab)
If you already write software, you are further along than you think. You understand programming, APIs, databases, systems, debugging and architecture. None of that gets thrown away.
What you need is one new layer on top of it: building software that can interpret, decide and act, and that you can still trust in production. Call it AI-native engineering.
This is the order I’d learn it in, what to build at each step, and the mistake I see developers make most.
You are building a reliable system around an unreliable component.
Traditional software is largely deterministic:
Input → Code → OutputAn AI application adds a probabilistic step in the middle:
Input → Model → Probabilistic output Input → Model → Tools → Data → Reasoning → ActionYour job is not to call an LLM. Anyone can do that in ten lines. Your job is to decide what the model is trusted with, what it is given, what checks its output passes through, and what happens when it is wrong. That is architecture, and it is why engineers have the advantage.
Use it across the whole workflow, not just to write functions.
Start with your own work. ChatGPT, Claude, Gemini, GitHub Copilot, Cursor, Claude Code and Codex all do the job; what matters is using them for debugging, design, review and testing, not only for “write this function.”
Don’t paste a stack trace and ask what’s wrong. Give it the relevant code, the logs and stack trace, the expected behaviour, recent changes and enough architecture to place the failure. Then ask for a process:
Here is the failing code, the last 50 log lines, the stack trace and the diff from yesterday’s deploy. List possible causes, ranked by evidence. For each, tell me what I would see if it were true, and give me the smallest test that would confirm or rule it out. Do not propose a fix until I pick a cause.
Now the model behaves like a collaborator who reasons out loud, rather than a search box that guesses.
You don’t need to train a model. You do need this vocabulary.
These eight ideas show up in every AI codebase you will touch.
| Concept | What to understand |
|---|---|
| Tokens | What to understand:How models read and are billed for text. |
| Context window | What to understand:How much the model can consider at once. |
| System instructions | What to understand:How you steer behaviour before the user says anything. |
| Temperature | What to understand:How sampling changes how much output varies run to run. |
| Structured outputs | What to understand:Getting predictable JSON instead of free text. |
| Tool calling | What to understand:Letting the model request an external capability. |
| Multimodal input | What to understand:Working with images, audio and documents, not just text. |
| Streaming | What to understand:Returning output progressively as it forms. |
Not another chatbot. Something that produces data your code can use.
Don’t start with an agent. Start with something small that teaches the core loop. A customer writes:
“I’ve been charged twice for the same subscription and I can’t access my account.”
Your system should understand the request, identify the issue type, extract the details, judge urgency, search the support docs, draft a reply and route the ticket. The output looks like this:
Category: BillingIssue: Duplicate chargePriority: HighCustomer ID: 12345Suggested response: ...In one small project you have used an LLM API, structured output, prompt design, validation and ordinary application logic. That is a far better first project than a generic chat window.
Models produce unstructured text by nature. Your software needs something it can rely on, so ask for a shape and validate it:
{ "category": "billing", "priority": "high", "sentiment": "frustrated", "requires_human": true}Then normal code takes over: routing rules, database writes, notifications.
Retrieval-augmented generation: give the model knowledge it was never trained on.
Sooner or later your app must know things that aren’t in the model, such as your docs, your policies, your data. The basic shape:
User asks a question→ Search for relevant information→ Retrieve the useful documents→ Send them to the model as context→ Generate the answerA company has HR policies, engineering docs, product manuals, architecture documents, incident reports and internal FAQs. Employees dig through hundreds of pages. Build “ask our company anything.”
Someone asks, “What is the process for getting production database access?” The system understands the question, retrieves the relevant documents, writes an answer, shows citations and links back to the originals. Companies pay for exactly this.
| Learn | Why it matters |
|---|---|
| Chunking | Why it matters:How you split documents decides what can be found. |
| Embeddings | Why it matters:How meaning becomes something searchable. |
| Retrieval | Why it matters:How the right passages are found. |
| Reranking | Why it matters:Reordering results so the best ones lead. |
| Metadata filtering | Why it matters:Restricting by department, permission or document type. |
| Citations | Why it matters:Showing exactly where an answer came from. |
| Evaluation | Why it matters:Measuring whether retrieval and answers are actually good. |
This is where a prototype becomes a system.
From “the AI answers” to “the AI can do something.”
A customer asks, “Why was my last payment rejected?” The AI could:
The model never touches your systems directly. You hand it a small set of controlled tools:
get_customer()get_transactions()search_documentation()create_support_ticket()The model decides which tool is useful. Your application decides whether to run it, with what permissions, and what comes back.
Build one that can do three things:
Then add the parts that make it real: permission controls, logging, and human approval for sensitive actions.
The most underrated skill in AI engineering.
Traditional tests compare expected output to actual output. Your support assistant just answered 1,000 questions. Was it good? You can’t tell by reading a handful, which is why you need evaluation.
| Measure | The question it answers |
|---|---|
| Retrieval quality | The question it answers:Did we fetch the right information? |
| Answer correctness | The question it answers:Was the answer right? |
| Groundedness | The question it answers:Did it stick to what was retrieved? |
| Tool accuracy | The question it answers:Did it pick the correct tool? |
| Safety | The question it answers:Did it avoid actions it shouldn’t take? |
| Cost | The question it answers:What does one interaction cost? |
| Latency | The question it answers:How long does the user wait? |
Take 100 real questions. For each, write down the expected information, an acceptable answer, the relevant documents and the tool calls it should make. Run the system. Change the prompt, the retrieval or the model. Run it again.
Now you are doing engineering, not vibe-checking a chatbot. I’ve put this before agents on purpose: an agent you can’t measure is one you can’t safely improve.
More than a chatbot: a loop that pursues a goal.
A simplified agent loop:
Goal→ Reason→ Choose a tool→ Execute it→ Observe the result→ Reason again→ Take the next action→ FinishAsk one, “Find out why our enterprise customer churned last quarter.” It queries customer data, reads support tickets, analyses usage, searches account notes, spots patterns and writes a report. The impressive part isn’t that a model wrote a report. It’s that the system carried out the investigation.
Everything that happens after the demo works.
| Concern | The question to answer |
|---|---|
| Reliability | The question to answer:What happens when the model fails? |
| Observability | The question to answer:What did it see, what did it produce, which tools did it call? |
| Security | The question to answer:What if a user tries to manipulate the system? |
| Permissions | The question to answer:What is it actually allowed to access? |
| Cost | The question to answer:Which model should handle which task? |
| Latency | The question to answer:Can it answer quickly enough? |
| Human in the loop | The question to answer:When must a person approve an action? |
Give a system read access to your company database, product analytics, customer feedback, support tickets and documentation. Then ask:
Why did customer retention decrease last month?
It queries retention metrics, compares cohorts, reads support tickets, analyses feedback, finds correlated changes, retrieves the release notes, forms a hypothesis and shows the evidence behind it. A human analyst still makes the final call. The time to get there drops dramatically.
After you’ve shipped a few things, not before.
Once you’ve built several applications, the theory starts to have somewhere to land.
That’s the point to decide whether you want to go deeper into AI/ML engineering or stay in AI application engineering.
| Must learn, in this order | Learn later |
|---|---|
| LLM APIs | Learn later:Fine-tuning |
| Structured outputs | Learn later:Model training |
| Tool calling | Learn later:Deep learning internals |
| RAG | Learn later:Inference optimisation |
| Evaluation | Learn later:Distributed model serving |
| Agents | Learn later: |
| Production AI | Learn later: |
The right-hand column moves up only if your specific career goal needs it earlier.
Use AI throughout your development workflow, and ship Project 1: an app built on an LLM API with structured output.
Ship Project 2, a document or knowledge system with citations. Then Project 3, an application where the AI can use at least three tools.
Pick a domain: healthcare, finance, sales, customer support, developer tooling, enterprise knowledge, education, legal or operations. Build something people can actually use. Then add evaluation, authentication, permissions, logging, error handling, cost monitoring and human approval.
It’s easy to think: “I know how to build software, I’ll just add an LLM.”
That isn’t enough. AI changes the architecture. You now design for probabilistic behaviour, context, evaluation, retrieval, tool use, permissions, human oversight and failure modes.
The best AI engineers aren’t the people who know how to call an LLM API. They understand where probabilistic intelligence belongs inside a software system, and where it doesn’t.
You don’t have to pick now. Build first; the direction gets clearer.
Traditional software asked, “What should the code do?” AI-native software asks, “What should the system be able to understand, decide and do?” Then you choose which part belongs to code, which to AI, which to tools, which to data and which to humans.
That is why this isn’t about learning another framework. It’s a new way of building software.