The AI and Machine-Learning Stack

When people refer to “the AI,” they often compress an entire engineered system into one word. A useful AI application can include user experience, workflow context, application logic, orchestration, retrieval, model inference, numerical libraries, compute infrastructure, data governance, observability, and mathematical foundations.

The model matters. It is often the most visible part of the conversation. But it is not the whole system.

MAGPAI exists to make that distinction visible. Chapter 1 introduced the reason for building a tiny transparent AI system: engineers need to see what normally disappears behind an API call or a chat interface. Chapter 2 provides the map. It explains the AI and machine-learning stack as a layered system so that later MAGPAI chapters can descend into tokenization, vocabulary lookup, embeddings, tensors, neural-network behavior, business processing, and bounded orchestration without treating everything as model magic.

Read Chapter 1: Why Build a Tiny Transparent AI System?

AI technology Stack view which uses sixteen layers to locate the language layer where text becomes tokens, token IDs, vectors, tensors, and model input.

Why a layered mental model matters

Engineers need layers because systems fail in layers.

A poor answer may come from model behavior, missing context, incorrect business data, malformed prompts, weak retrieval, unclear policy, or flawed application logic. Slow response time may originate in the browser, network, orchestration code, retrieval layer, inference runtime, hardware, or a downstream service. A safety issue may belong to input validation, policy enforcement, tool permissions, data access, output handling, or model behavior.

Those are different problems. They have different owners, tests, observability signals, and remedies.

Without a layered model, teams tend to argue about “the AI” as if one component owns every behavior. That makes diagnosis vague. It also makes vendor evaluation weaker because every claim sounds like a model claim. A layered model helps a team ask better questions:

• Which layer owns this behavior?
• Is the result learned, retrieved, calculated, or formatted?
• What data crossed a boundary?
• Which component can be tested deterministically?
• Where was latency introduced?
• What can be observed?
• What permissions does the model or agent actually have?
• How can the model be replaced without rewriting the workflow?
• What is the fallback when a layer fails?

Those questions are not academic. They are the difference between a fluent demonstration and an auditable system.

Artificial intelligence, machine learning, and deep learning

The terms around AI are often used interchangeably, but they do not mean the same thing.

Artificial intelligence is the broadest idea: systems that perform tasks associated with intelligent behavior. Machine learning is a major family inside that space: systems that learn patterns from data rather than relying only on hand-written rules. Deep learning is a machine-learning approach based on neural networks with multiple layers. Transformer-based language models are a modern neural-network architecture used heavily in large language models.

A useful learning hierarchy is:

Artificial Intelligence
  -> Machine Learning
    -> Deep Learning
      -> Neural Networks
        -> Transformer-based language models

That hierarchy is a learning model, not a complete taxonomy of every AI technique. Rule-based systems, deterministic calculations, retrieval systems, data pipelines, security controls, and workflow software can all be part of an AI-enabled application even when they are not learned models. MAGPAI keeps that boundary explicit because a reliable business answer should not depend on a language model inventing business facts.

The MAGPAI sixteen-layer stack

The MAGPAI source material defines sixteen conceptual layers:

The source presentation groups those layers as:

1 - 4:     Core AI literacy
5 - 8:     Modern Deep Learning
9 - 12:   Enterprise AI systems
13 - 16: Advanced / Strategic AI

For a website reader, it is also useful to describe the stack in broader engineering bands:

• Foundations 
• Model and machine learning
• Application knowledge and tools
• AI application systems
• Operations and governance.

Those bands are presentation aids. They do not replace the original layer names or order.

Experience, business intent, and accountability

An AI-enabled application starts before the model. A person has a goal, a workflow, a domain context, and an expected outcome.

In MAGPAI, the recurring question is:

Are MAG sales up in Chicago?

That is not just text. It carries a business metric, a company name, a location, a time comparison, and an expectation that the answer should be grounded in sales data. The user experience, chart, explanation, and decision context sit at the application layer. Business accountability also sits there. A model may help interpret language, but the application must decide what data is allowed, what calculation is valid, and what evidence is presented.

Application and orchestration

Application logic builds the path around model-facing behavior. It constructs prompts, manages state, calls tools, retrieves data, handles errors, enforces policy, formats output, and records observability signals.

In a MAGPAI-style flow, the application can turn a question into a structured request, read sample sales data, compare periods, generate a chart, and compose an explanation. Those steps should remain inspectable. Business rules and tool permissions should not be hidden inside a prompt and treated as if they were enforced.

This is where separation of concerns becomes concrete. The model interface is one boundary. The data contract is another. Tool access is another. The chart generator is another. Each boundary should have inputs, outputs, error behavior, and trace signals.

Model and inference layers

The language layer is where MAGPAI educational journey begins. Human text becomes normalized text, tokens, token IDs, vectors, tensors, and model input.

Tokenizers split text into model-facing units.
Vocabularies map those units to numeric identifiers.
Embedding layers turn identifiers into vectors.
Tensors arrange vectors into shapes that numerical operations can process.
Neural-network layers transform those representations.
Transformer layers use attention to refine representations across a sequence.
Large language models scale that architecture into fluent generation and reasoning-like behavior.

Chapter 2 does not try to explain every layer at implementation depth. It locates them so the later chapters can focus: Chapter 3 on tokenization, Chapter 4 on vocabulary and token IDs, Chapter 5 on vectors, Chapter 6 on learned embeddings, Chapter 7 on neural-network activations, and Chapter 8 on prompt tensors.

Numerical runtime, compute, and foundations

The software stack eventually becomes numerical computation. PyTorch and related tools provide tensor operations and neural-network building blocks in the MAGPAI repository. Under those tools are CPU and memory, and in larger systems often GPU acceleration, storage, networks, containers, service infrastructure, and monitoring systems.

Below the software is mathematics: linear algebra, calculus, probability, statistics, optimization, information theory, linguistics, systems engineering, and risk science. Readers do not need to become mathematicians to use AI tools. But the stack is easier to reason about when vectors, probabilities, tensors, attention, retrieval, and optimization are understood as engineering objects rather than magic words.

From intent to computation

The recurring MAG question moves through the stack as a series of transformations:

Business question
  -> User interface
  -> Application and orchestration logic
  -> Tokenizer and vocabulary
  -> Embeddings and tensors
  -> Model-like or deterministic interpretation
  -> Business data and calculations
  -> Chart and explanation
  -> User decision

Not every step belongs inside a machine-learning model. In fact, a reliable enterprise answer often improves when more of the critical path is explicit. Business data retrieval, sales comparison, policy enforcement, permissions, and chart generation should be governed by application code, data contracts, and observable tool calls.

MAGPAI makes that visible by using fictitious educational data. The point is not to publish a business analytics system. The point is to show how a human question becomes computable state and how a data-backed answer can be traced.

Boundaries between layers

Layer boundaries are interfaces. They define what crosses from one part of the system into another.

Good boundaries name their inputs and outputs. They use schemas where structure matters. They define model interfaces and runtime contracts. They make data ownership explicit. They surface errors instead of burying them. They expose observability signals that let a team reconstruct what happened.

Those boundaries also matter for security and privacy. If the model can request a tool, the application still controls whether the tool exists, what permissions it has, what data it can read, and what action it can take. Model intent is not an authorization model.

What changes and what remains stable

Some layers evolve faster than others.

Model providers can change. Tokenizers and model formats can change. Retrieval methods can change. Runtime libraries can change. Business rules, data ownership, compliance requirements, and user expectations may be more stable. Observability and safety controls should not disappear when a model changes.

That is why architecture should avoid coupling an enterprise workflow to one model implementation. The model boundary should be replaceable. The data contract should be governed. The audit path should remain visible. The fallback path should be designed before the failure.

The danger of collapsing the stack

Treating all AI behavior as “the model” creates avoidable errors.

Teams may attribute a data-quality issue to hallucination. They may put business rules only in prompts. They may assume generated output is a verified fact. They may fail to trace tool calls. They may hide deterministic transformations inside opaque orchestration. They may measure only total response time and miss the slow layer. They may allow application permissions to inherit model intent. They may treat a fluent answer as an auditable result.

The stack prevents that collapse. It gives teams a vocabulary for assigning behavior to the layer that owns it.

MAGPAI as a stack-oriented learning laboratory

MAGPAI is deliberately small. That is its value. It can expose the path that larger systems hide.

Chapter 1 explained why a tiny transparent system is useful. Chapter 2 provides the stack map. The following chapters move through the approved ten-chapter progression: tokenization; vocabulary and token IDs; vectors and embedding lookup; learned embeddings; neural-network activations; tensors and prompt representation; business data, calculation, and chart-backed results; and chatbot and agent orchestration.

Nothing beyond that ten-chapter series is announced here. The point is to keep the learning path focused and inspectable.

Engineering leadership implications

Technical leaders do not need to implement every layer from scratch. They do need enough depth to ask precise questions.

When an AI-enabled system succeeds, leaders should know which layers created the result. When it fails, they should know where to look first. When a vendor claims capability, they should know whether the claim belongs to the model, retrieval, orchestration, safety, operations, or user experience. When a team proposes an agent, leaders should ask what permissions it has, what tools it can call, what state it keeps, what data it reads, and how its behavior is observed.

That is why the stack matters. It turns a vague discussion about “the AI” into an engineering conversation about boundaries, evidence, ownership, and change.

Explore the Stack Explorer

The companion Praxis artifact turns the sixteen-layer MAGPAI stack into an accessible reader-facing explorer. It preserves the source layer order, maps the recurring MAG question across layers, and shows how the approved ten chapters relate to the stack.

Explore the MAGPAI Stack Explorer

Chapter 3, How an AI System Sees a Sentence, is coming next.

About the author

Vasili Katsoulis

Vasili Katsoulis believes the future belongs to engineering leaders who understand both timeless software engineering principles and modern AI. His work explores how technical depth, architectural thinking, and engineering judgment enable organizations to build better software in the age of AI.