Mastering Artificial Intelligence Systems

Chapter 1

Fundamentals and Language Mechanics

AI language models are a prediction engine, not a reasoning engine. This chapter connects the model lineage (perceptrons → attention) to the mechanics that drive text generation.

Perceptrons to Attention Architectures

Perceptron (linear classifier): weights + bias compute a linear score, then a threshold produces a class label.

Deep nets (representation learning): stacked layers learn features; still, the core output is a function of inputs.

Transformers / attention (sequence modeling): the model builds a weighted mixture of prior token representations to compute the next-token distribution.

Key operational point: an AI model does not “think.” It computes a statistical prediction of the next word (more precisely: the next token) given the tokens it has been shown.

Next-Word Prediction as the Core Objective

Most chat-style systems implement the same primitive:

• Input: token sequence x = [t1, t2,..., tn]

• Output: probabilities P(t_{n+1} | x)

• Generation: sample or decode the most likely next token, append, repeat.

Reference API shape (typical):

• messages[] → tokenized prompt

• model → transformer weights

• max_tokens → hard cap on generated tokens

Context Windows (Interaction Memory Capacity)

A context window is the maximum number of tokens the model can process in a single forward pass (prompt + generated tokens). When the prompt exceeds the window, older tokens are truncated according to the server/client policy.

Capacity accounting table

Component

Symbol

Implication

Prompt tokens

Tp