Topic 16.1
The AI Python Landscape: Which Library Does What
In one line
AI work in Python is a stack of layers: NumPy and pandas for data, scikit-learn for classic machine learning, PyTorch for neural networks, Hugging Face for trained models, SDKs for hosted LLMs, frameworks and FastAPI on top. Knowing what each layer is for tells you what to learn first.
Think of it like this
A kitchen. There is a pantry of raw ingredients (data), basic tools like knives and pans (NumPy and pandas), recipe books for common dishes (scikit-learn), a big professional oven you can build your own dishes in (PyTorch), a shop that sells ready-made meals (Hugging Face and hosted model APIs), and a restaurant front that serves customers (FastAPI). A cook who knows what each part is for can walk into any kitchen.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Library
- Code someone else wrote that you import and use.
- Machine learning (ML)
- Programs that learn a rule from example data instead of being given the rule.
- Neural network
- A large function made of many adjustable numbers, trained by nudging them to reduce mistakes.
- Tensor
- A grid of numbers with any number of dimensions. Like a NumPy array, but it can run on a GPU and track gradients.
- GPU
- A graphics card. It does huge amounts of simple arithmetic in parallel, which is what neural networks need.
- LLM
- Large language model: a neural network trained on huge amounts of text that predicts the next piece of text.
- SDK
- Software development kit: a library from a service's makers that wraps its web API in Python functions.
- Framework
- A library that gives you a ready-made structure for a kind of program, such as a RAG pipeline or an agent.
- Vector store
- A database specialised in finding the stored vectors (lists of numbers) closest to a query vector.
- Model
- A program whose behaviour comes from numbers it learned from example data, not from rules a person wrote.
- Weights
- The adjustable numbers inside a model. Training changes them. A big model has billions.
- Embedding
- A list of numbers that stands for the meaning of a piece of text, so that similar meanings get similar numbers (Topic 16.8).
Step by step
01The map in one picture
Read the diagram bottom to top. Everything stands on Python and NumPy. The two paths split after that: the model path goes through scikit-learn and PyTorch, the LLM path goes through SDKs and frameworks. Both end at serving.
02Each layer is just Python functions
No layer is magic. Whether it is a dot product or a call to a language model, from your code's point of view it is a function: values in, value out. The first example below lists the layers as data, which is also a good habit: describe a structure as data, then print or test it.
The snippet below is a tiny "model": one rule, y = w*x + b, where w and b are adjustable numbers (called weights in real models). Training means finding good values for them. A real model has billions of such numbers, but the idea is the same.
import numpy as np
x = np.array([1.0, 2.0, 3.0])
print(2.0 * x + 1.0) # a model is a function: y = w*x + b03Which layer for which job
Match the job to the layer. Cleaning a CSV is pandas. Predicting a number from a table of columns is scikit-learn. Training a custom image or text network is PyTorch. Answering a question in natural language is usually a hosted LLM called through an SDK. Putting any of these behind a web address is FastAPI.
A common beginner mistake is to reach for the biggest tool: training a neural network when a three-line scikit-learn model, or even a plain if statement, solves the problem.
04What to install and what to skip
On your own machine you add libraries with a package manager (Phase 6). Heavy ones, such as PyTorch, can be several hundred megabytes and may need a matching GPU driver. You do not need them to learn the LLM path: that needs only httpx or an SDK and an API key.
In this course the browser runs NumPy, pandas, Pydantic and httpx (with a mock transport, Topic 15.1). It cannot run PyTorch, it cannot call a real network, and it keeps scikit-learn as a reference only. So the scikit-learn and PyTorch topics show their real code in blocks you run later on your machine, and teach the same ideas live with small NumPy versions.
05Two honest warnings
Libraries in this area change quickly. Function names, defaults and recommended patterns move between versions, so read the current documentation and pin the versions you use (Topic 16.2). A tutorial from two years ago may already use a deprecated call.
Frameworks come and go. The ideas in this phase (messages, tokens, retrieval, tool loops, evals) stay the same. Learn the ideas by hand and any framework becomes a convenience instead of a mystery.
Try it yourself
- 1
Extend the map
In the first example add two more layers you have heard of. Decide for each: what is its one-sentence job, and when would a learner meet it?
- 2
Route your own tasks
Write three tasks from your own life or work and decide which layer you would reach for first. Then ask: could a simpler layer do it?
Code & diagrams
Each layer is a small record. Add a layer of your own (say "Docker: package it") and run it again.
Expected output
1. Python glue: files, functions, classes, errors
2. NumPy, pandas fast arrays and tables
3. scikit-learn classic machine learning
4. PyTorch tensors, autograd, neural nets
5. Hugging Face download and run trained models
6. LLM APIs and SDKs rent a big model over HTTP
7. Frameworks ready-made RAG and agent parts
8. FastAPI, vector stores serve it, store vectors
LLM engineers start at layer 6The "model" and the "LLM" here are tiny stand-ins. The call shape (values in, value out) is the same as the real thing.
Expected output
pure Python dot: 32
NumPy dot: 32
tiny model y = w*x + b: [3.0, 5.0, 7.0]
You asked: what is a tensor?Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Import a library that is not there
Run a program that imports a package that is not installed (here a misspelt name).
import numpyy
print(numpyy.array([1, 2, 3]))Myth vs fact
Myth
AI means PyTorch and neural networks.
Fact
Most day-to-day AI engineering today is calling hosted models and surrounding them with ordinary Python: validation, retrieval, retries, tests. Neural-network training is one specialised branch.
Myth
Frameworks are required to build with LLMs.
Fact
They are optional. A model call is an HTTP request, and a RAG pipeline or agent loop is a few dozen lines of plain Python, as you will build in Topics 16.9 and 16.10.
Myth
Python is slow, so it is a bad choice for AI.
Fact
The heavy arithmetic runs in compiled C, C++ and GPU code. Python only orchestrates, so its speed rarely matters for the model itself. It matters more for your own loops over data, which is why NumPy exists.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Most of the scientific stack shares one data model: the n-dimensional array. NumPy arrays, pandas columns, PyTorch tensors and many GPU libraries can convert between each other, sometimes without copying memory. Knowing about shapes and dtypes transfers across all of them.
- ▸
Dependency weight is a real engineering concern. A serving image that includes PyTorch can be gigabytes. Teams often split a small API service from a heavy training or batch environment, and use containers to pin both (see the Docker course at /docker).
- ▸
The boundary between "your code" and "the model" is where the engineering sits: input validation, output validation, timeouts, retries, cost control and logging. The rest of this phase is mostly about that boundary.
Remember this
- 1
Python itself is the glue. The heavy arithmetic in AI is done by code written in C, C++ or CUDA (the language for graphics cards). Python is the friendly steering wheel: you write short Python, and it calls those fast libraries. That is why this course taught functions, classes, errors and files first: the AI libraries are used through exactly those.
- 2
NumPy and pandas are the base layer. NumPy gives fast arrays (Phase 14). pandas gives tables with labelled columns. Every other library on the map either builds on NumPy arrays or imitates them. If you can slice an array and add a column, the rest becomes readable.
- 3
scikit-learn is classic machine learning. Machine learning means a program that learns a rule from examples instead of being given the rule. scikit-learn offers dozens of such learners behind one shape:
fiton training data,predicton new data,scoreto measure (Topic 16.12). It runs on a laptop with no graphics card. - 4
PyTorch is for neural networks. A neural network is a large function with millions of adjustable numbers, trained by repeatedly nudging them. PyTorch provides tensors (arrays that can live on a GPU), automatic gradients (numbers that tell you which way to nudge each adjustable number) and training tools (Topic 16.13). Hugging Face is a hub and library for downloading models other people have already trained and running them with a few lines.
- 5
LLM APIs and SDKs let you rent a very large language model over the internet. LLM stands for large language model: a neural network trained on huge amounts of text that writes text one small piece at a time. An API is the set of requests a service accepts, and an SDK wraps those requests in Python functions. You send messages as JSON (text in a fixed, labelled format; Topic 7.7) and get text back (Topic 16.3). Frameworks such as LangChain, LlamaIndex and agent SDKs package common patterns: retrieval (looking up useful text to give the model), tool calling (letting the model ask your code to do something) and chains of steps. They save time once you understand what they wrap. They hide the mechanics from you if you do not. Serving means FastAPI to expose your code as a web service. Vector stores search embeddings at scale. An embedding is a list of numbers that stands for the meaning of a piece of text (Topic 16.8).
- 6
Learn in this order: Python, then NumPy and pandas, then either the LLM path (APIs, structured output, RAG, agents, evals: Topics 16.3 to 16.11) or the model path (scikit-learn, then PyTorch). Most working AI engineers today use the LLM path daily and the model path occasionally. Neither path is more real than the other.
Explain it without notes
What is the role of Python in an AI system, given that the heavy computation is not done in Python?
Name the layers of the AI Python stack and say what each is for.
When would you use scikit-learn instead of PyTorch, and when would you call a hosted LLM instead of either?
A teammate proposes an LLM call to classify 2 million support tickets into 12 fixed categories you already have labelled history for. What do you ask, and what would you try first?
Practice
Write a function pick_layer(task) that returns the right library for a few tasks and a helpful default otherwise.
Write a function comes_first(a, b) that says whether layer a should be learned before layer b, using a list that gives the learning order.
Write a function layer_count(stack) that counts how many distinct layers a list mentions (the list may repeat names).
Trade-offs
- ↔
Frameworks save time and hide detail. They are quick for a first version. They can also lock you into someone else's abstractions, and their APIs change often. Build once by hand so you can judge what a framework adds.
- ↔
Hosted models versus training your own. Hosted models need no training data or GPU and are strong on language. They cost money per use, send your data to a third party, and can change under you. Your own small model is cheaper per call and private, but you must collect data and maintain it.
Done when you can
Done when you can name the layers of the stack and say in one sentence what each is for.
Done when you can choose the simplest layer that solves a described task.
Done when you can explain why Python is fast enough for AI work.
Done when you can explain the difference between the LLM path and the model path.
Done when you know that a full Hectal course on LLM engineering is planned, with others on machine learning and deep learning.