Python Programming
Zero → Job-Ready

🎯 Module goal: Set up a professional Python environment and write small scripts fluently — variables, core data types, operators and formatted input/output.

1.1 Installing Python & VS Code; running your first script

A Python program is just a text file of instructions that the Python interpreter executes top to bottom. Your job in this section is to install that interpreter plus an editor that makes working with it pleasant. Download the latest stable Python 3 from python.org. On Windows, tick "Add python.exe to PATH" during installation — skipping this checkbox is the single most common setup failure for beginners. Then verify everything works from a terminal:

python --version    # Windows
python3 --version   # macOS / Linux

Next install Visual Studio Code and add the official Python extension (search "Python" by Microsoft in the Extensions panel). Create a folder for your code, open it in VS Code, and make a file called hello.py:

# hello.py
print("Hello, world!")
print("My first script ran successfully.")

Run it from the terminal in that folder with python hello.py. You can also type python on its own to open the REPL — an interactive prompt where each line executes immediately. Treat the REPL as a scratchpad for testing one-liners, and treat .py files as the real programs you keep.

1.2 Variables, naming rules and dynamic typing

A variable is a name that refers to a value. You create one the moment you first assign to it — no declarations, no type annotations required:

name = "Aisha"
age = 21
price = 9.99
is_admin = False

Naming rules: names may contain letters, digits and underscores but cannot start with a digit; they are case-sensitive (Age and age are different variables); and you cannot use reserved words like if, class or for. Convention: Python uses snake_case for variables and functions, and reserves CapitalisedNames for classes (Module 6).

The second big idea is dynamic typing: a variable is not locked to one type. The name is just a label attached to an object, and tomorrow it can label a different object of a completely different type:

x = 42        # x refers to an int
x = "hello"   # perfectly legal: x now refers to a str

This flexibility speeds up small scripts, but it also means typos silently create new variables instead of raising errors — if you spell price as prcie, Python will happily create a second variable rather than warn you. Keep names consistent.

1.3 Data types: int, float, str, bool, NoneType

Every value in Python has a type, and five built-in types cover most beginner code:

• int — whole numbers of unlimited size: 42, -7, 10**100. Python integers never overflow.
• float — decimal numbers stored as binary floating point: 9.99, 2.5e8. They are approximations, which is why 0.1 + 0.2 == 0.30000000000000004.
• str — text, i.e. an immutable sequence of characters: "hello", 'a', even "".
• bool — logical values True and False; the result of every comparison.
• NoneType — the single value None, meaning "nothing here yet". Functions without a return give back None.

Use the built-in type() function to inspect any value — it returns a description you can print or compare:

print(type(42))       # <class 'int'>
print(type(9.99))     # <class 'float'>
print(type("hi"))     # <class 'str'>
print(type(True))     # <class 'bool'>
print(type(None))     # <class 'NoneType'>

1.4 Operators: arithmetic, comparison, logical, assignment shortcuts

Operators let you combine values into expressions. Arithmetic operators produce numbers:

Note that / always returns a float, even for whole results (6 / 3 is 2.0) — reach for // when you want integer division. The modulo operator % is everywhere in real code: n % 2 == 0 tests evenness, and i % 10 cycles through digits 0–9.

Comparison operators (== != < > <= >=) return a bool, and they chain naturally — 18 <= age < 65 means exactly what it reads like, no and needed. Logical operators and, or, not combine booleans using short-circuit evaluation: in a and b, if a is false, b never runs. Finally, every arithmetic operator has a shortcut form that updates a variable in place:

score = 10
score += 5   # same as score = score + 5  → 15
score *= 2   # → 30
score //= 7  # → 4

1.5 Input/output, f-strings and string formatting

print() writes to the screen and input() reads a line from the keyboard. One critical detail: input() always returns a string, even when the user types digits — converting is your responsibility (next section). For building output text, modern Python uses f-strings: prefix the string with f and drop any expression inside curly braces:

name = input("Your name: ")
age = int(input("Your age: "))
print(f"Hi {name}, next year you'll be {age + 1}.")

Anything legal in Python goes inside the braces — arithmetic, function calls, other variables. Add a format specifier after a colon for fine control: {price:.2f} forces two decimal places, {total:,} inserts thousands separators, {name:>10} right-aligns in ten characters:

price = 1234.5
print(f"Total: ${price:,.2f}")   # Total: $1,234.50

1.6 Type conversion and the type() function

Because types matter, you constantly convert between them with constructor-style functions: int(), float(), str() and bool(). Conversion from strings fails loudly when the content isn't numeric — int("abc") raises a ValueError, which is exactly what you want to discover bad input early:

raw = input("How many tickets? ")
count = int(raw)              # may crash on bad input — fine for now
print(f"{count} tickets = ${count * 12.99:.2f}")
print(str(count) + "!")       # str() lets numbers join text safely

Two conversions worth memorising: int(9.99) truncates toward zero to give 9 (it does not round — use round(9.99) for that), and bool() maps "empty" values like 0, "" and None to False while anything else becomes True — an idea called truthiness that Module 2 builds on. When debugging weird behaviour, ask type(x) before anything else; nine beginner bugs are "it was actually a string".

🎯 Module goal: Make programs decide and repeat — master conditionals, truthiness, both loop types, Python's loop controls, and the modern match statement.

2.1 if / elif / else and nested conditions

Conditionals let your program choose between paths at runtime. Python checks each condition top to bottom and runs the first branch whose test is true, skipping all the rest — so ordering is part of your logic:

score = 87
if score >= 90:
    grade = "A"
elif score >= 80:
    grade = "B"
elif score >= 70:
    grade = "C"
else:
    grade = "F"

Three rules make this click. First, indentation defines the blocks: everything indented under an if belongs to it (use 4 spaces, always). Second, elif chains stop at the first true condition — in the example above, a score of 95 never reaches the >= 80 check because the first branch already fired. Third, else has no condition; it catches everything the chain missed.

You can nest conditions inside conditions, but deep nesting is where bugs breed. Prefer "flattening" with early exits or combined tests:

# nested (harder to read)
if logged_in:
    if has_permission:
        do_action()

# flat (cleaner)
if logged_in and has_permission:
    do_action()

2.2 Truthy/falsy values — what counts as False

Every value in Python can be tested in an if, not just booleans. A short list of values counts as falsy: False, None, zero (0, 0.0), empty strings ("") and empty collections ([], {}, set(), range(0)). Everything else is truthy — including negative numbers and strings like "0".

This enables Python's most idiomatic patterns. Instead of writing if len(items) > 0:, professionals write if items:. And instead of long default logic, you can exploit or's short-circuit: name = user_name or "anonymous" falls back when the name is empty.

2.3 while loops, break, continue, else-on-loop

A while loop repeats as long as its condition stays true. Use it when you can't know in advance how many repetitions you need — validating input, waiting for events, playing game rounds:

secret = 7
guess = None
attempts = 0
while guess != secret:
    guess = int(input("Guess: "))
    attempts += 1
    if guess < secret:
        print("Too low!")
    elif guess > secret:
        print("Too high!")
print(f"Got it in {attempts} tries!")

Two keywords control any loop from inside: break abandons the loop immediately, and continue skips to the next iteration while staying inside the loop. A quirk worth knowing: a loop can have its own else block, which runs only if the loop finished without hitting break — handy for search loops ("found it" vs "checked everything").

for n in [3, 5, 9]:
    if n % 2 == 0:
        print("Found even:", n)
        break
else:
    print("No even number found")   # runs here

If a while loop never ends (condition never becomes false), press Ctrl+C in the terminal to interrupt it — then figure out why the exit condition never fired. Every while loop needs something inside that moves it toward stopping.

2.4 for loops with range() and iterables

A for loop visits each item of a sequence one at a time. It works on anything iterable — lists, strings, dictionaries, file lines, results of range():

for fruit in ["apple", "kiwi", "plum"]:
    print(fruit)

for ch in "abc":
    print(ch)

range() generates number sequences without building them in memory. It takes up to three arguments — start (default 0), stop (excluded!), step (default 1):

for i in range(3):          # 0, 1, 2
    print(i)
for i in range(10, 0, -1):  # countdown 10..1
    print(i)

2.5 Structural pattern matching (match/case, Python 3.10+)

The match statement is Python's modern answer to giant if/elif ladders, especially when branching on the shape or value of data:

command = input("> ").strip().lower()
match command.split():
    case ["quit"]:
        print("Bye!")
    case ["go", direction]:
        print(f"Moving {direction}...")
    case ["look", *items] | ["inspect", *items]:
        print("Examining", items)
    case _:
        print("Unknown command")

Read it like this: each case is a pattern the subject is tested against. Literal values must match exactly; a bare lowercase name like direction captures whatever arrived; *items collects the rest of a list; | means "or"; and _ is the wildcard that matches anything — your equivalent of else. Patterns can also add guards: case n if n < 0: matches only negatives.

2.6 Nested loops and loop performance intuition

Loops nest naturally whenever every item of one collection pairs with items of another — printing grids, comparing all combinations, building tables:

for row in range(1, 4):
    for col in range(1, 4):
        print(f"{row * col:3}", end=" ")
    print()   # newline after each row

Now build the performance instinct: two nested loops over N items run N×N iterations. Loop over 1,000 students once — trivial. Compare every student with every other student — a million operations, noticeably slow. This multiplication effect (called quadratic time) is why experienced developers ask "does this inner loop really need to see every item?" Often the fix is a smarter data structure (a set lookup instead of an inner scan — Module 4) rather than cleverer looping. Also remember break early: once you've found what you were searching for, stop iterating.

🎯 Module goal: Package logic into reusable, well-documented functions — master parameters, scope, lambdas, recursion and your first decorator.

3.1 Defining functions: parameters vs arguments, return values

A function is a named, reusable block of code. Define one with def; the names in parentheses are parameters (placeholders), and the values you pass when calling are arguments (actual data):

def greet(name, punctuation):
    return f"Hello{name}, {punctuation}"

message = greet("Aisha", "!")   # arguments: "Aisha", "!"

Two things happen at a call: Python jumps to the function body with the arguments bound to the parameters, then returns to where you called from, bringing back whatever return handed over. A function without an explicit return gives back None — so if you see None printed where you expected data, you forgot to return. You can also pass arguments by keyword, which makes calls self-documenting and order-proof: greet(punctuation="?", name="Sam").

return ends the function immediately, which enables early-exit guards. And since any expression can be returned, a function can hand back multiple values as a tuple that unpacks on arrival:

def min_max(numbers):
    return min(numbers), max(numbers)

low, high = min_max([4, 9, 1])   # low=1, high=9

3.2 Default, keyword-only, *args and **kwargs

Parameters can declare defaults, making them optional at call time:

def power(base, exponent=2):
    return base ** exponent

power(5)        # 25
power(5, 3)     # 125

Sometimes you want to accept any number of arguments. *args collects extra positional arguments into a tuple; **kwargs collects extra keyword arguments into a dict:

def log(level, *values, timestamp=True, **extras):
    print(level, values, extras)

log("INFO", "disk", "full", retry=3)
# INFO ('disk', 'full') {'retry': 3}

The full parameter order is fixed: positional, then *args, then keyword-only ones (anything after * must be passed by name — like timestamp above), then **kwargs. The star syntax also works at call sites to unpack: log(*my_list, **my_dict).

3.3 Scope: LEGB rule, global/nonlocal

Name lookup follows the LEGB rule — Local, Enclosing, Global, Builtins. Python searches in that order and uses the first match: variables assigned inside a function are local to each call; enclosing functions sit just outside that; module-level names are global; and builtins like len come last:

x = "global"
def outer():
    x = "enclosing"
    def inner():
        x = "local"
        print(x)    # local
    inner()
outer()

Reading outward is automatic; writing inward needs permission. Assigning to a global name inside a function actually creates a new local variable unless you declare global x — and nonlocal x does the same for enclosing scopes:

count = 0
def bump():
    global count
    count += 1

Treat both keywords as smells: functions that quietly rewrite globals are hard to test and reason about. Prefer passing values in through parameters and out through return — note that mutating an object passed in (like items.append(...)) needs no declaration at all, because you're changing the object, not rebinding the name.

3.4 Lambda functions, map/filter/sorted with key functions

A lambda is a one-expression anonymous function — a tiny throwaway callable you pass where a full def would be ceremony:

double = lambda n: n * 2          # works, but named def is clearer

words = ["banana", "fig", "cherry"]
print(sorted(words, key=len))               # sort by length
print(sorted(words, key=lambda w: w[-1]))   # sort by last letter

Lambdas shine as key functions. sorted(iterable, key=f) ranks items by what f(item) returns rather than by raw comparison. The same idea powers max(data, key=...) and min(...). The classics map(f, seq) (apply) and filter(pred, seq) (keep matching) return lazy iterators — wrap them in list() to see results:

nums = [1, 2, 3, 4]
doubled = list(map(lambda n: n * 2, nums))       # [2, 4, 6, 8]
evens = list(filter(lambda n: n % 2 == 0, nums)) # [2, 4]

In modern code, list comprehensions (Module 4) usually replace map/filter for readability — but you must recognise this style because every codebase uses it.

3.5 Docstrings and clean function design (single responsibility)

A docstring is a triple-quoted string placed as the first statement of a function; tools like help() and IDEs surface it, so it's documentation that lives next to the code:

def celsius_to_fahrenheit(celsius):
    """Convert a Celsius temperature to Fahrenheit."""
    return celsius * 9 / 5 + 32

Beyond documenting, design functions around the single-responsibility principle: one function, one job, described accurately by its name. If you can't summarise what a function does without saying "and", split it. Good functions are also small enough to read without scrolling, take what they need via parameters instead of reaching for globals, and avoid surprising side effects. A quick self-test: could you write a test for this function knowing only its signature and docstring? If yes, its boundaries are clean.

3.6 Recursion basics: factorial, fibonacci

A recursive function calls itself on a smaller version of the problem. Every correct recursion has two parts: a base case that stops the descent, and a recursive step that moves toward it. Without a reachable base case you get RecursionError:

def factorial(n):
    if n <= 1:            # base case
        return 1
    return n * factorial(n - 1)   # recursive step

def fib(n):
    if n <= 1:
        return n
    return fib(n - 1) + fib(n - 2)

Trace factorial(3) mentally: it waits on factorial(2), which waits on factorial(1), which returns 1 — then results flow back up: 2, then 6. Each waiting call occupies stack memory, which is why naive fib becomes painfully slow around n≈35 (it recomputes the same values exponentially many times) and why Python caps recursion depth near 1000. Use recursion for naturally nested problems — trees, folders, JSON — and loops elsewhere.

3.7 First taste of decorators as reusable wrappers

Functions are objects in Python: you can store them in variables, pass them to other functions, and return them. A decorator exploits this — it's a function that takes a function and returns an enhanced replacement, letting you bolt behaviour (logging, timing, caching) onto existing code without editing it:

import time

def timed(func):
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        print(f"{func.__name__} took {time.perf_counter() - start:.4f}s")
        return result
    return wrapper

@timed                       # means: slow_sum = timed(slow_sum)
def slow_sum(a, b):
    time.sleep(1)
    return a + b

slow_sum(2, 3)   # prints the timing automatically

Read the @timed line as pure shorthand. Inside, wrapper uses *args, **kwargs so it forwards any arguments transparently. Don't worry if this feels like magic — it's just higher-order functions plus syntax sugar, and after Modules 6–7 you'll write these comfortably. For now, recognise decorators when you meet them (@property, @staticmethod, pytest fixtures) and know they mean "this function got wrapped".

🎯 Module goal: Choose and wield the right container automatically — lists, tuples, dicts, sets, comprehensions, and the collections power-tools.

4.1 Lists: indexing, slicing, methods, mutability gotchas

A list is an ordered, mutable collection — your everyday container. Items are indexed from 0, negative indexes count from the end, and slices extract sub-lists as new lists using the pattern [start:stop:step] where stop is excluded:

langs = ["py", "js", "go", "rs"]
langs[0]      # "py"
langs[-1]     # "rs"
langs[1:3]    # ["js", "go"]   (index 3 excluded)
langs[::-1]   # reversed copy

The workhorse methods: .append(x) adds to the end, .insert(i, x) inserts at a position, .pop() removes-and-returns (last item by default), .remove(x) deletes by value, and .sort() sorts in place while sorted(lst) returns a fresh sorted copy. Know the difference — confusing them is a classic bug.

4.2 Tuples, unpacking and swapping

A tuple is an immutable list: created with parentheses, impossible to append to or sort after creation. Immutability makes tuples safe as fixed-shape records — coordinates, RGB colours, database rows — and allows them to be dictionary keys (lists can't):

point = (3, 5)
single = (42,)          # one-element tuple NEEDS the comma
x, y = point            # unpacking: x=3, y=5

Unpacking is the superpower. Multiple assignment is just tuple packing/unpacking, which gives you the famous one-line swap without any temp variable: x, y = y, x. Starred targets collect leftovers into a list, which is perfect for "first and everything else" patterns:

first, *rest = [1, 2, 3, 4]   # first=1, rest=[2, 3, 4]
head, *mid, tail = [1, 2, 3, 4]  # head=1, mid=[2, 3], tail=4

Rule of thumb: reach for a list when contents will grow or change; reach for a tuple when the shape is fixed and you're promising callers "this won't be modified".

4.3 Dictionaries deep dive: get/setdefault, items, merging (| operator)

A dict maps unique, hashable keys to values — think of it as an index card catalogue rather than a numbered shelf. Since Python 3.7 dictionaries preserve insertion order, so what you put in first comes out first:

user = {"name": "Aisha", "level": 3}
user["level"] = 4              # update
user["email"] = "a@x.dev"      # add

Reading a missing key with square brackets raises KeyError; the professional habit is .get() with a fallback: user.get("email", "n/a"). Its sibling setdefault(key, default) fetches like get, but when the key is absent it also stores the default — ideal for building grouped data in loops:

groups = {}
for word in words:
    groups.setdefault(word[0], []).append(word)

Iterate over key-value pairs together with .items(): for k, v in user.items():. To combine dictionaries, Python 3.9+ gives you merge operators: merged = defaults | overrides (values on the right win) and the in-place d |= other.

4.4 Sets: dedupe, union/intersection, membership speed

A set stores unordered unique items. Its killer features: instant de-duplication and blazing membership tests:

tagged = ["a", "b", "a", "c", "b"]
unique = set(tagged)          # {"a", "b", "c"}

"b" in unique                 # True — near-instant even for millions

Sets support mathematical operations with intuitive symbols: union a | b (in either), intersection a & b (in both), difference a - b (in a but not b). Typical real uses: comparing two lists' overlaps ("which emails are in both files?"), tracking seen-items during a loop, and removing duplicates before sorting (sorted(set(items))). Remember that sets can't contain unhashable items — no lists or dicts inside a set; convert to tuples if needed.

4.5 List/dict/set comprehensions with conditions

Comprehensions build collections in a single readable line, replacing the create-loop-append ritual. The mental template reads left to right: "collect this expression for each item, if it passes the filter":

nums = [1, 2, 3, 4, 5, 6]

squares = [n * n for n in nums]              # [1, 4, 9, 16, 25, 36]
evens = [n for n in nums if n % 2 == 0]      # filter: [2, 4, 6]
labels = ["even" if n % 2 == 0 else "odd" for n in nums]

lengths = {w: len(w) for w in words}         # dict comprehension
vowels = {ch for ch in "banana"}             # set comprehension

Note the subtle difference between a trailing if (filters which items enter) and a leading conditional expression X if cond else Y (transforms every item). Comprehensions nest for 2D data — [[c for c in row] for row in grid] — but the moment a comprehension needs two nested loops plus a condition, a regular loop is kinder to future readers.

4.6 Choosing structures: Big-O intuition for lookups

You don't need formal computer science to pick containers well — you need three facts about cost growth. Checking whether an item exists in a list scans item by item: twice the data, twice the wait (linear). Doing the same check on a set or getting a key from a dict takes roughly constant time regardless of size, thanks to hash tables. Indexing a list by position is also constant.

4.7 collections module: Counter, defaultdict, deque, namedtuple

The standard library's collections module ships upgraded containers for the most common jobs. Import them individually: from collections import Counter, defaultdict, deque, namedtuple.

from collections import Counter, defaultdict, deque

votes = ["ana", "bo", "ana", "cy", "ana"]
Counter(votes).most_common(1)     # [('ana', 3)] — top vote-getter

posts_by_tag = defaultdict(list)  # missing keys auto-create []
for tag, post in pairs:
    posts_by_tag[tag].append(post)

history = deque(maxlen=100)       # fixed-size rolling log
history.append("event")           # appends are O(1)
history.popleft()                 # oldest item out — O(1), unlike list.pop(0)

Counter tallies anything hashable in one call. defaultdict(list) replaces the setdefault dance from 4.3 with cleaner code. deque (double-ended queue) adds/removes at both ends in constant time — use it for queues and sliding windows instead of list.pop(0), which is secretly O(n). Finally, namedtuple gives tuples named fields (Point = namedtuple("Point", "x y")) — a featherweight record type that predates Module 6's classes.

🎯 Module goal: Slice, clean and validate text like a specialist — string methods, slicing tricks, and regular expressions that extract anything.

5.1 Immutability, common methods: split/join/strip/replace/find

Strings are immutable: no method ever changes a string in place — each one returns a brand-new string, so you must capture the result. name.strip() on its own does nothing useful; name = name.strip() is the real move:

s = "  Hello, World  "
s = s.strip()            # "Hello, World" — trims whitespace both ends
parts = "a,b,c".split(",")       # ['a', 'b', 'c']
joined = "-".join(parts)         # "a-b-c"
s2 = s.replace("World", "Python")

These four are the daily drivers of text cleaning. split() with no argument splits on any run of whitespace (great for messy data); split(",") splits on exact separators. join() is called on the separator string and handed an iterable — the standard way to build delimited output. Other essentials: .lower()/.upper()/.title() for case work, .find(sub) which returns the index or -1 if absent, and .index(sub) which raises instead — pick based on whether absence is normal or exceptional. Methods chain left to right: " Hi ".strip().lower() → "hi".

5.2 String testing: isdigit, startswith, in-operator patterns

Before processing text you usually need to ask questions about it, and strings ship with a battery of boolean testers:

"42".isdigit()          # True — safe to int()
"3.14".isdigit()        # False! dot breaks it
"report.csv".startswith("report")
"2026-08-23.csv".endswith(".csv")
"py" in "smoothpython"  # True — substring test

startswith/endswith accept tuples too — f.endswith((".jpg", ".png")) checks several suffixes at once, which is exactly how file filters are written. The in operator doubles as a membership check for substrings and pairs beautifully with loops: filter lines with if "ERROR" in line:. For validation beyond these basics — emails, IDs, codes — that's your cue to reach for regex in sections 5.4–5.6.

5.3 Slicing tricks: reversal, stepping

Slicing syntax s[start:stop:step] has a third field most beginners never use, yet it unlocks elegant one-liners. A negative step walks backwards; omitting start/stop means "from the edges":

s = "abcdefgh"
s[::-1]      # "hgfedcba" — the idiomatic reversal
s[::2]       # "aceg" — every second char
s[-3:]       # "fgh" — last three chars
s[:-3]       # "abcde" — all but last three
s[1:-1]      # strip first & last characters

Slicing works identically on lists, so every trick here transfers: reverse a list with lst[::-1], take the top five scores with sorted(scores, reverse=True)[:5].

5.4 Regular expressions: re.search, findall, sub

A regular expression (regex) is a compact pattern language for describing text shapes — "three digits, a dash, four digits". Python's re module compiles patterns into matchers. The three functions you'll use constantly:

import re

text = "Order 1024 shipped on 2026-08-23, order 2048 pending."

re.search(r"\d{4}", text).group()     # '1024' — first match only
re.findall(r"\d{4}", text)            # ['1024', '2048', '2026', '08', '23']
re.sub(r"\d{4}", "XXXX", text)        # replace every match

re.search returns a Match object (truthy!) or None, so it slots straight into conditionals: if m := re.search(pat, line):. Call .group() to get matched text and .groups() for captured groups. findall returns a list of all matches — perfect for extraction jobs like pulling every IP from a log. When a pattern runs in a hot loop, compile once — pattern = re.compile(r"...") — then call pattern.search(...).

5.5 Regex building blocks: classes, quantifiers, groups, anchors

Regex reads like a mini-language of atoms. Learn this table and you can assemble 90% of real patterns:

Parenthesised groups both organise patterns and capture pieces for later: m = re.search(r"(\d{2})/(\d{2})/(\d{4})", date) then m.group(1) is the day. Name groups for readability: (?P<day>\d{2}) lets you write m["day"]. One nuance: quantifiers are greedy — they grab the most possible. Add ? (<.+?>) to make them lazy and stop at the first closing bracket rather than the last.

5.6 Real patterns: emails, phone numbers, dates, passwords

Assemble the vocabulary above into production-grade patterns. Study how each one composes simple parts:

email_re   = r"[\w.+-]+@[\w-]+\.[\w.]+"
phone_re   = r"(\+?\d{1,3}[- ]?)?\(?\d{3}\)?[- ]?\d{3}[- ]?\d{4}"
date_re    = r"(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})"
strong_pw  = r"^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[\W_]).{8,}$"

if re.search(email_re, "ping me at dev@site.io"):
    print(re.findall(email_re, "dev@site.io, sales@shop.co.uk"))
# ['dev@site.io', 'sales@shop.co.uk']

The password pattern uses lookaheads (?=.*X) — zero-width assertions meaning "somewhere ahead there must be an X" — letting one expression enforce multiple independent rules. Two honesty notes from the trenches: regex validates formats, not truth (an email pattern can't confirm the address exists), and for deeply structured markup like full HTML you should graduate to a parser (Module 10's BeautifulSoup) instead of fighting regex. Test everything you build on regex101.com before shipping it into code.

🎯 Module goal: Model real-world systems with classes — instances, inheritance, dunder protocols, composition and dataclasses.

6.1 Classes, instances, __init__, instance vs class attributes

A class is a blueprint; an instance (object) is one concrete thing built from it. The blueprint bundles state (attributes) with behaviour (methods) so the two travel together instead of drifting apart in loose variables:

class Player:
    def __init__(self, name, hp=100):
        self.name = name      # instance attribute
        self.hp = hp

p1 = Player("Aisha")
p2 = Player("Bo", hp=80)
print(p1.name, p2.hp)

__init__ is the initialiser that runs automatically at creation time; self is the specific instance being built, and every method receives it as the invisible first argument. Attributes assigned through self belong to that one object. A trap to know: attributes written directly in the class body are class attributes, shared by all instances — fine for constants like MAX_HP = 100, dangerous for mutable values like inventory = [] (all players would share one list!). Assign mutable state inside __init__.

6.2 Methods, self, and @property

Methods are functions defined inside a class whose first parameter always receives the instance — you never pass self yourself, Python does it when you call p1.take_damage(10). Methods read and mutate the object's own state, which is the whole point of bundling data with behaviour:

class Player:
    def __init__(self, name, hp=100):
        self.name = name
        self.hp = hp

    def take_damage(self, amount):
        self.hp = max(0, self.hp - amount)
        return self.hp

    @property
    def is_alive(self):
        return self.hp > 0

if player.is_alive:   # note: no parentheses — property, not call
    ...

@property turns a method into a computed attribute: callers access player.is_alive like plain data while the code recalculates it on the fly. Use properties whenever a value is derived from other state — they let you start with simple attributes and swap in computed ones later without breaking any caller.

6.3 Inheritance, super(), method overriding

Inheritance models "is-a" relationships: a subclass automatically gets everything from its parent, then extends or specialises it. Override a method by redefining it; call the parent's version with super() when you want to build on it rather than replace it:

class Enemy:
    def __init__(self, name, hp):
        self.name = name
        self.hp = hp
    def attack(self):
        return 5

class Boss(Enemy):                    # Boss IS-A Enemy
    def __init__(self, name, hp, rage):
        super().__init__(name, hp)    # reuse parent setup
        self.rage = rage
    def attack(self):                 # override
        return super().attack() * 3   # extend parent behaviour

isinstance(obj, Class) tests lineage (a Boss counts as an Enemy), and this polymorphism is the payoff: a loop of enemies can call e.attack() without caring about each one's concrete type — each responds its own way. Keep hierarchies shallow though; two levels usually suffice, and deeper trees signal design trouble (see 6.6).

6.4 Dunder methods: __str__, __repr__, __eq__, __len__

Dunder ("double underscore") methods hook your objects into Python's built-in syntax. Define them once and every function, print statement and operator knows how to treat your class:

class Playlist:
    def __init__(self, name, tracks):
        self.name, self.tracks = name, tracks
    def __len__(self):
        return len(self.tracks)          # enables len(playlist)
    def __str__(self):
        return f"Playlist '{self.name}' ({len(self)} tracks)"
    def __repr__(self):
        return f"Playlist({self.name!r}, {self.tracks!r})"
    def __eq__(self, other):
        return isinstance(other, Playlist) and self.tracks == other.tracks

The convention worth memorising: __str__ produces friendly text for users (print(p)), while __repr__ produces unambiguous developer output shown in the REPL and debuggers — ideally something that could recreate the object. __eq__ powers ==; always guard with isinstance because you'll receive foreign objects. Other dunders follow the same pattern on demand: __add__ for +, __getitem__ for indexing, __contains__ for in.

6.5 @classmethod, @staticmethod

Two decorators create methods that aren't ordinary instance methods. A classmethod receives the class itself (cls) instead of an instance — its signature use is alternative constructors. A staticmethod takes neither; it's just a namespaced helper function living inside the class for organisational reasons:

class Date:
    def __init__(self, y, m, d):
        self.y, self.m, self.d = y, m, d

    @classmethod
    def from_string(cls, s):                 # second constructor
        y, m, d = map(int, s.split("-"))
        return cls(y, m, d)

    @staticmethod
    def leap(year):
        return year % 4 == 0 and (year % 100 != 0 or year % 400 == 0)

d = Date.from_string("2026-08-23")
Date.leap(2028)   # True — called on the class, no instance needed

6.6 Composition over inheritance — when to prefer which

Inheritance means "is-a": Dog is an Animal. Composition means "has-a": a Car has-an Engine. Beginners overuse inheritance because textbooks lead with it; professionals reach for composition most of the time, because wiring objects together is more flexible than locking them into a family tree that must be designed perfectly up front:

class Engine:
    def start(self): return "Vroom"

class Car:
    def __init__(self):
        self.engine = Engine()       # Car HAS-A Engine
    def drive(self):
        return self.engine.start()

class ElectricCar(Car):
    def __init__(self):
        super().__init__()
        self.engine = ElectricEngine()   # swap parts freely

The decision rule: inherit only when the subclass genuinely substitutes for the parent everywhere (a Boss works anywhere an Enemy does). If you catch yourself inheriting just to reuse some code, compose instead — hold an instance of the other class and delegate. You'll know inheritance is wrong if the "child" breaks the parent's promises.

6.7 Dataclasses for clean model objects

Much OOP code is just data with structure: user records, config, API payloads. The @dataclass decorator auto-generates __init__, __repr__ and __eq__ from a field list, deleting boilerplate entirely:

from dataclasses import dataclass, field

@dataclass
class Task:
    title: str
    priority: int = 2
    tags: list = field(default_factory=list)

t = Task("Ship portfolio", priority=1)
print(t)          # Task(title='Ship portfolio', priority=1, tags=[])

Type annotations become constructor parameters; defaults work exactly as you'd hope; equality compares field by field. Add frozen=True to make instances immutable (great for hashable record types). Reach for a dataclass by default when a class mostly holds data, and hand-write a full class only when behaviour dominates — that single habit removes half the boilerplate in typical projects.

6.8 Encapsulation conventions (_private), name mangling

Python has no enforced private fields — it runs on trust and conventions. One leading underscore (_balance) signals "internal, don't touch unless you own this class". Two leading underscores (__secret) triggers name mangling: Python physically renames it to _ClassName__secret so subclasses can't collide with it accidentally:

class Account:
    def __init__(self, balance):
        self._currency = "USD"     # internal by convention
        self.__pin = "1234"        # mangled to _Account__pin
    def check(self, pin):
        return pin == self.__pin

Note what this philosophy buys you: instead of guarding raw attributes, expose intentional methods or @property getters/setters that validate (account.deposit(-50) can refuse negative amounts inside the method). Design classes so the public interface invites correct use and the underscored internals make misuse obvious rather than impossible.

🎯 Module goal: Ship resilient programs — read any traceback, handle failures deliberately, work with files and paths, and organise code into importable packages with clean environments.

7.1 Reading tracebacks like a pro

A traceback is not an insult — it's a map. Python prints it when an exception escapes unhandled, and the reading order is bottom-up: the last line names the exception type and message (start here), and each line above shows the exact file, function and line number that led there:

Traceback (most recent call last):
  File "shop.py", line 12, in <module>
    total = price * count
TypeError: can't multiply sequence by non-int of type 'str'

This one says: at shop.py line 12, you multiplied something by a string. The type name is your diagnosis shortcut — learn the greatest hits: NameError (typo or used-before-assignment), TypeError (wrong kind of value), ValueError (right type, bad content), IndexError/KeyError (missing item), FileNotFoundError, ZeroDivisionError. When the stack has many frames, the deepest frame (just above the error line) usually contains the actual bug; frames above it show how you got there.

7.2 try/except/else/finally; raising; custom exception classes

Error handling lets a program respond to failure instead of dying. Wrap only the risky statement(s); catch specific exception types so real bugs still surface loudly:

try:
    count = int(input("How many? "))
except ValueError as err:
    print(f"Please type a whole number ({err})")
else:
    print("Parsed:", count)      # runs only if NO exception occurred
finally:
    print("Attempt logged")      # always runs — cleanup lives here

You raise errors yourself with raise ValueError("amount must be positive") — turning silent corruption into loud, early failure is a feature called "fail fast". For domain-specific failures define custom exceptions by subclassing Exception:

class InsufficientFunds(Exception):
    """Raised when a withdrawal exceeds the balance."""

def withdraw(balance, amount):
    if amount > balance:
        raise InsufficientFunds(f"short by {amount - balance:.2f}")
    return balance - amount

The underlying philosophy has a name — EAFP, "easier to ask forgiveness than permission": Pythonistas try the operation and handle the exception rather than pre-checking every condition.

7.3 File I/O: open modes, encoding, reading line by line

open(path, mode) connects your program to a file. Modes decide intent: "r" read (default), "w" write (truncates existing content!), "a" append, plus "rb"/"wb" for binary data like images. Always pass encoding="utf-8" explicitly — it makes text files portable across Windows, macOS and Linux:

with open("notes.txt", "w", encoding="utf-8") as f:
    f.write("line one\n")
    f.writelines(["two\n", "three\n"])

with open("notes.txt", encoding="utf-8") as f:
    for i, line in enumerate(f, start=1):   # streams lazily
        print(i, line.strip())

The iteration pattern above matters more than it looks: files are iterable line by line, so this loop reads a 10 GB log with constant memory — no giant list ever materialises. Whole-file shortcuts exist for small files: text = f.read() or f.readlines(). Writing converts nothing automatically, so stringify before writing: f.write(str(score)).

7.4 Context managers and the with statement (+ writing your own)

The with statement is a context manager in action: it guarantees setup/teardown pairs run even if an exception explodes mid-block. That's why every file open uses it — the handle closes whether the block succeeds, fails or returns. You can build your own with the @contextmanager decorator: write code before yield (setup) and after it (teardown):

from contextlib import contextmanager
import time

@contextmanager
def timer(label):
    start = time.perf_counter()
    try:
        yield                    # ← your block executes here
    finally:
        ms = (time.perf_counter() - start) * 1000
        print(f"[{label}] {ms:.1f} ms")

with timer("parse"):
    sum(range(1_000_000))

The class-based route exists too — implement __enter__/__exit__ — but the generator version covers most needs in five lines. Once you internalise context managers you'll spot them everywhere: database transactions, locks, temporary directory changes.

7.5 pathlib for modern path handling; os and shutil essentials

Stop gluing strings with / manually — the pathlib.Path object handles separators per-OS, joins with operators, and carries useful methods:

from pathlib import Path

downloads = Path.home() / "Downloads"
for pdf in downloads.glob("*.pdf"):
    print(pdf.name, pdf.stat().st_size)

new_dir = Path("exports/2026")
new_dir.mkdir(parents=True, exist_ok=True)
(downloads / "old.csv").rename(new_dir / "old.csv")
text = Path("notes.txt").read_text(encoding="utf-8")   # tiny-file shortcut

Around it sit two veterans worth knowing: os for environment variables (os.environ["KEY"] — critical in Module 9) and system info, and shutil for high-level file operations — shutil.copy2(src, dst), shutil.move(), shutil.rmtree(path) (delete a whole tree — triple-check the path!), shutil.disk_usage(). Rule of thumb: pathlib for paths and simple I/O, shutil for copying/moving/deleting trees, os for environment and low-level details.

7.6 Modules, packages, imports, __name__ == "__main__"

Any .py file is a module; importing it runs its top-level code once and caches the result under its name. Import styles from most to least preferred: plain import converters, aliased import pandas as pd, targeted from converters import km_to_miles — avoid wildcard import *, which pollutes your namespace unpredictably. A package is just a folder of modules with an __init__.py inside, imported via dotted paths like from mytools.text import slugify.

The famous guard solves an annoying problem: code at a file's top level runs on import, but sometimes you only want it to run when executed directly. The magic comparison detects which case applies:

# dice.py
def roll():
    ...

if __name__ == "__main__":
    # true ONLY when run as: python dice.py
    # false when another file does: import dice
    print(roll())

Every script you write should keep its reusable functions import-safe behind this guard — that single habit turns throwaway scripts into libraries.

7.7 Virtual environments (venv) and pip workflows; requirements.txt

Third-party packages install via pip — pip install requests — but installing into your global Python eventually causes dependency conflicts between projects. The fix is a virtual environment: an isolated, project-private copy where installs stay contained:

python -m venv .venv          # create (once per project)

# activate:
.venv\Scripts\Activate.ps1    # Windows PowerShell
source .venv/bin/activate     # macOS / Linux

pip install requests          # goes ONLY into .venv
pip freeze > requirements.txt # snapshot exact versions
deactivate                    # step back out

A teammate (or future you) recreates the identical environment with python -m venv .venv, activate, then pip install -r requirements.txt. Two hygiene rules: commit requirements.txt to Git but never the .venv folder itself (it's machine-specific and enormous — .gitignore it in Module 11), and recreate venvs rather than upgrading them in place when versions shift.

🎯 Module goal: Ingest, clean and summarise real-world data — CSV, JSON, dates, randomness and statistics — entirely with the standard library.

8.1 CSV reading/writing (csv module)

CSV (comma-separated values) is the universal exchange format for tabular data — every spreadsheet exports it. Python's csv module handles the fiddly parts (quoted fields containing commas, newlines inside cells) that naive split(",") code gets wrong:

import csv

with open("sales.csv", newline="", encoding="utf-8") as f:
    reader = csv.DictReader(f)          # rows become dicts
    for row in reader:
        print(row["region"], row["revenue"])

with open("summary.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["region", "total"])
    writer.writeheader()
    writer.writerow({"region": "North", "total": 1250})

Prefer DictReader/DictWriter: accessing columns by name (row["revenue"]) survives column reordering, while plain-index access breaks silently. Remember all values arrive as strings — convert numbers with float()/int() as you process each row.

8.2 JSON: load/dump, nested access, API payloads

JSON is the language of web APIs and config files. Its types map almost perfectly onto Python's: objects become dicts, arrays become lists, strings/numbers/booleans/null become str/int/float/bool/None. The four functions differ by direction and target: json.load(f) reads a file object, json.loads(s) reads a string ("s" = string), json.dump(obj, f) writes to a file, json.dumps(obj) produces a string:

import json

payload = '{"user": {"name": "Aisha", "tags": ["admin", "dev"]}}'
data = json.loads(payload)
print(data["user"]["name"])            # nested chains are normal
print(data["user"].get("email", "n/a"))  # safe access for optional keys

with open("data.json", "w", encoding="utf-8") as f:
    json.dump(data, f, indent=2, ensure_ascii=False)   # human-readable file

Nested payloads reward defensive navigation: chain .get() calls with defaults instead of assuming every key exists. The round trip is the mental model — any dict/list of plain values can be frozen into JSON text and thawed back; that's exactly how Module 9 will read API responses.

8.3 datetime and timedelta; timezone-aware basics

The datetime module turns time from strings into objects you can compare and do arithmetic on. Parse strings into datetimes with strptime ("parse"), format them back with strftime ("format") using letter codes like %Y %m %d %H:%M:

from datetime import datetime, timedelta, date
from zoneinfo import ZoneInfo

now = datetime.now()
launch = datetime.strptime("2026-09-01 09:00", "%Y-%m-%d %H:%M")
delta = launch - now                    # a timedelta object!
print(f"{delta.days} days {delta.seconds // 3600} h until launch")

deadline = now + timedelta(days=3, hours=6)
utc_now = datetime.now(ZoneInfo("UTC"))          # timezone-aware
tokyo = utc_now.astimezone(ZoneInfo("Asia/Tokyo"))

timedelta is the arithmetic workhorse: add/subtract durations, compare two datetimes to get one. Know the difference between naive datetimes (no timezone attached — ambiguous) and aware ones (constructed with a tzinfo via ZoneInfo, safe to compare across zones). Professional rule: store timestamps in UTC, convert to local zones only at display time.

8.4 random, math, statistics stdlib tour

Three small modules cover most numeric needs without installing anything. random powers simulations and sampling: randint(a, b) inclusive dice rolls, choice(seq) picks an item, shuffle(lst) mixes in place, sample(pop, k) draws k unique items, and seed(42) makes runs reproducible for testing:

import random, math, statistics

rolls = [random.randint(1, 6) for _ in range(1000)]
deck = list(range(52)); random.shuffle(deck)

math.sqrt(144); math.ceil(7.2); math.pi
scores = [72, 85, 90, 60]
statistics.mean(scores)     # 76.75
statistics.median(scores)   # 78.5
statistics.stdev(scores)    # spread of the data

One honesty note: random is pseudo-random — fine for games, simulations and shuffles, but never use it for security purposes (passwords, tokens); that's the secrets module's job.

8.5 Data cleaning patterns: missing values, normalization

Real-world data is dirty — always. Cleaning follows a repeatable checklist you'll apply in every data task from now on:

• Normalise text: strip whitespace and unify case before comparing — v.strip().lower(); treat "N/A", "", "-" as missing sentinels.
• Convert types defensively: wrap casts in try/except or a helper that returns None on failure, then filter Nones out.
• Deduplicate: normalise first, then dedupe with a set (or keep first occurrence while preserving order).
• Validate counts: report how many rows were dropped/fixed — silent data loss is a bug wearing a costume.

def parse_amount(raw):
    try:
        return float(raw.replace("$", "").replace(",", "").strip())
    except ValueError:
        return None                      # missing marker

amounts = [a for a in map(parse_amount, raw_column) if a is not None]
highest = max(amounts)
scaled = [(a - min(amounts)) / (max(amounts) - min(amounts)) for a in amounts]

The last line demonstrates min-max normalisation — squeezing values into 0–1 so wildly different scales (dollars vs quantities) can be compared or fed to algorithms together. Clean first, analyse second; analysis on dirty input is confident nonsense.

8.6 Intro to pandas mindset (optional preview module)

When datasets grow past a few thousand rows, loops start feeling heavy and the pandas library takes over. Its core idea inverts beginner thinking: stop processing rows one at a time and operate on whole columns at once (vectorisation). Where stdlib-Python writes a loop plus accumulator, pandas expresses intent directly:

# stdlib loop version
totals = {}
for row in rows:
    totals[row["region"]] = totals.get(row["region"], 0) + float(row["amt"])

# pandas version — no loop visible
df.groupby("region")["amt"].sum().sort_values(ascending=False)

A DataFrame is a table with labelled columns you select, filter (df[df["amt"] > 100]), aggregate and export — usually reading files directly with pd.read_csv(). You don't need pandas for this course's projects; what you need now is the mindset shift: recognise when a task has outgrown plain dicts and know that a whole ecosystem exists for the next step. Everything you learned in 8.1–8.5 remains the foundation pandas itself is built on.

🎯 Module goal: Give your programs real storage and real reach — model data in SQLite and consume live web APIs safely with requests.

9.1 Relational concepts: tables, keys, relationships

A relational database stores data in tables — rows are records, columns are fields — and lets you query across tables without loading everything into memory. Two kinds of keys hold the model together. A primary key uniquely identifies each row (usually an auto-incrementing id). A foreign key is a column that stores another table's primary key, creating a link:

users            orders
----              ----
id  name          id  user_id  total
1   Aisha         1   1        49.99     ← order 1 belongs to user 1
2   Bo            2   1        12.50     ← so does order 2 (one-to-many)

This one-to-many pattern (one user, many orders) is the workhorse; many-to-many relationships get resolved through a junction table holding pairs of foreign keys. Why not just use files? Databases give you indexed lookups in milliseconds on millions of rows, enforced integrity (no order pointing at a ghost user), and safe concurrent access — the difference between a notebook and a filing system.

9.2 SQLite with sqlite3: create, insert, query, parameters (SQL injection safety!)

SQLite is a full SQL database living in a single file — zero setup, built into Python. The workflow is always connect → cursor → execute → commit:

import sqlite3

con = sqlite3.connect("shop.db")
cur = con.cursor()
cur.execute("""CREATE TABLE IF NOT EXISTS products (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL,
    price REAL)""")

cur.execute("INSERT INTO products (name, price) VALUES (?, ?)",
            ("keyboard", 39.99))
con.commit()

cur.execute("SELECT name, price FROM products WHERE price > ?", (10,))
for name, price in cur.fetchall():
    print(name, price)
con.close()

The with con: statement is a shortcut worth adopting: it commits on success and rolls back automatically if your block raises — transactions for free. Read results with fetchall() (list of tuples), fetchone(), or iterate the cursor directly.

9.3 HTTP fundamentals: verbs, status codes, headers

The web speaks HTTP: your client sends a request, the server answers with a response, and both have structure you must read fluently. Requests carry a method (verb) expressing intent: GET retrieves data, POST creates it, PUT/PATCH update, DELETE removes. Responses carry a three-digit status code summarising the outcome — learn the families and you can diagnose any API conversation at a glance:

• 2xx success — 200 OK, 201 Created (after a POST), 204 No Content
• 3xx redirect — resource moved; libraries usually follow these silently
• 4xx your fault — 400 bad input, 401 unauthenticated, 403 forbidden, 404 missing, 429 rate-limited
• 5xx their fault — 500 server crash, 503 unavailable: retry later, don't blame your code

Headers are metadata lines riding on both sides: Content-Type: application/json declares payload format, Authorization: Bearer ... carries credentials, User-Agent identifies your client. When an API "doesn't work", the status code plus headers tell you which of these layers broke.

9.4 requests library: GET/POST, params, headers, timeouts

The third-party requests library (pip install requests) turns raw HTTP into friendly Python objects. Query strings become a dict via params=; JSON bodies go straight through json=; and every call deserves a timeout so your script never hangs forever on a dead server:

import requests

r = requests.get(
    "https://api.example.com/v1/search",
    params={"q": "python", "limit": 5},
    headers={"Accept": "application/json"},
    timeout=10,
)
r.raise_for_status()                 # throws for 4xx/5xx — fail loudly
data = r.json()                      # parsed body as dict/list

r = requests.post(
    "https://api.example.com/v1/items",
    json={"name": "keyboard", "price": 39.99},
    timeout=10,
)

The response object bundles everything: r.status_code, r.headers, r.text (raw string), r.json() (parsed). Make raise_for_status() reflexive — without it a 404 quietly hands you an error page as if it were data, and you debug the wrong layer entirely.

9.5 Consuming REST APIs + pagination loops

A REST API exposes resources at URLs you manipulate with the HTTP verbs from 9.3 — GET /users/7/orders reads, POST /users creates. Most list endpoints return data page by page to protect their servers, and consuming them follows one reusable loop shape: fetch, process, advance the page, stop when empty:

def fetch_all_items(base_url):
    items, page = [], 1
    while True:
        r = requests.get(base_url, params={"page": page}, timeout=10)
        r.raise_for_status()
        batch = r.json()["results"]
        if not batch:
            return items                     # ran out of pages
        items.extend(batch)
        page += 1
        time.sleep(0.5)                      # be polite

APIs paginate differently — page numbers, cursor tokens, or "next" URLs in the body — but all reduce to this loop with a different stop condition. Respect documented rate limits (sleep between calls, honour 429 responses by backing off), cache what you've already fetched locally instead of re-downloading, and log raw payloads when debugging so you can see exactly what arrived.

9.6 API keys and environment variables (.env, never hardcode secrets)

Nearly every useful API requires a key identifying you. The iron rule: keys never live in source code. Hardcoded keys leak through Git history, screenshots and shared folders within days — and public repos are scraped by bots continuously. Instead, load secrets from environment variables, optionally backed by a local .env file:

# pip install python-dotenv
import os
from dotenv import load_dotenv

load_dotenv()                        # reads .env into os.environ
API_KEY = os.environ["WEATHER_KEY"]  # KeyError = loud, early failure

headers = {"Authorization": f"Bearer {API_KEY}"}

Your project then contains a committed template .env.example (variable names, no values) while the real .env stays local and appears in .gitignore before your first commit. If a secret ever does land in a commit, treat it as compromised: revoke/regenerate it at the provider immediately — deleting the file afterwards doesn't scrub history. This habit costs two minutes; leaking a credit-card-linked cloud key costs real money.

🎯 Module goal: Turn repetitive computer work into scheduled, logged, configurable scripts — CLIs, scraping, Excel reports and email.

10.1 sys.argv and argparse for CLIs

A script becomes a tool when strangers can run it with arguments instead of editing the source. The raw approach reads the list of command-line tokens directly — fast to write, zero safety:

import sys

folder = sys.argv[1]          # python tidy.py ~/Downloads
limit  = int(sys.argv[2]) if len(sys.argv) > 2 else 10

The standard library's argparse upgrades this into a real interface: typed options, defaults, validation and a free --help page generated from your declarations. Declare what the tool accepts; argparse handles parsing, errors and documentation:

import argparse

parser = argparse.ArgumentParser(description="Sort files by type")
parser.add_argument("folder", help="directory to organise")
parser.add_argument("-n", "--dry-run", action="store_true",
                    help="show plan without moving anything")
parser.add_argument("--min-size", type=int, default=0)
args = parser.parse_args()

print(args.folder, args.dry_run, args.min_size)

10.2 Web scraping politely: requests + BeautifulSoup, robots.txt ethics

Scraping is reading web pages as data. Fetch HTML with requests, then parse it with BeautifulSoup (pip install beautifulsoup4), which turns markup into a searchable tree you query by tag, class or CSS selector:

import requests
from bs4 import BeautifulSoup

html = requests.get("https://news.example.com", timeout=10,
                    headers={"User-Agent": "my-digest-bot/0.1"})
soup = BeautifulSoup(html.text, "html.parser")

headlines = soup.select("h2.headline")     # CSS selectors!
for h in headlines[:5]:
    print(h.get_text(strip=True))

"Politely" is the operative word — scraping has etiquette and rules. Check /robots.txt on any site (it states which paths bots may fetch), read the terms of service, identify yourself with an honest User-Agent, rate-limit yourself with time.sleep() between requests, and never scrape personal data or anything behind login that isn't yours to take. Prefer official APIs when they exist — scraping is the fallback, not the default. Sites also change their HTML without notice, so build scrapers defensively: guard against missing elements (if not soup.select(...)) and fail with clear messages.

10.3 openpyxl: generating real Excel reports from Python

When stakeholders live in spreadsheets, deliver spreadsheets. openpyxl (pip install openpyxl) creates genuine .xlsx workbooks with formatting — far more impressive than a CSV dump:

from openpyxl import Workbook
from openpyxl.styles import Font

wb = Workbook()
ws = wb.active
ws.title = "Sales Report"

ws.append(["Region", "Revenue", "Orders"])          # header row
for cell in ws[1]:
    cell.font = Font(bold=True)

for region, revenue, orders in aggregated_rows:
    ws.append([region, revenue, orders])

ws.column_dimensions["A"].width = 18
wb.save("sales_report_2026_08.xlsx")

Beyond basics you can add formulas as strings (ws["B8"] = "=SUM(B2:B7)"), number formats, multiple sheets, and conditional styling. Choose deliberately between formats: CSV for pipelines and other programs, XLSX for humans who open, filter and present. And remember from Module 8 that every number arriving from source data is a string until you convert it.

10.4 Email automation (smtplib) concepts and safety

Sending mail from Python uses the stdlib SMTP client plus TLS encryption. Modern providers block plain passwords, so generate an app-specific password in your email provider's security settings and store it like any secret (Module 9's .env pattern):

import os, smtplib, ssl
from email.message import EmailMessage

msg = EmailMessage()
msg["From"], msg["To"], msg["Subject"] = (
    os.environ["MAIL_USER"], os.environ["MAIL_TO"], "Daily digest")
msg.set_content("Plain-text fallback body")
msg.add_alternative("<html><body><h1>Daily digest</h1>...</body></html>", subtype="html")

with smtplib.SMTP_SSL("smtp.gmail.com", 465,
                      context=ssl.create_default_context()) as server:
    server.login(os.environ["MAIL_USER"], os.environ["MAIL_PASS"])
    server.send_message(msg)

The safety rules are non-negotiable: credentials only via environment variables, recipients opt in, volumes tiny and honest — automation that spams gets accounts banned fast. Use it for personal digests and alerting ("script finished", "error threshold hit"), which is where automated mail genuinely shines.

10.5 Scheduling: time.sleep patterns, cron/Task Scheduler integration

Automation earns its name when it runs without you. The simplest pattern is a polling loop inside the script itself — do work, sleep, repeat:

import time

while True:
    run_digest()                     # your job function
    time.sleep(60 * 60 * 24)         # once per day

It works but keeps a terminal open forever and dies with reboots. Production scheduling flips the model: the OS wakes Python up. On Linux/macOS, cron schedules by minute-hour-day-month-weekday — 0 7 * * * means daily at 07:00 (crontab -e to edit); use absolute paths and log output, since cron runs blind. On Windows, Task Scheduler points at pythonw.exe (no console window) with your script path and sets the trigger. Either way your script should log its runs (next section) so silent failures don't accumulate invisibly.

10.6 Structuring bigger scripts: config, logging module

Past ~150 lines, scripts need structure or they rot. Three upgrades make them maintainable: constants collected at the top (or in a config.py/.env), logic decomposed into named functions with one job each, and a main() entry point behind the module guard. Replace scattered print() debugging with the logging module — same effort, but output carries timestamps, severity levels and can switch destinations without code edits:

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(message)s",
    handlers=[logging.FileHandler("bot.log"), logging.StreamHandler()],
)
log = logging.getLogger("digest")

def run_digest():
    log.info("Fetching headlines...")
    ...

if __name__ == "__main__":
    try:
        run_digest()
    except Exception:
        log.exception("Digest run failed")   # logs full traceback

Use DEBUG for detail you enable while developing, INFO for normal progress, WARNING for oddities, ERROR for failures. log.exception() inside except blocks is the crown jewel — it records the entire traceback automatically, which turns 2 a.m. failures into five-minute fixes.

🎯 Module goal: Work like a professional — tested code, version control fluency, clean style and a project layout employers recognise.

11.1 pytest: assertions, parametrize, fixtures

Tests are executable proof your code works — and insurance that it keeps working while you change it. The pytest framework (pip install pytest) has the lowest friction of any test tool: write functions named test_* in files named test_*.py, use plain assert statements, run everything by typing pytest:

# test_converters.py
from converters import c_to_f

def test_freezing_point():
    assert c_to_f(0) == 32          # arrange → act → assert

def test_boiling_point():
    assert c_to_f(100) == 212

def test_negative():
    assert c_to_f(-40) == -40       # the famous crossover point

Parametrize runs one test against a table of cases, multiplying coverage without copy-paste. Fixtures provide reusable setup (a temp file, a database connection) that each test receives as an argument:

import pytest

@pytest.mark.parametrize("c, f", [(0, 32), (100, 212), (-40, -40), (37, 98.6)])
def test_conversion_table(c, f):
    assert c_to_f(c) == f

@pytest.fixture
def sample_csv(tmp_path):           # tmp_path = built-in temp dir fixture
    p = tmp_path / "data.csv"
    p.write_text("region,amt\nNorth,10\n")
    return p

def test_reader(sample_csv):
    assert len(read_rows(sample_csv)) == 2

Aim tests at behaviour and edges — zero, negatives, empties — not at internal implementation details.

11.2 Debugging strategy: print vs pdb/breakpoints

Debugging is a discipline, not guesswork. The professional loop: reproduce the bug reliably, minimise it to the smallest input that still fails, then locate. Quick print statements are legitimate for locating ("is the value sane here?"), but upgrade to a real debugger when state gets complex:

def process(rows):
    import pdb; pdb.set_trace()     # or just: breakpoint()
    ...                             # program pauses here

The debugger pauses execution and hands you an interactive prompt where four commands do everything: p variable prints any expression's current value, n steps to the next line, s steps into a call, c continues, q quits. In VS Code, clicking the gutter sets visual breakpoints with the same powers plus a live variables panel — use whichever you'll actually reach for. The final technique costs nothing and works everywhere: explain the bug aloud line by line (to a colleague, or a rubber duck). Verbalising forces assumptions into the open, and half of all bugs dissolve before you finish the sentence.

11.3 Git essentials: init, add, commit, branch, merge, .gitignore

Git is a time machine for folders: every commit is a restorable snapshot, so experimentation becomes safe. Start a repository once per project, then repeat the core cycle forever — stage what changed, snapshot it with a message:

git init                        # birth of the repo (once)
git status                      # what's changed / staged
git add app.py                  # stage a file (or . for all)
git commit -m "Add CSV export"  # snapshot; present-tense message
git log --oneline               # history at a glance
git diff                        # unstaged changes, line by line

Branches let you develop features without breaking the working version. Branch, work, then merge back:

git switch -c feature/export    # create + enter branch
# ...work, add, commit...
git switch main                 # back to mainline
git merge feature/export        # fold the work in

Merge conflicts look scary but are honest: Git shows both versions between <<<<<<< and >>>>>>> markers, and you edit the file to the correct combined result, then add + commit. Finally, .gitignore keeps junk out of history — every Python repo starts with entries for .venv/, __pycache__/, .env, and editor folders. Commit .env even once and consider its secrets burned.

11.4 GitHub: repos, README craft, pull request flow

GitHub is Git hosting plus your public portfolio. Push an existing project up in three commands after creating an empty repo on the site:

git remote add origin https://github.com/you/project.git
git push -u origin main         # first push links branches
git push                        # thereafter, just this

The README is your project's shop window — recruiters read it for sixty seconds and decide. A winning structure: one-line description, screenshot or demo GIF, quick-start install/run commands, features list, what you learned / challenges solved. Treat it as part of the product, not an afterthought.

The pull request (PR) flow is how teams collaborate: fork or branch, make changes on your branch, push it, open a PR proposing the merge, discuss in review, then merge when approved. Contributing to open source follows exactly this route, which is why practising it matters even solo: clone your own repo fresh, change something on a branch, open a PR against yourself, and review your own diff critically — it builds the muscle interviews expect.

11.5 Code style: PEP8, black formatter, meaningful names

Code is read far more often than written, so professionals standardise format to spend brainpower on logic instead of spacing debates. PEP8 is Python's style canon — key rules: snake_case for functions/variables, PascalCase for classes, UPPER_CASE for constants, four-space indents, two blank lines between top-level functions, imports grouped (stdlib / third-party / local) at the top:

# before
def calc(x,y=2):return x**y

# after black formats it
def calc(x, y=2):
    return x ** y

Beyond formatting, naming is where craft lives. Rename until names tell the truth: d becomes duration_days, temp() becomes normalise_text(), booleans read as predicates (is_valid, has_permission). If a name needs a comment to be understood, rename instead — the comment will drift, the name can't.

11.6 Project structure for small-to-medium apps

Consistent structure makes any repo instantly navigable — reviewers and future-you find things where they expect them. Memorise this skeleton; it scales from scripts to real applications:

my-project/
├── README.md            # shop window (11.4)
├── requirements.txt     # pinned deps from pip freeze
├── .gitignore           # venv, cache, .env
├── src/
│   └── myapp/
│       ├── __init__.py
│       ├── main.py      # entry point, thin CLI layer
│       ├── core.py      # the actual logic
│       └── io_utils.py  # file/db helpers
└── tests/
    ├── test_core.py
    └── test_io_utils.py

The principles behind it matter more than the exact shape: separate the entry point from logic (so main.py only parses arguments and delegates), keep pure logic apart from I/O (logic that touches no files is trivially testable), mirror tests beside source, and grow new modules when a file passes ~300 lines rather than letting it sprawl. When a stranger can clone, install, run and test your project in under two minutes using only the README, you've built it professionally.

🎯 Module goal: Convert everything you've learned into one shipped flagship project and a job-ready positioning strategy.

12.1 Capstone A: Task manager API with FastAPI + SQLite

This capstone proves backend fundamentals: endpoints, validation, persistence. FastAPI (pip install fastapi uvicorn) is the modern Python web framework — type-annotated functions become documented HTTP endpoints automatically:

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()

class TaskIn(BaseModel):          # validation for free
    title: str
    priority: int = 2

tasks: dict[int, dict] = {}

@app.post("/tasks", status_code=201)
def create(task: TaskIn):
    new_id = len(tasks) + 1
    tasks[new_id] = task.model_dump()
    return {"id": new_id, **task.model_dump()}

@app.get("/tasks/{task_id}")
def read(task_id: int):
    if task_id not in tasks:
        raise HTTPException(404, "Task not found")
    return tasks[task_id]

Run with uvicorn main:app --reload, then open the auto-generated interactive docs at /docs — instant demo material. Build order: start with in-memory storage (above), add GET list + DELETE, then swap the dict for SQLite using Module 9's parameterised queries, and finish with pytest tests hitting each endpoint via TestClient. The final repo shows you can design resources, validate input and persist data — the exact junior-backend interview loop.

12.2 Capstone B: Web scraper dashboard exporting to Excel

This one proves the full automation stack from Modules 8–10 working together — recruiters love it because it maps directly onto analyst/ops jobs. Architecture first, code second, four stages connected by clear functions:

1. Collect — requests + BeautifulSoup pull structured items (prices, headlines, listings) politely, with sleep and error guards.
2. Clean — normalise text, parse amounts with your Module 8 helper, drop invalid rows while counting them.
3. Analyse — aggregate with dicts or Counter: averages per category, changes since last run (store yesterday's JSON).
4. Report — openpyxl writes a formatted workbook: bold headers, column widths, a summary sheet plus raw-data sheet.

Make it configurable via argparse (target URL, output filename, --dry-run) and schedule-ready via logging. One deliberate stretch goal: run it daily via cron/Task Scheduler so the Excel file on disk is always current — an unattended pipeline is worth ten scripts that need hand-cranking.

12.3 Capstone C: Personal finance analyzer (files + charts)

The most relatable capstone — everyone understands money, which makes interviews easy. Export real transactions from your bank as CSV (most banks do), then build an analyzer around Modules 5–8:

import csv
from collections import defaultdict
from datetime import datetime

by_month = defaultdict(float)
with open("transactions.csv", newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        month = datetime.strptime(row["date"][:7], "%Y-%m")
        by_month[month.strftime("%Y-%m")] += float(row["amount"])

Layer on: regex-based category guessing from descriptions (Module 5 patterns like r"(?i)coffee|cafe|restaurant" → "Dining"), a monthly summary table, biggest expense per category, and a savings-rate calculation. For charts, matplotlib (pip install matplotlib) turns summaries into PNGs your README can embed — a bar chart of monthly spend is five lines and transforms how finished the project looks. Handle dirty bank exports explicitly: unknown date formats, missing categories, duplicate rows — cleaning narrative belongs in the README.

12.4 Reading job posts → mapping skills you now have

Job posts are skill checklists wearing prose. Pull up five junior postings that interest you and extract every concrete requirement into a spreadsheet; within minutes patterns emerge — certain skills repeat everywhere ("SQL", "REST APIs", "Git"), others are niche. Then map honestly against this course: functions/OOP/data structures → core language questions; requests/APIs/databases → Modules 9–10; Git/tests/project structure → Module 11; a deployed capstone → proof of all of it.

The gaps that remain are your personal curriculum — usually one or two technologies deep, not twenty, and learnable in weeks at this point because you now know how to learn technical material: docs, small experiments, projects. Rewrite your CV bullet points in the same vocabulary the postings use ("Built REST API with FastAPI and SQLite covering CRUD operations") — applicant-tracking systems literally match keywords, and honest alignment costs nothing.

12.5 Portfolio site basics and project write-ups

A portfolio doesn't need custom domains or frameworks — a GitHub profile with three excellent pinned repos outperforms ten mediocre ones. Each project needs a write-up that answers four questions in under a minute of reading:

• Problem: what annoyance or question does this solve? One sentence.
• Approach: architecture in two sentences plus one screenshot/GIF.
• Stack: libraries and why each earned its place.
• Result & lessons: what works, what you'd change, hardest bug fixed.

The "hardest bug" section is secret weapon material — it proves genuine engineering struggle rather than tutorial-following, and gives interviewers exactly the story they want to probe. If you want a website later, free static hosting on GitHub Pages suffices; but remember the repos are the portfolio, the site is just packaging.

12.6 Interview prep: common Python questions, live-coding drills

Python interviews mix knowledge checks with live coding. Drill these classics until answers are reflexive: mutable default arguments (3.2's trap); is vs == (identity vs equality); how dicts achieve O(1) lookups; what decorators are with one example; list comprehensions vs loops; shallow vs deep copies; exception handling philosophy; why x = [] then y = x aliases. Explain each aloud in under sixty seconds — articulating beats recognising.

Live coding is a separate skill practised separately: narrate your thinking before typing ("I'll use a dict to count, then find the max"), state assumptions, write the simple version first, test with a quick example, only then optimise. Standard drill sets include: reverse a string, FizzBuzz, two-sum, word frequency, palindrome check, fizz-buzz variants with dicts. Twenty minutes of daily drills for two weeks beats a weekend cram. And when stuck, think aloud anyway — interviewers grade reasoning under uncertainty as much as solutions.

12.7 Where next: web dev (Django/FastAPI), data (pandas/SQL), automation engineer

You're at a fork where all roads are good. Pick by taste, informed by which capstone felt most alive:

• Backend / web dev: deepen FastAPI (authentication, background tasks), then Django for batteries-included full apps; add PostgreSQL and Docker. Junior backend roles follow.
• Data / analytics: pandas properly, SQL seriously (it appears in nearly every posting), then dashboards (Plotly) and eventually statistics — the analyst-to-data-scientist ladder starts here.
• Automation / DevSecOps: scale scripting to infrastructure — Ansible, CI pipelines, cloud CLIs; your scraper-bot experience is the seed of this career path.