PHASE 15Intermediate ~28 min· topic 8 of 10

Topic 15.8

Automation Scripts: Files, Dates and Subprocesses

In one line

Automation scripts do boring work reliably: they rename and move files with pathlib, handle dates and time zones correctly with datetime and zoneinfo, and run other programs with subprocess.run using a list of arguments, a timeout and a return-code check, never a shell built from user input.

Think of it like this

A mail room at the end of each day. A clerk takes every parcel from the in-tray, writes the date on it, files it in the right shelf, and phones the courier when something is missing. It is dull and the same every day, which is exactly why a script should do it. Automation is teaching Python to do the dull, repeatable jobs without mistakes.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Automation: Making a program do a repetitive job so people do not have to.
pathlib: Python's standard module for working with file and folder paths as objects.
Dry run: Running a script so it only reports what it would do, without changing anything.
Idempotent: Safe to run more than once: the second run changes nothing more.
Naive / aware datetime: A datetime without a time zone, and one that knows its offset from UTC.
UTC: The world's reference time, with no daylight saving. Store times in UTC.
IANA time zone: A named zone from the shared database, such as Asia/Kolkata or Europe/London.
Subprocess: Another program started by your program.
Return code: The exit number of a finished program. 0 means success.
Shell injection: Running unintended commands because untrusted text was given to a shell.

Step by step

01List, glob and inspect files

A Path knows its parts. path.name is the full file name, path.stem is the name without the last suffix, path.suffix is the dot-extension, and path.parent is the folder. / joins paths in a way that works on every operating system.

main.pywhole filepython

from pathlib import Path

inbox = Path("inbox")
for path in sorted(inbox.glob("*.txt")):
    print(path.name, path.stem, path.suffix, path.stat().st_size)

target = Path("archive") / "report.txt"

02Rename safely in a loop

Build the new name, check the target does not exist, then rename. The runnable example normalises names (lower case, spaces to underscores) and skips clashes. It creates its own fake files so it can run anywhere.

Rename safely in a loopdiagram

Rendering diagram…

03Aware datetimes and time zones

Create times in UTC (tzinfo=timezone.utc), and use astimezone(ZoneInfo(...)) to show them in another zone. The offset changes with daylight saving automatically. In the example, London is +00:00 on 28 March 2026 and +01:00 the next day, because the clocks went forward on 29 March.

04Naive and aware do not mix

Subtracting a naive datetime from an aware one raises TypeError: can't subtract offset-naive and offset-aware datetimes. Python refuses to guess which zone the naive time meant. Make both aware before comparing.

05Run a program safely

Pass a list, capture the output, set a timeout, and check the code. sys.executable is the path of the Python that is running your script, a safe way to start another Python.

main.pywhole filepython

import subprocess
import sys

result = subprocess.run(
    [sys.executable, "-c", "print('hello from a child process')"],
    capture_output=True, text=True, timeout=10, check=True,
)
print(result.stdout)

06What can go wrong

CalledProcessError means the program ran and returned a non-zero code (with check=True). TimeoutExpired means it did not finish in time (subprocess kills it). FileNotFoundError means the program itself was not found on the PATH. Each deserves its own except branch in a real script.

07The unsafe shape to avoid

The shape to avoid is a single string passed to the shell with data pasted in: subprocess.run(f"convert {filename} out.png", shell=True). A file name containing shell characters becomes part of the command. The safe shape passes the same data as one list item, which the program receives as a plain argument: subprocess.run(["convert", filename, "out.png"], check=True).

08Scheduling

Python is not the scheduler. Use cron or Task Scheduler on a machine, or a scheduled CI job, to start the script. The cron line below runs it every day at 02:00 server time. Always log to a file or stdout, and make failures loud (non-zero exit code).

crontabwhole filebash

# minute hour day-of-month month day-of-week  command
0 2 * * * /usr/bin/python3 /opt/tools/archive_inbox.py >> /var/log/archive_inbox.log 2>&1

Try it yourself

1
Add a dry run
In the rename example add dry_run = True. When it is true, print the planned move instead of calling path.rename. Run it and check that inbox still has all four files.
2
Change the time zone
In the dates example change the meeting to 2026-10-25 00:30 UTC and look at London (the clocks go back that night) and New York. Predict the offsets first: London is +01:00 before the change and +00:00 after.
3
Make a naive/aware mistake
Create datetime(2026, 10, 10, 9, 0) (no tzinfo) and subtract it from the aware now. Read the TypeError, then fix it by adding tzinfo=timezone.utc.

Code & diagrams

Batch-rename files with pathlib New tab

The program creates its own fake inbox first. Each run starts in a fresh empty folder, so it is repeatable.

Expected output

Report Feb.txt -> archive/report_feb.txt
Report Jan.TXT -> archive/report_jan.txt
notes.md -> archive/notes.md
photo 1.JPG -> archive/photo_1.jpg
inbox now: []
text files: ['report_feb.txt', 'report_jan.txt']
all sizes: {'notes.md': 13, 'photo_1.jpg': 16, 'report_feb.txt': 19, 'report_jan.txt': 19}

Dates, UTC and time zones New tab

All dates are fixed, so the output never changes. Look at the daylight-saving line: one calendar day later is only 23 real hours later.

Expected output

UTC:      2026-03-29T09:30:00+00:00
Asia/Kolkata      2026-03-29T15:00:00+05:30
Europe/London     2026-03-29T10:30:00+01:00
America/New_York  2026-03-29T05:30:00-04:00
London 2026-03-28T12:00:00+00:00 -> 2026-03-29T12:00:00+01:00
real hours between: 23.0
10 Oct 2026, 02:45 UTC
next run: 2026-10-11T02:00:00+00:00 in 17:45:00

subprocess.run: capture, check and time out (cannot run in the browser)python

Not runnable here: the browser cannot start other processes. This output is from a real run on Python 3.12.

import subprocess
import sys

result = subprocess.run(
    [sys.executable, "-c", "print('hello from a child process')"],
    capture_output=True,
    text=True,
    timeout=10,
    check=True,
)
print("return code:", result.returncode)
print("stdout:", repr(result.stdout))
print("stderr:", repr(result.stderr))

try:
    subprocess.run([sys.executable, "-c", "import sys; sys.exit(3)"], check=True)
except subprocess.CalledProcessError as e:
    print("failed with code", e.returncode)

try:
    subprocess.run([sys.executable, "-c", "import time; time.sleep(5)"], timeout=0.5)
except subprocess.TimeoutExpired as e:
    print("timed out after", e.timeout, "seconds")

try:
    subprocess.run(["no-such-program-xyz"])
except FileNotFoundError as e:
    print("program not found")

Output

return code: 0
stdout: 'hello from a child process\n'
stderr: ''
failed with code 3
timed out after 0.5 seconds
program not found

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Subtract a naive time from an aware time

Mix a datetime with no time zone and one with a time zone.

main.pywhole filepython

from datetime import datetime, timezone

started = datetime(2026, 10, 10, 9, 0)
now = datetime(2026, 10, 10, 9, 30, tzinfo=timezone.utc)
print(now - started)

terminal

$ python main.py

── what you'll see ──

Traceback (most recent call last):

File "main.py", line 5, in <module>

print(now - started)

~~~~^~~~~~~~~

TypeError: can't subtract offset-naive and offset-aware datetimes

Myth vs fact

Myth

shell=True is fine because I only use my own strings.

Fact

Strings often contain data you did not write: file names, user input, web content, model output. Keeping shell=False and a list of arguments removes the whole class of problem.

Myth

A day always has 24 hours.

Fact

Not in a zone with daylight saving: some days have 23 or 25 hours. Do arithmetic in UTC and convert for display.

Myth

datetime.now() gives me UTC.

Fact

It gives naive local time. Use datetime.now(timezone.utc) for an aware UTC time.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Prefer Python libraries to shelling out: use shutil, zipfile, httpx or sqlite3 instead of calling cp, zip, curl or the sqlite3 command. They work on every platform, give real exceptions, and need no quoting.
▸
Read a long-running child's output line by line with subprocess.Popen(..., stdout=PIPE, text=True) and iterate over proc.stdout; run(capture_output=True) collects everything in memory first. Use asyncio.create_subprocess_exec (Topic 12.6) to run several at once.
▸
On Windows some tools are .cmd/.bat files or built into cmd.exe and do not behave like programs found by name; shutil.which("tool") shows what would actually run. Always test automation on the operating system it will run on.
▸
Subprocess details that bite in production: text=True decodes with the locale encoding, so pass encoding="utf-8" when you know the child's output. capture_output=True plus a very chatty child holds everything in memory. With Popen and both stdout=PIPE and stderr=PIPE, read with communicate(), because reading one pipe while the other fills up can deadlock the child. By default the child inherits your whole environment, so pass env={"PATH": ..., ...} with only what it needs when it should not see your API keys (Topic 15.3). On timeout, run kills the child but not any grandchildren it started.
▸
Defensive detail on shell=False: it removes shell injection, but not argument injection. If a user-supplied value is passed to a program that treats a leading dash as an option (["git", "log", user_value] with --output=...), the value can still change behaviour. Put "--" before user values where the program supports it, validate against an allow-list, and prefer a library to a command-line tool. On Windows, a list is joined into one command-line string, and .bat/.cmd files are re-parsed by cmd.exe, so untrusted values passed to a batch file can still be interpreted. Avoid running batch files with untrusted arguments.
▸
Dates in production: fromisoformat accepts a trailing Z only from Python 3.11, and the browser here runs 3.13, but a script that must also run on 3.10 needs .replace("Z", "+00:00"). A wall-clock time that happens twice when clocks go back is distinguished by fold=0 or fold=1, and one that does not exist (clocks go forward) is accepted silently by datetime, so reject local times for schedules and store UTC. A cron job at 02:00 local time can run twice or not at all on a daylight-saving night, so run servers and schedulers in UTC.
▸
AI agents that run commands are subprocess users too. Give them an allow-list of programs and arguments, a timeout, a working directory, and ask for human approval for anything destructive (Topic 16.10).

Remember this

1
Files. pathlib.Path (Topic 7.6) makes file work readable: Path("inbox").iterdir() lists a folder, .glob("*.txt") finds matches (and .rglob searches subfolders), .stem and .suffix split a name, .rename(target) moves, .mkdir(parents=True, exist_ok=True) creates folders, .read_text()/.write_text() read and write, shutil copies whole trees. Sort listings (sorted(path.iterdir())) because folder order is not guaranteed, and check before overwriting: rename may silently replace an existing file on some systems.
2
Make destructive scripts safe by design: support a --dry-run that only prints what would happen, never delete when you can move to an archive folder, make the script idempotent (running it twice does not break things), and log what it did. A script that renames 10,000 files wrongly in two seconds is the worst kind of fast.
3
Dates and times. Recall from Topic 7.11: datetime objects can be *naive* (no time zone: "9:30" on an unknown clock) or *aware* (carry a UTC offset, the number of hours the local clock is ahead of or behind UTC). Mixing them raises TypeError. The professional rule: store and compute in UTC (timezone.utc), and convert to a person's zone only when showing it. zoneinfo.ZoneInfo("Asia/Kolkata") provides the real IANA time-zone database, including daylight saving rules. timedelta is a length of time. Parse and print with fromisoformat and isoformat, which use the unambiguous standard form 2026-10-10T08:15:00+05:30.
4
Time zones have traps. A day is not always 24 hours (in a zone that changes its clocks, a "day later" at the same wall-clock time can be 23 or 25 real hours later). Local times can repeat or not exist when clocks change. Never use naive local times for deadlines, schedules or logs. datetime.now() is naive; use datetime.now(timezone.utc) (your scripts will still need the clock; the examples here use fixed dates so they print the same every time).
5
Running other programs. subprocess.run([...]) starts another program and waits. Pass the command as a list of strings (["git", "status"]), not one string. Useful options: capture_output=True, text=True to collect output as text, check=True to raise CalledProcessError on a non-zero exit code, and **timeout=** so a hung program cannot hang yours. The result has returncode, stdout and stderr.
6
Safety rule: never build a shell command from untrusted text. With shell=True the string goes through the shell, which interprets characters like ;, &, | and $(). If any part comes from a user, a file name, a web request or an AI model, someone can make the shell run something else. The defence is to keep the default shell=False and pass a list: each item is one argument, handed to the program exactly as is, with no shell to interpret it. Validate or allow-list anything that chooses *which* program runs. Scheduling is the last piece: use the system scheduler (cron on Linux/macOS, Task Scheduler on Windows, or a CI schedule) to run the script on time, and make the script log and exit with a proper code (Topic 15.7). Note: subprocess does not work in the browser, so those blocks below are shown, not run.
7
A shell is the program that reads the commands you type in a terminal (bash, PowerShell). You used one when you ran pip install. With subprocess your Python program types commands for you.

Explain it without notes

How do you make a file-renaming script safe?

What is the difference between naive and aware datetimes, and which should you store?

Why is subprocess.run(command_string, shell=True) dangerous with untrusted input, and what is the safe alternative?

What do check=True and timeout= do in subprocess.run?

Practice

Write archive_name(filename) that returns the lower-case name with spaces replaced by underscores, keeping the extension.

Write to_utc_string(text) that takes an ISO time with an offset such as 2026-10-10T08:15:00+05:30 and returns it in UTC in the same format.

Write (non-runnable) a function git_branch() that returns the current git branch using subprocess.run with a list, check=True and a timeout.

Trade-offs

↔
Python versus a shell script. Shell is shorter for chaining programs; Python is better when you need logic, error handling, dates, JSON, tests and portability.
↔
A dry-run mode versus simpler code. It costs a few lines but turns a risky script into one you can try safely. Worth it for anything that moves or deletes.

Done when you can

Done when you can list, glob, rename and move files with pathlib, and check for clashes first.
Done when you can add a dry-run mode and explain idempotence.
Done when you store times in UTC and convert with ZoneInfo for display.
Done when you can explain the naive versus aware TypeError.
Done when you run programs with a list, check=True and a timeout, and can explain why shell=True with user input is unsafe.

Command Palette

Words you'll meet

Step by step

01List, glob and inspect files

02Rename safely in a loop

03Aware datetimes and time zones

04Naive and aware do not mix

05Run a program safely

06What can go wrong

07The unsafe shape to avoid

08Scheduling

Try it yourself

Add a dry run

Change the time zone

Make a naive/aware mistake

Code & diagrams

Break it on purpose

Subtract a naive time from an aware time

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can