Grouping and Summarizing Data

Part 2 introduced totals, counts, and best-so-far values. You can now keep one such result for each category instead of one result for an entire list.

The main work is deciding what a key represents and what should be accumulated under it. The ownership question from Chapter 5 also matters: when dictionary values are mutable summary records or lists, each category must receive its own newly created value.

From Repeated Scanning to One Pass

One way to calculate a total for each subject is to first find the subjects and then scan all sessions again for every subject:

def totals_by_repeated_scanning(sessions):
    subjects = []

    for session in sessions:
        subject = session["subject"]
        if subject not in subjects:
            subjects.append(subject)

    results = []
    for subject in subjects:
        total = 0
        for session in sessions:
            if session["subject"] == subject:
                total = total + session["minutes"]
        results.append({"subject": subject, "total": total})

    return results

This returns a list of dictionary records, each containing a subject and its total. It preserves the order in which subjects first appear. Its structure is indirect, however: collect the categories, then repeatedly search for their records.

A dictionary of running totals can represent the question more directly. Each subject is a key, and the value under that key is its running total. A single pass can update the appropriate total as soon as it sees a session. This version returns the dictionary of totals, so the result has a different shape from the list above. Compare how each approach finds and updates a subject’s total.

Totals for Each Subject

sessions = [
    {"subject": "Programming", "minutes": 25},
    {"subject": "Math", "minutes": 40},
    {"subject": "Programming", "minutes": 15}
]


def totals_by_subject(sessions):
    totals = {}

    for session in sessions:
        subject = session["subject"]

        if subject not in totals:
            totals[subject] = 0

        totals[subject] = totals[subject] + session["minutes"]

    return totals


assert totals_by_subject(sessions) == {"Programming": 40, "Math": 40}
assert totals_by_subject([]) == {}

Compare this with the word counter from Chapter 3. The dictionary setup is the same. The difference is what the loop adds: minutes rather than one occurrence.

The records need not be next to other records with the same subject. The key connects each session with the correct accumulated total wherever the session appears.

A category may need both a count and a total. Trace the following programs, then use those ideas to summarize a list of activities.

To combine these ideas correctly, create a fresh {"count": 0, "minutes": 0} record inside the missing-category condition. Then update the record stored under the current category. Creating one summary before the loop and storing it under several keys would make the categories share their counts.

Keeping the Individual Values

Sometimes we need each session length for a subject, rather than just their total. A dictionary can keep a list under each subject key:

def minutes_by_subject(sessions):
    grouped = {}

    for session in sessions:
        subject = session["subject"]
        if subject not in grouped:
            grouped[subject] = []
        grouped[subject].append(session["minutes"])

    return grouped


sessions = [
    {"subject": "Programming", "minutes": 25},
    {"subject": "Math", "minutes": 40},
    {"subject": "Programming", "minutes": 15}
]
print(minutes_by_subject(sessions))

The output is:

{'Programming': [25, 15], 'Math': [40]}

The first session for a subject creates an empty list. Each session then appends its minutes to that subject’s list. The values keep their order within each group, including repeated lengths.

Each new key gets its own list. Appending to the Programming list therefore does not change the Math list. The original session records also stay unchanged.

Step through the grouping loop with these three sessions. Watch a separate list appear for each new subject, then follow the third session back to the existing Programming list.

Before moving on, suppose the input contains a fourth session: {"subject": "Math", "minutes": 10}. Predict the complete result dictionary. Would this iteration create another list, or which existing list would it change? Explain why the Programming group would stay unchanged.

Connect this back to category_summary: each new category also needs its own mutable value, but there that value is a summary dictionary rather than a list. In both cases, the missing-key condition creates the value once, and later records update the value already stored under that key.

Ordering Results Deliberately

Dictionary iteration follows insertion order. A report may instead need alphabetical order:

totals = {"Programming": 40, "Math": 40}

for subject in sorted(totals):
    print(f"{subject}: {totals[subject]}")

sorted(...) returns a new sorted list. When given a dictionary, it sorts its keys. The output is:

Math: 40
Programming: 40

The original dictionary is not rearranged. For the simple consistently capitalized names used here, the default string ordering gives the report order we want. Sorting arbitrary human-language names raises additional questions that this exercise does not need to solve.

Sorting Records Needs a Rule

Python cannot infer which field should determine the order of several dictionaries. Supply a key function: a function that receives one record and returns the value to compare.

The named function uses nothing new internally. The new idea is passing the function itself to another operation:

key=session_minutes

There are no call parentheses after session_minutes. sorted calls that function for records whose comparison values it needs. Writing session_minutes() would instead try to call it immediately without its required record.

key=... is a keyword argument. It supplies an argument by parameter name rather than by position. You can also request descending order:

ordered = sorted(sessions, key=session_minutes, reverse=True)

This places larger minute values first. The input records remain unchanged.

A Sorted List Is Still a Shallow Result

sorted(sessions, ...) creates a new outer list containing the existing records. It does not copy each dictionary. The list order is independent; the records remain shared.

When two records have equal sorting keys, their original relative order is preserved. A contract can use that rule deliberately:

Sort by minutes from largest to smallest. Keep input order when minutes are equal.

A different tie rule would need to be stated and implemented separately.

Debugging a sort is also a contract-reading exercise. Diagnose the value produced by the key function separately from the direction requested from sorted.

Check how the key function and sort direction determine the order, including ties. Explain which objects the sorted result shares with its input.

Check Your Understanding

1. How does grouping totals differ from counting occurrences?
2. Why does the grouping loop work when a category appears again later?
3. What does sorted(a_dictionary) sort?
4. Why is the key function passed without call parentheses?
5. What stays shared after sorting a list of records?