The lambda you no longer need

Three snippets from this blog’s own posts, run again on pandas 3 — and the shorter way to do the same job with pd.col().
pandas
Method Chaining
Lambda Functions
Data Wrangling
Author

Ricky Macharm

Published

September 25, 2026

A few of the posts on this blog lean on the same small trick: a function written inside a pandas chain, so that one step can see what the step before it just built. It works, it is short, and reading it does not tell you why it is there.

pandas 3 has a shorter way to write it. This post takes three snippets I published here, runs them again exactly as they were written, and then does the same jobs with pd.col() so the two can be put side by side. Every code block below runs when this page is built, and the output under it is what the code printed.

First, what a lambda is

A lambda is a function written where it is used, without a name:

double = lambda x: x * 2

def double_named(x):
    return x * 2

print(double(21), double_named(21))
42 42

Same job, two spellings. The only reason to reach for the first is that it fits inside another call, which is somewhere a def statement cannot go. That is the whole idea.

In pandas it turns up for a less obvious reason. When pandas runs a method, it works out the arguments first and then runs the method. Inside a chain, the frame that a step is about to produce does not exist yet at that moment. A lambda puts the calculation off until the step is actually running, when the frame does exist.

The trick, from the 2024 post

This is the one I wrote about in Lambda Functions: How I tend to use them, using the same CSV that post used. Here it is again, unchanged:

import pandas as pd

def tweak_data(url):
    return (pd.read_csv(url)
            .assign(Date=lambda x: pd.to_datetime(x['Date']),
                    Month=lambda x: x['Date'].dt.month_name()))

data = tweak_data('synthetic_stock_data.csv')
print(data.head(3).to_string(index=False))
print()
print(data.dtypes.to_string())
      Date       Close   Month
2023-01-02 3773.956049 January
2023-01-03 3438.878440 January
2023-01-04 3858.597920 January

Date     datetime64[us]
Close           float64
Month               str

Look at the second line. Month reads the Date column, and that column only becomes a date on the line above — inside the same assign call. Both lines are the same call, so nothing is committed until the call runs. Writing it without the lambda, as most people first try, does not work:

# the same lines without a lambda, the way most people first write it
raw = pd.read_csv('synthetic_stock_data.csv')
try:
    raw.assign(Date=pd.to_datetime(raw['Date']),
               Month=raw['Date'].dt.month_name())
except AttributeError as exc:
    print(type(exc).__name__, "-", exc)
AttributeError - Can only use .dt accessor with datetimelike values

The same post used a second pattern, a lambda handed to agg so that each group is reduced by my own calculation:

import numpy as np

print(data
 .groupby('Month')
 ['Close']
 .agg(lambda x: np.percentile(x, 25))
 .round(2)
 .to_string())
Month
April        3245.80
August       3268.60
December     3247.84
February     3226.91
January      3165.53
July         3331.57
June         3183.17
March        3323.71
May          3269.06
November     3290.07
October      3183.56
September    3240.97

What was going on

The lambda is not a loop, and that matters. It is called once, with the whole frame as its argument, and inside that call x['Date'] is a full column — the arithmetic runs on the column, at pandas speed. A lambda handed to .apply() row by row is a different animal entirely: that one is Python running per row and it is slow. The chain lambdas in these posts are the fast kind.

The reason they are needed at all is ordering. pandas evaluates the arguments of a method before it runs that method. In a chain, the value being built is not available to its own arguments yet. So the lambda is a way of saying calculate this later, when the frame is ready.

The same three jobs with pd.col

pandas 3 adds a second way to say exactly that. pd.col("name") is not the column; it is a stand-in for a column — the documentation calls it “a deferred object representing a column of a DataFrame”, and anywhere that accepts lambda df: df[col_name] will also accept pd.col(col_name).

Here are the three snippets again, written that way:

def tweak_data_v3(url):
    return (pd.read_csv(url, parse_dates=["Date"])
            .assign(Month=pd.col("Date").dt.month_name()))

new = tweak_data_v3('synthetic_stock_data.csv')
print(new.head(3).to_string(index=False))
print()
print("same result as the lambda version:", new.equals(data))
      Date       Close   Month
2023-01-02 3773.956049 January
2023-01-03 3438.878440 January
2023-01-04 3858.597920 January

same result as the lambda version: True

The column is named once — pd.col("Date") — instead of being named once for the frame and again inside the function body.

The second snippet needs a correction, and it is a useful one. That agg(lambda ...) was never doing the deferral job at all. agg hands each group to the function one at a time, so a lambda there is just a small function, and np.percentile with its own keyword does the same work. What the two lines below check is that the numbers are identical either way:

old_months = (data.groupby('Month')['Close']
              .agg(lambda x: np.percentile(x, 25)).round(2))
new_months = (data.groupby('Month')['Close']
              .agg(np.percentile, q=25).round(2))
print("quarterly percentiles identical:", old_months.equals(new_months))
print(new_months.to_string())
quarterly percentiles identical: True
Month
April        3245.80
August       3268.60
December     3247.84
February     3226.91
January      3165.53
July         3331.57
June         3183.17
March        3323.71
May          3269.06
November     3290.07
October      3183.56
September    3240.97
v3_bikes = (bikes
            .assign(dteday=pd.to_datetime(bikes['dteday']))
            .assign(mnth=pd.col("dteday").dt.month))

with_lambda = (bikes
               .assign(dteday=pd.to_datetime(bikes['dteday']),
                       mnth=lambda x: x['dteday'].dt.month))

print("month column identical:", v3_bikes["mnth"].tolist() == with_lambda["mnth"].tolist())
print("both month columns:", with_lambda["mnth"].tolist())
month column identical: True
both month columns: [1, 2, 3]

The third snippet is where pd.col() earns its place, because there the lambda really was postponing the calculation. Nothing is delayed with pd.col("dteday"), so the line reads in the order it runs, and the conversion is separated from the step that uses its result — which is what makes the failure in the earlier section impossible to write by accident.

Where pd.col does not help

It is not a replacement for every lambda in those posts. The rename(columns=...) lambda has no equivalent, because rename is not working on a frame’s contents — it is working on the labels:

try:
    karachi.rename(columns=pd.col("property type"))
except TypeError as exc:
    print(type(exc).__name__, "-", " ".join(str(exc).split()))
TypeError - boolean value of an expression is ambiguous

The rule of thumb: if the argument is about a column of the frame, pd.col() can probably write it. If it is about the column labels, or about rows one at a time, it cannot, and the lambda stays. The release notes for 3.0 call this “initial support” and name three places it works — assign, loc, and setting by column — so it is early.

What else changed in pandas 3

Two things worth knowing before the next run of an old notebook, in case the reader hits them at the same time:

print("a string column now reports:", pd.Series(["a", "b"]).dtype)
print("the old default was:", pd.Series(["a", "b"], dtype=object).dtype)
a string column now reports: str
the old default was: object
import warnings

df = pd.DataFrame({"a": [1, 2, 3], "b": [10, 20, 30]})
with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    df["a"][df["b"] > 15] = 99

print("warning:", caught[0].category.__name__ if caught else "none")
print("the write did not land:", df.to_dict("records"))
warning: ChainedAssignmentError
the write did not land: [{'a': 1, 'b': 10}, {'a': 2, 'b': 20}, {'a': 3, 'b': 30}]

Text columns no longer report as object — they report as str. And a two-step write, df["a"][df["b"] > 15] = 99, no longer writes anything: pandas warns and the frame is left alone. The single-step df.loc[df["b"] > 15, "a"] = 99 is what works now.

Where to read more

Back to top