Lessons · Python · in: list vs set
Where 'in' is fast
x in a_set is a direct lookup; x in a_list checks items one by one, so it slows down as the list grows.
Hone is a place to practise programming. This is one of its lessons, written out in full and free to read without an account.
What it is for
Is this email already registered, is this id on the blocklist, have we seen this row: thousands of checks against thousands of items is millions of comparisons with a list and a blink with a set.
How to think about it
If you test membership more than a couple of times, build a set once (seen = set(ids)) and test against that. Dict keys behave like a set too.
Worked example
ids = list(range(100000))A long list.
ids_set = set(ids)Built once.
print(99999 in ids)True, after walking 100000 items.
print(99999 in ids_set)True, after one hash lookup.
print(-1 in ids_set)False, still one lookup: a miss costs the same.
Your turn
Fast repeated checks against the blocklist.
blocked = (rows)
Solve one with the tests running
The trap
Building a set costs a pass over the data, so for a single check a list is fine. The win comes from checking many times.