Lessons · Regex · matching bytes rather than text
Bytes are not text
A bytes pattern matches bytes and a str pattern matches str; re refuses to mix them, and in bytes mode \w is ASCII-only.
Hone is a place to practise programming. This is one of its lessons, written out in full and free to read without an account.
What it is for
Framing a protocol, or finding a marker in a file you would rather not decode. Both are real, and both are about bytes.
How to think about it
If the question is about words a person wrote, decode first. If it is about the bytes on the wire, keep them bytes and write rb'...'.
Worked example
re.findall(rb'\w+', 'caf\xe9'.encode())In bytes mode there is no Unicode to consult.
re.findall(r'\w+', 'caf\xe9')Decoded first, the answer is about characters.
isinstance(re.findall(rb'\w+', b'ab')[0], bytes)What comes back matches what went in.
Your turn
Match words in a bytes value.
re.findall('\w+', 'caf\xe9'.encode())Test a pattern against real text
The trap
Mixing them raises TypeError rather than guessing an encoding. That refusal is the feature.
Practise matching bytes rather than text on HoneA question on it now, a coding challenge where there is one, and it is remembered for review. Free, no email needed.