Python Language Regular Expressions (Regex) Match an expression only in specific locations


Often you want to match an expression only in specific places (leaving them untouched in others, that is). Consider the following sentence:

An apple a day keeps the doctor away (I eat an apple everyday).

Here the "apple" occurs twice which can be solved with so called backtracking control verbs which are supported by the newer regex module. The idea is:

forget_this | or this | and this as well | (but keep this)

With our apple example, this would be:

import regex as re
string = "An apple a day keeps the doctor away (I eat an apple everyday)."
rx = re.compile(r'''
    \([^()]*\) (*SKIP)(*FAIL)  # match anything in parentheses and "throw it away"
    |                          # or
    apple                      # match an apple
    ''', re.VERBOSE)
apples = rx.findall(string)
# only one

This matches "apple" only when it can be found outside of the parentheses.

Here's how it works:
  • While looking from left to right, the regex engine consumes everything to the left, the (*SKIP) acts as an "always-true-assertion". Afterwards, it correctly fails on (*FAIL) and backtracks.
  • Now it gets to the point of (*SKIP) from right to left (aka while backtracking) where it is forbidden to go any further to the left. Instead, the engine is told to throw away anything to the left and jump to the point where the (*SKIP) was invoked.