Understanding Python’s Iteration and Membership: A Information to __contains__ and __iter__ Magic Strategies

Picture by Creator

If you happen to’re new to Python, you will have come throughout the phrases “iteration” and “membership” and questioned what they imply. These ideas are elementary to understanding how Python handles collections of information, reminiscent of lists, tuples, and dictionaries. Python employs particular dunder strategies to allow these functionalities.

However what precisely are dunder strategies? Dunder/Magic strategies are particular strategies in Python that begin and finish with a double underscore, therefore the title “dunder.” They’re used to implement numerous protocols and can be utilized to carry out a variety of duties, reminiscent of checking membership, iterating over parts, and extra. On this article, we will likely be specializing in two of crucial dunder strategies: __contains__ and __iter__. So, let’s get began.

Understanding Pythonic Loops with Iter Methodology

Take into account a primary implementation of a file listing utilizing Python lessons as follows:

class File:
	def __init__(self, file_path: str) -> None:
    	    self.file_path = file_path
   	 
class Listing:
	def __init__(self, recordsdata: Record[File]) -> None:
    	    self._files = recordsdata

An easy code the place the listing has an occasion parameter that accommodates a listing of File objects. Now, if we wish to iterate over the listing object, we should always be capable of use a for loop as follows:

listing = Listing(
	recordsdata=[File(f"file_{i}") for i in range(10)]
)
for _file in listing:
	print(_file)

We initialize a listing object with ten randomly named recordsdata and use a for loop to iterate over every merchandise. Easy sufficient, However whoops! You get an error message: TypeError: ‘Listing’ object is just not iterable.

What went mistaken? ​​Properly, our Listing class is not set as much as be looped via. In Python, for a category object to turn out to be iterable, it should implement the __iter__ dunder technique. All iterables in Python like Record, Dictionaries, and Set implement this performance so we will use them in a loop.

So, to make our Listing object iterable, we have to create an iterator. Consider an iterator as a helper that provides us objects one after the other after we ask for them. For instance, after we loop over a listing, the iterator object will present us with the following factor on every iteration till we attain the tip of the loop. That’s merely how an iterator is outlined and carried out in Python.

In Python, an iterator should know the best way to present the following merchandise in a sequence. It does this utilizing a way referred to as __next__. When there aren’t any extra objects to present, it raises a particular sign referred to as StopIteration to say, “Hey, we’re completed right here.” Within the case of an infinite iteration, we don’t increase the StopIteration exception.

Allow us to create an iterator class for our listing. It can take within the listing of recordsdata as an argument and implement the following technique to present us the following file within the sequence. It retains observe of the present place utilizing an index. The implementation appears as follows:

class FileIterator:
    def __init__(self, recordsdata: Record[File]) -> None:
        self.recordsdata = recordsdata
        self._index = 0
    
    def __next__(self):
        if self._index >= len(self.recordsdata):
        	increase StopIteration
        worth = self.recordsdata[self._index]
        self._index += 1
        return worth

We initialize an index worth at 0 and settle for the recordsdata as an initialization argument. The __next__ technique checks if the index overflows. Whether it is, it raises a StopIteration exception to sign the tip of the iteration. In any other case, it returns the file on the present index and strikes to the following one by incrementing the index. This course of continues till all recordsdata have been iterated over.

Nonetheless, we aren’t completed but! We now have nonetheless not carried out the iter technique. The iter technique should return an iterator object. Now that now we have carried out the FileIterator class, we will lastly transfer in the direction of the iter technique.

class Listing:
    def __init__(self, recordsdata: Record[File]) -> None:
        self._files = recordsdata
    
    def __iter__(self):
        return FileIterator(self._files)

The iter technique merely initializes a FileIterator object with its listing of recordsdata and returns the iterator object. That is all it takes! With this implementation, we will now loop over our Listing construction utilizing Python’s loops. Let’s have a look at it in motion:

listing = Listing(
	recordsdata=[File(f"file_{i}") for i in range(10)]
)
for _file in listing:
	print(_file, finish=", ")

# Output: file_0, file_1, file_2, file_3, file_4, file_5, file_6, file_7, file_8, file_9,

The for loop internally calls the __iter__ technique to show this outcome. Though this works, you may nonetheless be confused in regards to the underlying workings of the iterator in Python. To grasp it higher, let’s use some time loop to implement the identical mechanism manually.

listing = Listing(
	recordsdata=[File(f"file_{i}") for i in range(10)]
)

iterator = iter(listing)
whereas True:
    attempt:
        # Get the following merchandise if obtainable. Will increase StopIteration error if no merchandise is left.
        merchandise = subsequent(iterator)   
        print(merchandise, finish=', ')
    besides StopIteration as e:
        break   # Catch error and exit the whereas loop

# Output: file_0, file_1, file_2, file_3, file_4, file_5, file_6, file_7, file_8, file_9,

We invoke the iter perform on the listing object to accumulate the FileIterator. Then, we manually make the most of the following operator to invoke the following dunder technique on the FileIterator object. We deal with the StopIteration exception to gracefully terminate the whereas loop as soon as all objects have been exhausted. As anticipated, we obtained the identical output as earlier than!

Testing for Membership with Accommodates Methodology

It’s a pretty widespread use case to test for the existence of an merchandise in a set of objects. For instance in our above instance, we might want to test if a file exists in a listing very often. So Python makes it less complicated syntactically utilizing the “in” operator.

print(0 in [1,2,3,4,5]) # False
print(1 in [1,2,3,4,5]) # True

These are majorly used with conditional expressions and evaluations. However what occurs if we do that with our listing instance?

print("file_1" in listing)  # False
print("file_12" in listing) # False

Each give us False, which is wrong! Why? To test for membership, we wish to implement the __contains__ dunder technique. When it isn’t carried out, Python fall backs to utilizing the __iter__ technique and evaluates every merchandise with the == operator. In our case, it’ll iterate over every merchandise and test if the “file_1” string matches any File object within the listing. Since we’re evaluating a string to customized File objects, not one of the objects match, leading to a False analysis

To repair this, we have to implement the __contains__ dunder technique in our Listing class.

class Listing:
    def __init__(self, recordsdata: Record[File]) -> None:
        self._files = recordsdata
    
    def __iter__(self):
        return FileIterator(self._files)
    
    def __contains__(self, merchandise):
        for _file in self._files:
        	# Test if file_path matches the merchandise being checked
        	if merchandise == _file.file_path:
            	return True
    	return False

Right here, we alter the performance to iterate over every object and match the file_path from the File object with the string being handed to the perform. Now if we run the identical code to test for existence, we get the right output!

listing = Listing(
	recordsdata=[File(f"file_{i}") for i in range(10)]
)

print("file_1" in listing)	# True
print("file_12" in listing) # False

Wrapping Up

And that’s it! Utilizing our easy listing construction instance, we constructed a easy iterator and membership checker to grasp the interior workings of the Pythonic loops. We see such design selections and implementations pretty usually in production-level code and utilizing this real-world instance, we went over the integral ideas behind the __iter__ and __contains__ strategies. Preserve training with these methods to strengthen your understanding and turn out to be a more adept Python programmer!
 
 

Kanwal Mehreen Kanwal is a machine studying engineer and a technical author with a profound ardour for information science and the intersection of AI with medication. She co-authored the e book “Maximizing Productiveness with ChatGPT”. As a Google Era Scholar 2022 for APAC, she champions variety and tutorial excellence. She’s additionally acknowledged as a Teradata Variety in Tech Scholar, Mitacs Globalink Analysis Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having based FEMCodes to empower ladies in STEM fields.