Python Beginner Tutorial: for Loops and Iterators

Introduction and the Anatomy of a For Loop

So we'll start off with some basics so everyone is on the same page. The basic anatomy of a for loop is a little something like:

for x in y:
    do_stuff(x,y)

• do_stuff(x,y) is a function call. If you don't know how functions work yet, it'll be worth looking that up before continuing here
• y is an iterable. We'll get into the details of what that means in a little bit. For now, an iterable is anything you can iterate over (don't panic)
• x is a dummy variable and can come in a few different forms

Here are a few examples of for loops that should cement the basic concepts:

Range

range is pretty useful if we want to do something some integer number of times

for i in range(4):
    print i

This outputs:

0
1
2
3

This example uses range in its simplest form. For more info about how range works type the following at the python prompt:

help(range)

Strings

Plain old strings are also iterable.

for s in "hello":
    print s

This outputs:

h
e
l
l
o

Lists

These are probably the most obvious of iterables.

for x in [None,3,4.5,"foo",lambda : "moo",object,object()]:
    print "{0}  ({1})".format(x,type(x))

This outputs:

None  (<type 'NoneType'>)
3  (<type 'int'>)
4.5  (<type 'float'>)
foo  (<type 'str'>)
<function <lambda> at 0x7feec7fa7578>  (<type 'function'>)
<type 'object'>  (<type 'type'>)
<object object at 0x7feec7fcc090>  (<type 'object'>)

This may look complicated but the take home message is that a list can contain anything, and has an implied order.

Tuples

Tuples vary from lists in some fundamental ways which are beyond the scope of this tutorial. You'll notice that the iterable in the following example uses round brackets instead of square, and that the output is the same as the output for the list example above.

for x in (None,3,4.5,"foo",lambda : "moo",object,object()):
    print "{0}  ({1})".format(x,type(x))

This outputs:

None  (<type 'NoneType'>)
3  (<type 'int'>)
4.5  (<type 'float'>)
foo  (<type 'str'>)
<function <lambda> at 0x7feec7fa7578>  (<type 'function'>)
<type 'object'>  (<type 'type'>)
<object object at 0x7feec7fcc090>  (<type 'object'>)

Dictionaries

A dictionary is an unordered list of key value pairs. When you iterate over a dictionary using a for loop, your dummy variable will be populated with the various keys.

d = {
  'apples' : 'tasty',
  'bananas' : 'the best',
  'brussel sprouts' : 'evil',
  'cauliflower' : 'pretty good'
}

for sKey in d:
  print "{0} are {1}".format(sKey,d[sKey])

This outputs:

brussel sprouts are evil
apples are tasty
cauliflower are pretty good
bananas are the best

But maybe not quite in that order. I repeat: Dictionaries are unordered!

For Loops Advanced: Using break and continue

Sometimes it is necessary to stop a loop. This is achieved using the break statement. Here is a simple example:

for i in range(4000000000):
        print i
        if i>=2:
            break

This will output:

0
1
2

The break statement looks at the nearest containing loop end causes it to exit. It works for while loops as well. Here is another example showing how it would work with nested loops

for ii in range(2):
    print "outer loop iteration"
    for i in range(4000000000):     # inner loop
        print i
        if i>=2:
            break

This outputs:

outer loop iteration
0
1
2
outer loop iteration
0
1
2

Another common scenario is needing to skip over the rest of the current loop iteration but stay in the loop. This is achieved with the continue statement. Like the break statement, it also looks just at the nearest containing loop. Here's an example:

iSum = 0
    iProduct =1

for i in range(5):
    print i
    if i%2==0:           # we aren't interested in the even numbers, skip those
        print "even" 
        continue 
    else:
        print "odd"
    print "calculating"
    iSum = iSum + i
    iProduct = iProduct * i

print "========="
print iSum
print iProduct

This outputs:

0
even
1
odd
calculating
2
even
3
odd
calculating
4
even
=========
4
3

For Loops Behind the Scenes

So what is an iterable anyway? Well, the short answer is, anything with an __iter__ function. The __iter__ function is one of those special functions that Python relies on. Type this stuff at the python prompt and see what happens:

'__iter__' in dir([1,2,3])           # returns True
'__iter__' in dir((1,2,3))           # returns True
'__iter__' in dir(range(3))      # returns True
'__iter__' in dir(3)                   # returns False

So 3, an integer, is not an iterable. Try this:

for i in 3:
    print i

This gives us the error:

TypeError: 'int' object is not iterable

Ok. So __iter__ is important. What does it do? Well, it returns an iterator.

Here's a really simple example loop that we will be using for the rest of this discussion:

my_list = [1,2,3]
for i in my_list:
    print i

Here's what it outputs:

1
2
3

So that was pretty simple. Let's see if we can get the same output without using a for loop (assuming we don't know what is in my_list beforehand). This is where iterators come in.

my_iterator = my_list.__iter__()

my_iterator is a listiterator object. That sounds pretty fancy but don't be intimidated. Soon you will be able to create your own iterators.

Now, my_iterator has a number of functions. The one we are interested in is called next. Try this out:

while True:
    print my_iterator.next()  

This prints out something like this:

1
2
3
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
 StopIteration

Which is almost what we want. But now my_iterator seems to be broken: It has reached the end of the list and can't go any further.

Here's a little summary of what we have learned to far:

• For loops loop over iterables.
• iterables have __iter__ functions
• __iter__ functions return Iterators
• Iterators make use of the next method to move from element to element within their associated iterable
• once an iterator runs out of things to return from the next function it raises the StopIteration exception whenever next is called

Now let's make a function that works just like the for loop at the beginning of this section:

def my_for_loop(some_iterable):
    oIter = some_iterable.__iter__()
    while True:
        try:
            print oIter.next()
        except StopIteration:
            break

And let's try it out:

my_for_loop([1,2,3])

This outputs:

1
2
3

Just what we wanted. Play with it a little, see what happens if you pass in the iterables from the introductory examples of this tutorial.

Using __iter__ to Make Iterators from Your Objects

And finally, the cool stuff. We will now make our own objects iterable. I'm going to detail the approach I usually use. I use it because it is simple and concise. I use the same class for both the iterator and the iterable. This is useful if you only need one iterator for your class (which is most of the time)

class FibIterable:
    """
    this class is a generates a well known sequence of numbers 
    """
    def __init__(self,iLast=1,iSecondLast=0,iMax=50):
        self.iLast = iLast 
        self.iSecondLast = iSecondLast
        self.iMax = iMax  #cutoff
    def __iter__(self):
        return self    # because the object is both the iterable and the itorator
    def next(self):
        iNext = self.iLast + self.iSecondLast
        if iNext > self.iMax:
            raise StopIteration()
        self.iSecondLast = self.iLast
        self.iLast = iNext
        return iNext

o = FibIterable()
for i in o:
    print(i)

This prints:

1
2
3
5 
8
13
21
34

• So to summarise what happened here, we initialised an object , o of class FibIterable.
• The for loop called o.__iter__ which just returnedo
• For each iteration of the for loop, the loop called o.next() which calculated the next value in the sequence
• if the next value was too high then the next raised a StopIteration in order to break out of the loop
• otherwise the changed state of o was stored and the correct next value was returned.

This approach of doing things has a few pros that I have already mentioned: it is concise and simple. But there are instances where you might want two iterators over the same data. If, following from above, we again run the code:

for i in o:
    print(i)

Nothing will be printed. We would have to reinitialise the class by setting o.iLast ando.iSecondLast to their original values. While this is a definite weakness the solution is trivial and left as an exercise.

Conclusion

Well done for getting this far. Knowing the ins and outs of for loops and iterators makes a lot of things much easier in Python. Now you should be in a position that you really get the for loop and its internal mechanisms. You can write for loops for iterating over many kinds of Python's build in iterables and, if that's not enough, you can even make your own iterables.

If your code is very loopy and deals with a lot of collections then these skills will be indespensable and I would suggest continuing your education by learning about Python generators, list comprehension and sorting mechanisms. Also the itertools module should be of interest. On the other hand, if your code tends to involve tree or graph type structures and algorithms I would suggest looking into recursion. An elegant recursive function can replace many lines of nested loops.