Understanding Iterable and Iterator in Python

[Rajeev Bagra]

Whenever we write a simple Python loop like:

for item in data:
    print(item)

Python internally performs a sophisticated process involving:

• iterables
• iterators
• the iterator protocol

Understanding these concepts helps developers deeply understand how Python handles:

• for loops
• lists
• files
• generators
• streaming data
• machine learning datasets
• large-scale processing systems

---

What Is an Iterable?

An iterable is:

An object that can be looped over.

Examples of iterables:

[1, 2, 3]

("a", "b", "c")

"hello"

{"x": 1, "y": 2}

{1, 2, 3}

All these objects can be used inside a loop:

numbers = [10, 20, 30]

for n in numbers:
    print(n)

---

Why Are They Called Iterables?

Because they implement the special method:

__iter__()

This method tells Python:

“How iteration should begin.”

An iterable does not usually produce values directly.

Instead, it creates and returns an iterator object.

---

What Is an Iterator?

An iterator is:

An object that produces values one-by-one during iteration.

An iterator remembers:

• current position
• what item comes next
• when iteration should stop

Iterators implement two important methods:

__iter__()
__next__()

---

The Difference Between Iterable and Iterator

Concept	Meaning
Iterable	An object you can loop through
Iterator	The object that produces next values
Iterable stores data	Iterator traverses data
Usually implements __iter__()	Implements __next__()

---

How Python Internally Processes a Loop

When Python sees:

for x in numbers:
    print(x)

It internally performs something similar to:

iterator = numbers.__iter__()

while True:

    try:
        item = iterator.__next__()
        print(item)

    except StopIteration:
        break

This is the hidden engine behind Python iteration.

---

Converting an Iterable into an Iterator

Python provides the built-in function:

iter()

Example:

numbers = [1, 2, 3]

iterator = iter(numbers)

Internally this calls:

numbers.__iter__()

Now iterator becomes an iterator object.

---

Using next()

The built-in function:

next()

retrieves values one-by-one from an iterator.

Example:

numbers = [1, 2, 3]

iterator = iter(numbers)

print(next(iterator))

Output:

1

Again:

print(next(iterator))

Output:

2

Again:

print(next(iterator))

Output:

3

Another call:

print(next(iterator))

raises:

StopIteration

---

What Is StopIteration?

When no values remain, iterators raise:

raise StopIteration

This tells Python:

“The iteration is finished.”

Without this exception, loops would continue forever.

---

Simple Custom Iterator Example

class Counter:

    def __init__(self, limit):
        self.limit = limit

    def __iter__(self):

        self.current = 1
        return self

    def __next__(self):

        if self.current <= self.limit:

            number = self.current
            self.current += 1

            return number

        raise StopIteration

Usage:

counter = Counter(5)

for n in counter:
    print(n)

Output:

1
2
3
4
5

---

What Happens Internally in This Example?

When Python sees:

for n in counter:

Python approximately performs:

iterator = counter.__iter__()

while True:

    try:
        value = iterator.__next__()
        print(value)

    except StopIteration:
        break

The iterator object keeps returning values one at a time.

---

Important Insight — Iterators Maintain State

Iterators remember where iteration currently is.

Example:

it = iter([10, 20, 30])

Internally the iterator stores something conceptually similar to:

Current Position = 0

After:

next(it)

the position becomes:

Current Position = 1

and continues advancing.

---

Lists Already Support the Iterator Protocol

When you do:

for x in [1, 2, 3]:

Python automatically creates an iterator internally.

That means iterator protocol powers almost every loop in Python.

---

Why Iterators Are Powerful

1. Memory Efficiency

Iterators generate values one at a time.

This avoids loading huge amounts of data into RAM simultaneously.

Example:

range(1000000000)

Python does not create one billion integers immediately.

Values are produced lazily as needed.

---

2. Lazy Evaluation

Data can be generated only when required.

This is extremely useful for:

• streaming systems
• large files
• APIs
• machine learning
• real-time processing

---

Real-World Uses of Iterators

Iterator protocol is heavily used in:

• file handling
• CSV readers
• database cursors
• Django QuerySets
• generators
• streaming APIs
• TensorFlow and PyTorch datasets
• web scraping pipelines
• asynchronous systems

---

File Objects Use Iterators

Example:

with open("data.txt") as f:

    for line in f:
        print(line)

Python reads one line at a time instead of loading the full file into memory.

That is iterator protocol in action.

---

Generators Automatically Implement Iterator Protocol

Generators are simplified iterator creators.

Example:

def count():

    yield 1
    yield 2
    yield 3

Usage:

for x in count():
    print(x)

Generators automatically implement:

• __iter__()
• __next__()

internally.

---

Important Takeaway

The iterator protocol is one of Python’s foundational design concepts.

It enables:

• clean loops
• lazy evaluation
• memory-efficient systems
• scalable data processing

Without iterators, Python would lose much of its elegance and efficiency.

---

Key Takeaway

Whenever you write:

for item in something:

Python is secretly performing operations similar to:

iterator = something.__iter__()

value = iterator.__next__()

The iterator protocol is the invisible engine powering Python iteration.

[Author]

Posts