将迭代器返回函数转换为 "proper iterable" 返回函数的标准方法？

Question

为了避免混淆，让我定义:

proper iterable: an iterable object that is not an iterator.

问： Python 的标准库是否已经提供了一种将 "iterator-returning function" 转换为“适当的可迭代对象的方法-返回函数"?

我以为我在哪里见过这个，但现在找不到了。特别是，我浏览了 itertools 的文档，但没有发现它。

FWIW，这个本土实现似乎有效：

def to_iterable_maker(iterator_maker):

    def iterable_maker(*args, **kwargs):
        class nonce_iterable(object):
            def __iter__(self):
                return iterator_maker(*args, **kwargs)
        return nonce_iterable()

    return iterable_maker

...但是一次性 nonce_iterable class 在我看来很笨拙。我确信从标准库中实现这样的东西会好得多。

@Nikita

试试这个：

import itertools

base = range(3)
poops_out = itertools.permutations(base)

print list(poops_out)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(poops_out)
# []

myperms = to_iterable_maker(itertools.permutations)
keeps_going = myperms(base)

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

itertools.permutations 和 to_iterable_maker(itertools.permutations) 返回的值之间存在差异。我的问题是：标准库是否已经提供了类似于 to_iterable_maker 的东西？

Answer 1

没有意义 - "iterator-returning function" 变成 "iterable returning function"。如果一个函数正在返回一个迭代器，那么它已经返回一个可迭代对象，因为迭代器是可迭代对象，因为它们需要具有 __iter__ 方法。

来自the docs：

iterable

An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() or __getitem__() method. Iterables can be used in a for loop and in many other places where a sequence is needed (zip(), map(), ...). When an iterable object is passed as an argument to the built-in function iter(), it returns an iterator for the object. This iterator is good for one pass over the set of values. When using iterables, it is usually not necessary to call iter() or deal with iterator objects yourself. The for statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop. See also iterator, sequence, and generator.

iterator

An object representing a stream of data. Repeated calls to the iterator’s __next__() method (or passing it to the built-in function next()) return successive items in the stream. When no more data are available a StopIteration exception is raised instead. At this point, the iterator object is exhausted and any further calls to its __next__() method just raise StopIteration again.

Iterators are required to have an __iter__() method that returns the iterator object itself so every iterator is also iterable

and may be used in most places where other iterables are accepted. One notable exception is code which attempts multiple iteration passes. A container object (such as a list) produces a fresh new iterator each time you pass it to the iter() function or use it in a for loop. Attempting this with an iterator will just return the same exhausted iterator object used in the previous iteration pass, making it appear like an empty container.

UPD:

我的意思是...

（按步骤显示比较）

案例一:

f = to_iterable_maker(iterator_maker);
i = f(some_var), i 是 nonce_iterable 和 __iter__;
j = iter(i),j是iterator_maker(some_var)返回的迭代器；
next(j)、returns 某些值取决于 some_var。

案例二:

f = iterator_maker;
i = f(some_var)，i 是等于 iterator_maker(some_var) 的迭代器，它有 __iter__（根据迭代器协议）；
j = iter(i),j是iterator_maker(some_var)返回的迭代器，因为在迭代器returns本身调用__iter__，所以j is ireturns true;
next(j)、returns 某些值取决于 some_var。

如您所见，除了准备步骤的额外复杂性外，没有什么真正的改变。

也许您可以提供更多信息，说明您试图通过此类 'wrapping' 实现的目标，以了解真正的问题。

根据你的问题，我想不出任何库函数可以将迭代器变成可迭代的，因为它已经是。如果你想复制迭代器，可以看看 itertools.tee().

UPD2:

所以，现在我明白了，目标是将单遍迭代器转换为多遍迭代器...

我的回答：

"does the Standard Library already provide something analogous to to_iterable_maker?"

是"No"。但最接近的是 itertools.tee()，它可以帮助您将单个迭代器克隆为多个，您可以在之后使用。关于你的例子：

import itertools

base = range(3)
poops_out = itertools.permutations(base)

iterators = itertools.tee(poops_out, 4)

#You shouldn't use original iterator after clonning, so make it refer to a clone
#to be used again, otherwise ignore the following line
poops_out, iterators = iterators[0], iterators[1:]

for it in iterators:
    print list(it)

#Prints:
#[(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]
#[(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]
#[(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

从迭代器获取可迭代对象的另一种常见方法是使用 list() 或 tuple() 转换它，这将允许多次通过：

import itertools

base = range(3)

poops_out = itertools.permutations(base)

#Obviously poops_out gets consumed at the next line, so it won't iterate anymore
keeps_going = tuple(poops_out)

print list(poops_out)
# []

print list(poops_out)
# []

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

上述两种方法都可能会占用大量内存，因此有时不可行。在这种情况下，您找到的解决方案会很有效。我能想到的另一个实现，它更面向对象，但与你的没有太大区别：

class IterableMaker(object):
    '''Wraps function returning iterator into "proper" iterable'''

    def __init__(self, iterator_maker):
        self.f = iterator_maker

    def __call__(self, *args, **kwargs):
        self.args = args
        self.kwargs = kwargs
        return self

    def __iter__(self):
        return self.f(*self.args, **self.kwargs)

用法相同：

import itertools

class IterableMaker(object):

    def __init__(self, iterator_maker):
        self.f = iterator_maker

    def __call__(self, *args, **kwargs):
        self.args = args
        self.kwargs = kwargs
        return self

    def __iter__(self):
        return self.f(*self.args, **self.kwargs)

base = range(3)
poops_out = itertools.permutations(base)

print list(poops_out)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(poops_out)
# []

my_perm = IterableMaker(itertools.permutations)

keeps_going = my_perm(base)

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

print list(keeps_going)
# [(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]

将迭代器返回函数转换为 "proper iterable" 返回函数的标准方法？

Standard way to convert iterator-returning function into "proper iterable"-returning function?

python

iterator

decorator

iterable