Feature request: Something like list comprehension about cppitertools HOT 12 CLOSED

Always good to hear people are enjoying the library! I think iter::imap is what you're looking for. As runnable example:

#include <cppitertools/imap.hpp>
#include <cppitertools/range.hpp>
#include <iostream>

int main() {
  for (auto&& p : iter::imap([](int x) { return x*x; }, iter::range(10))) {
    std::cout << p << '\n';
  }
}

is that right? A list comprehension of course is more involved and produces a list instead of the above which is roughly a generator. I've toyed with adding something like to<std::vector> or to_vector to actually generate a list, but the only nice way to get a list comprehension is language support

from cppitertools.

How unsatisfying is it to do something like the below?

int main() {
    std::vector<int> ret;
    for(auto &&p: iter::imap([ret&](int x)-> void{return x*x;}, iter::range(10)))
        ret.push_back(p);
    for(auto i: ret) puts(std::to_string(i).data());
}

from cppitertools.

[ret&](int x)-> void{return x*x;}
I don't understand. Did you mean[&ret] (int x) { ret.push_back(x*x);}?

from cppitertools.

Sorry, I spliced together two versions I was toying with.
It should be

int main() {
    std::vector<int> ret;
    for(auto &&p: iter::imap([&ret](int x){return x*x;}, iter::range(10)))
        ret.push_back(p);
    for(auto i: ret) puts(std::to_string(i).data());
}

or

int main() {
    std::vector<int> ret;
    for(auto &&p: iter::imap([&ret](int x)-> void{ret.push_back(x*x);}, iter::range(10)));
    for(auto i: ret) puts(std::to_string(i).data());
}

I suppose the latter is more what I'd hope to use.

Unfortunately, returning void is illegal for p, so we have to add some kind of return value. This compiles and behaves as expected RE: list comprehension:

#include <vector>
#include <cstdlib>
#include <cstdio>
#include <utility>
#include <string>
#include <cppitertools/imap.hpp>
#include <cppitertools/range.hpp>
template<typename RET, typename C, typename fn> 
std::vector<RET> &&range_vec(C &&container, fn func) {
    std::vector<RET> ret;
    for(auto &&p: iter::imap(func, container)) ret.emplace_back(p);
    return std::move(ret);
}

int main() {
    int inc(0);
    std::vector<int> &&ret(range_vec<int>(iter::range(400), [&inc](int x){return x * x * x/13./17 - ++inc;}));
    for(auto i: ret) puts(std::to_string(i).data());
}

Edit: cleaner version.

Now, I believe that this copies the elements being emplaced, which is less than ideal for large objects.

from cppitertools.

In the first example I don't think you need to be capturing the ret in the lambda. I'm not sure why you're jumping through so many hoops to print out an int like that when you could std::cout or at least std::printf it.

I feel like the right way to do this, assuming you need an intermediate vector would be to use the vector constructor that takes a pair of iterators, rather than repeated push_back calls:

int main() {
  auto sqs = iter::imap([](int x){return x*x;}, iter::range(10));
  std::vector<int> ret(std::begin(sqs), std::end(sqs));
  for(auto i: ret) {
    std::cout << i << '\n';
  }
}

As for doing this with a conversion to std::vector specifically, this is a first draft:

template <typename Seq>
std::vector<std::decay_t<decltype(*std::begin(std::declval<Seq&>()))>>
to_vector(Seq&& seq) {
  return {std::begin(seq), std::end(seq)};
}

int main() {
  auto ret = to_vector(iter::imap([](int x){return x*x;}, iter::range(10)));
  for(auto i: ret) {
    std::cout << i << '\n';
  }
}

from cppitertools.

@adamsd5, is imap + vector constructor good enough for your purposes?

from cppitertools.

I'm gonna go ahead and close this, please reopen if you wanna discuss this further

from cppitertools.

@ryanhaining, is something like to_vector part of this library? In particular I'm looking for the same for other containers as well, like set. Of course I can easily add it myself, but I was woundering if cppitertools has such container constructors?

from cppitertools.

Though I definitely see the utility and end up writing vector constructors myself, I haven't come up with an intuitive way to deduce things that would be safe in the majority of cases. Consider this:

std::string get_str() { return {}; }

int main() {
  auto v = iter::to_vector(
      iter::enumerate(get_str())
  );
}

enumerate itself is fine handling a temporary, the std::string returned from get_str will be moved into the Enumerable object that enumerate returns, so a line like for (auto&& [i, c] : iter::enumerate(get_str())` works just fine.

The problem is that the type that enumerate returns will use the underlying iterable's reference type (effectively), so what you end up with is enumerate yielding a subclass of std::pair<std::size_t, char&>. Again, this is fine for normal use, but if you put those into a vector, now you have a std::vector<std::pair<std::size_t, char&>>, but all of those references are dangling since the vector won't extend the lifetime of the Enumerable.

I'd have to be pretty creative about this to remove all the references, but who even knows when keeping the reference might be the right thing to do? Of course, if I required the caller to specify iter::to_vector<std::pair<std::size_t, char>> it could work, alternatively allow any container with iter::to<std::vector<std::pair<std::size_t, char>>> but I don't even know if it's still useful at that point.

from cppitertools.

Hm... thanks for the explanation, I see roughly what the issue is.

In the example above to_vector(iter::imap([](int x){return x*x;}, iter::range(10))) it is pretty clear that we expect vector<int> but for example in the following I can see both vector<string> and vector<string&> being useful in specific situations:

struct Foo {
 std::string& name();
}

std::vector<Foo> foo = { ... };

auto names = iter::to_vector(iter::imap([](auto& x){return x.name();}, foo));

However, I think as a default, it should be vector of non-references (maybe with variant iter::refs_to<std::vector>(...)). But that of course leaves the issue highlighted with enumerate. Not sure if it makes sense or could even be implemented, but intuitively it should ignore all the "levels of nesting" that were added by intermediate range expressions and make sure it makes a "deep copy" up until the "original input". I.e. in your example it should somehow detect that it needs to copy at least until the level of elements of what was returned by get_str() (but no further), no matter how many levels were added by enumerate or similar. When you say enumerate returns subclasses of std::pair, that sounds like something you could use in template specialization, however, I have no clue if such reasoning always help, e.g. when mixing with other (custom) range utilities...

from cppitertools.

PS: In your example, the compiler doesn't happen to be not smart enough to detect that we store references to a temporary? I was thinking that a simple implementation, i.e. a generatlization of your proposal above ideally to all containers (maybe with template templates, if that can be implemented, i.e. iter::to<std::vector>(...) or iter::as<std::vector>(...) or iter::copy_to<std::vector>(...), ...), that only removes references "one level deep" would already be helpful in many situations. Question is if potentially harmful situations could be detected by the compiler to warn the user.

template <typename Seq>
std::vector<std::decay_t<decltype(*std::begin(std::declval<Seq&>()))>>
to_vector(Seq&& seq) {
  return {std::begin(seq), std::end(seq)};
}

from cppitertools.

In your first example I would always do vector<string> and that wouldn't be too hard, as you've already demonstrated.

I believe that even in cases where it's not an rvalue, if you can't remove all the references then it won't be intuitively safe

auto get_vector() {
  std::string s{"test string"};
  auto e = iter::enumerate(s);
  return iter::to_vector(e); // full of references to local variable `s`
}

int main() {
  auto v = get_vector(); 
  v[0].second = 'b'; // dangling reference
}

Some of the itertools would work fine with this, removing all the nesting is certainly possible as long as I know all the types that could be contained, but safety is my top concern with this.

from cppitertools.