0

Suppose you have a lazily-populated Iterator which you would like to convert to a Stream. It can be assumed that the number of elements in the Iterator is finite.

It is possible to instruct the Iterator to skip elements (via a skip method on the implementation) to avoid unecessary element generation, but all elements of the Iterator must have been either skipped or generated once the Stream has had a terminal operation called on it.

I am aware of:

StreamSupport.stream(
        Spliterators.spliteratorUnknownSize(
                theIterator, Spliterator.ORDERED), false);

for generating a Stream from an Iterator, however this does not provide a means to skip or ensure consumption of all elements.

Is there any basic Stream implementations which would allow for this via reasonably simple extension?

Delegation seems messy given the sheer number of methods of the Stream interface with no promise as to which will call which others as part of their internal implementation, and all the JDK implementations one could use as a starting point (at least in the JDK I'm using) are package-private and I assume there's good reason for that (e.g. they don't want me extending them...).

BeUndead
  • 3,463
  • 2
  • 17
  • 21
  • *with no promise as to which will call which others as part of their internal implementation*... that's a problem with inheritance, not delegation. You can use [proxies](https://stackoverflow.com/questions/933993/what-are-dynamic-proxy-classes-and-why-would-i-use-one) to avoid some of the delegation boilerplate. Btw, the JDK's `skip()` implementation is `final`, so it wouldn't help you to extend it. – shmosel May 31 '19 at 01:58
  • so you have an `Iterator` that you want to convert to a `Stream`, but once `skip(x)` is called in the `Stream`, you want to not generate the elements on the `Iterator` side? how would that be possible? an `Iterator` has `hasNext/next` - in order to be able to "skip", you _need_ to call `next` on it, there is no other way. also _however this does not provide a means to skip or ensure consumption of all elements_ - what if the terminal operation is `findFirst`? not all elements would have to be taken from the `Iterator`. – Eugene May 31 '19 at 09:20
  • 1
    I think, the simplest and cleanest solution is to register an action via `onClose​` which would enforce skipping all remaining elements. Then, the using code must use `try(...) {}`, unless in a `flatMap` context. There is no other way to ensure an action after use. – Holger May 31 '19 at 09:58
  • @Eugene It is an implementation of `Iterator` which also has a `skip` method. So (for example) if you had your stream with a `limit` of `5`, you could skip all elements after the first 5. Or if you'd called `skip` on the `Stream`, the `Iterator` could skip those elements. – BeUndead May 31 '19 at 12:16
  • @Holger I'd thought the same, but I couldn't see a way to allow that to `skip` elements not at the _end_ of the `Stream`. For example (as per above comment) if `skip` had been called on the `Stream`. That does allow resolution of the latter part for 'ensure all elements have been generated' though. – BeUndead May 31 '19 at 12:17
  • When you close a stream, you *are* at the end of the stream, by definition. Any element of the iterator not consumed yet, won't get consumed by the stream at all, after closure. So skipping all remaining on close would do the right thing. Directing Stream.skip to call a non-standard skip method on the source iterator is not supported. – Holger May 31 '19 at 15:42

1 Answers1

0

Well I took a swing at doing this. The idea was to create a TypeAdapter for Streams when reading through gson for json arrays, without having to convert every element:

https://github.com/BeUndead/gson-stream

From my quick round of testing it does everything I wanted it to, but there's almost certainly issues I didn't come across when quickly testing it.

BeUndead
  • 3,463
  • 2
  • 17
  • 21
  • Quiet a lot of code for handling just a corner case, i.e. when `skip` is the very first operation on that stream, i.e. not even `.limit(a).skip(b)` would get the benefit… – Holger Jun 03 '19 at 12:38
  • It still benefits from the skip-all-remaining part, which is a fair chunk of that code. The only bit which is specific to skipping for the first operation is the `SkippingDelegatingSkippableIterator` and the `skip` method of the `Stream` delegate. But yes, I agree ideally I would like to have gotten the benefit there, but without rewriting fairly hefty chunks of the JDK `Stream` classes it didn't look like it was going to happen. – BeUndead Jun 03 '19 at 14:44
  • Well, the “skip-all-remaining part” is done by registering the action via `onClose`, which could have been done in much lesser code, without implementing your own `Stream`. So the majority of that code is for the less useful feature. – Holger Jun 03 '19 at 16:23
  • Fair point. Honestly hadn’t registered that bit as a lot of code as all except the skip method and constructor was just right click > generate > delegate. – BeUndead Jun 03 '19 at 19:07