Iterators to read and process file in Java

Question

Suppose I have a class Point and a function to process Point instances

class Point { private final int x, y; ... } 
...
void handlePoints(Iterable<Point> points) { for (Point p: points) {...} }

Now I would like to read points from a file. Each line of the file contains two numbers, so I have a function ("factory method") to create a point from a line.

Point makePoint(String line) { ... }

What should I do now? I can write a function to read the file to a list of points and call the handlePoints function.

List<Point> readPoints(BufferedReader reader) {...} // use makePoint here

void handlePoints(BufferedReader reader) {
   List<Point> points = readPoints(reader); 
   handlePoints(points);
}

Unfortunately this function does not seem particularly elegant since it creates an unnecessary list of points in memory.

Wouldn't it be better to use iterators ?

void handlePoints(Iterator<Point> points) {...}

Iterator<Point> readPoints(BufferedReader reader) {...} // use makePoint here

void handlePoints(BufferedReader reader) {
   Iterator<Point> points = readPoints(reader); 
   handlePoints(points);
}

Does it make sense? Won't be this code too "noisy" in Java?

Bernhard Barker · Accepted Answer · 2015-05-21T11:40:07.207

If you don't need to have all points in memory, think of something more along these lines:

while (reader.ready())
{
  String line = reader.readLine();
  Point point = makePoint(line);
  handlePoint(point);
}

How to do this with an iterator and handlePoints: (code for handling exceptions to be added)

class PointIterator implements Iterator<Point>
{
  BufferedReader reader;
  PointIterator(BufferedReader myReader) { reader = myReader; };
  @Override
  public boolean hasNext() { return myReader.ready(); };
  @Override
  public Point next() { return makePoint(myReader.readLine()); };
  @Override
  public void remove()
  { throw new UnsupportedOperationException("Remove not supported!"); };
}

And because handlePoints takes an Iterable:

class PointIterable implements Iterable<Point>
{
  BufferedReader reader;
  public PointIterable(BufferedReader myReader) { reader = myReader; };
  @Override
  public Iterator<Point> iterator() { return new PointIterator(reader); }
}

To use:

handlePoints(new PointIterable(reader));

I would like to _reuse_ `handlePoints`, which handles all points together. — Michael, Jan 01 '13 at 11:27
If you don't mind adding it to your dependencies, then Apache Commons IO provides `FileUtils.lineIterator(file, "UTF-8")` method that returns a `LineIterator`. You must close the LineIterator once you're done. Mapping the line to your Java object is still done as above. See here: https://commons.apache.org/proper/commons-io/javadocs/api-release/index.html?org/apache/commons/io/package-summary.html — acorello, May 20 '15 at 16:02

score 1 · Answer 2 · answered Jan 01 '13 at 11:18

1

From a memory point of view, you won't really save any memory by using iterators - I'm guessing you'll be reading all the points into memory, so they'll all have to be stored somehow.

An iterator isn't a different collection type: it is simply a different way of iterating through a collection. For example, you could go list.iterator() to get an iterator to loop through your list (or any collection).

The choice of what collection to use to hold all the points in memory is the one that will affect memory (eg. ArrayList vs. LinkedList).

answered Jan 01 '13 at 11:18

John Farrelly

7,289
9
42
52

I would like to create an `iterator`, which reads a line on `next` and converts it to a `Point` instance. – Michael Jan 01 '13 at 11:30
@Michael What if the file is invalid (contains rubbish data or does not end correctly)? Most of the time, I read in an entire file before processing to ensure the file and all its data is valid. Unless you have a very large number of points, you shouldn't run into any significant memory problems. Otherwise, you would create your own class which implements the `Iterator` interface, takes the `File` in the constructor and reads the next line in the `next()` method – John Farrelly Jan 01 '13 at 11:35

score 1 · Answer 3 · answered Jan 01 '13 at 11:30

1

Just read once from file and have it in memory instead of reading every time from file

List<Points> points ;

public List<Point> readPoints(BufferedReader reader) {
     if(points == null) {
        points = new ArrayList();
        // read from file and populate 
        points.add(point) ;
     }
 return points;
}

answered Jan 01 '13 at 11:30

vels4j

11,208
5
38
63

What if I do not want to read the whole file in memory before handling points ? It looks like a waste of resources for me. – Michael Jan 01 '13 at 11:32
I dont have any idea. better you can have it in a db then read it – vels4j Jan 01 '13 at 12:05

Iterators to read and process file in Java

3 Answers3

Linked