How do I get a function to return a Vec of string-like things?

Question

I had a large block of code that opens files and searches the contents line by line, then does something to each matching line. I want to factor this out into its own function that takes the path to a file and gives you the matching lines, but I cannot figure out how to correctly factor this out.

Here's what I think is close, but I get a compiler error:

/// get matching lines from a path
fn matching_lines(p: PathBuf, pattern: &Regex) ->  Vec<String> {
    let mut buffer = String::new();
    // TODO: maybe move this side effect out, hand it a
    //       stream of lines or otherwise opened file
    let mut f = File::open(&p).unwrap();
    match f.read_to_string(&mut buffer) {
        Ok(yay_read) => yay_read,
        Err(_) => 0,
    };
    let m_lines: Vec<String> = buffer.lines()
        .filter(|&x| pattern.is_match(x)).collect();
    return m_lines;
}

And the compiler error:

src/main.rs:109:43: 109:52 error: the trait `core::iter::FromIterator<&str>` is not implemented for the type `collections::vec::Vec<collections::string::String>` [E0277]
src/main.rs:109         .filter(|&x| pattern.is_match(x)).collect();
                                                          ^~~~~~~~~
src/main.rs:109:43: 109:52 help: run `rustc --explain E0277` to see a detailed explanation
src/main.rs:109:43: 109:52 note: a collection of type `collections::vec::Vec<collections::string::String>` cannot be built from an iterator over elements of type `&str`
src/main.rs:109         .filter(|&x| pattern.is_match(x)).collect();
                                                          ^~~~~~~~~
error: aborting due to previous error

If I use String instead of &str I instead get this error:

src/main.rs:108:30: 108:36 error: `buffer` does not live long enough
src/main.rs:108     let m_lines: Vec<&str> = buffer.lines()
                                             ^~~~~~

Which kind of makes sense. I guess the lines stay inside the buffer which goes out of scope at the end of the function, so collecting a vector of references to strings doesn't really help us.

How do I return a collection of lines?

Answer 1

Let's start with this version, which runs on the Rust Playground (it's a good idea to make a MCVE when asking a question):

use std::path::PathBuf;
use std::fs::File;
use std::io::Read;

fn matching_lines(p: PathBuf, pattern: &str) -> Vec<String> {
    let mut buffer = String::new();
    let mut f = File::open(&p).unwrap();
    match f.read_to_string(&mut buffer) {
        Ok(yay_read) => yay_read,
        Err(_) => 0,
    };
    let m_lines: Vec<String> = buffer.lines()
        .filter(|&x| x.contains(pattern)).collect();
    return m_lines;
}

fn main() {
    let path = PathBuf::from("/etc/hosts");
    let lines = matching_lines(path, "local");    
}

Let's look at the signature for str::lines:

fn lines(&self) -> Lines // with lifetime elision
fn lines<'a>(&'a self) -> Lines<'a> // without

I've shown what it looks like in the source first, and what you can mentally translate it to second. It will return an iterator of string slices that are backed by the String you've read. This is a nice thing, as it's very efficient as only one allocation needs to have been made. However, you cannot return an owned value and a reference to that value at the same time. The easiest thing to do is convert each of the lines into an owned string, as Benjamin Lindley suggests:

let m_lines: Vec<String> =
    buffer
    .lines()
    .filter(|&x| x.contains(pattern))
    .map(ToOwned::to_owned)
    .collect();

That gets your code to compile, but it can still be made nicer. Your match statement can be replaced with unwrap_or, but since you are ignoring the error case completely, you might as well just use _:

let _ = f.read_to_string(&mut buffer);

Note that this really isn't a good idea. Errors are important to report, and throwing away to error will bite you when you need it to be reported the most! It's probably safer to use unwrap and let your program die when an error occurs.

Next, don't use explicit return statements and don't provide type annotations unless you need to. Since your function returns a Vec<String>, you can replace the last two lines with just:

buffer
    .lines()
    .filter(|&x| x.contains(pattern))
    .map(ToOwned::to_owned)
    .collect()

You also could be more open about the types you accept for p, to better match what File::open supports:

fn matching_lines<P>(p: P, pattern: &str) -> Vec<String>
    where P: AsRef<Path>

All together:

use std::path::{Path, PathBuf};
use std::fs::File;
use std::io::Read;

fn matching_lines<P>(p: P, pattern: &str) -> Vec<String>
    where P: AsRef<Path>
{
    let mut buffer = String::new();
    let mut f = File::open(p).unwrap();
    let _ = f.read_to_string(&mut buffer);

    buffer
        .lines()
        .filter(|&x| x.contains(pattern))
        .map(ToOwned::to_owned)
        .collect()
}

fn main() {
    let path = PathBuf::from("/etc/hosts");
    let lines = matching_lines(path, "local");
    println!("{:?}", lines);
}

Answer 2

You can convert your string slices to owned String objects with the map function.

let m_lines: Vec<String> = buffer.lines()
        .filter(|&x| pattern.is_match(x))
        .map(|x| x.to_owned())
        .collect();

You should then be able to return m_lines from the function.