Vec to &[&Path] without allocations?

Question

I have function that return Vec<PathBuf> and function that accept &[&Path], basically like this:

use std::path::{Path, PathBuf};

fn f(paths: &[&Path]) {
}

fn main() {
    let a: Vec<PathBuf> = vec![PathBuf::from("/tmp/a.txt"), PathBuf::from("/tmp/b.txt")];

    f(&a[..]);
}

Is it possible to convert Vec<PathBuf> to &[&Path] without memory allocations?

If not, how should I change f signature to accept slices with Path and PathBuf?

Can't you just change `f` to: `fn f(paths: &[PathBuf])`? It's still accepting a slice, so there is no additional allocation. — Peter Hall, Mar 11 '17 at 13:52
@PeterHall signature of `f` not accidental, I pass also `Vec<&Path>` to it. — user1244932, Mar 11 '17 at 14:01
If you want it to be generic over Path, PathBuf, Vec and [] you probably need to take an `T, I where T: Iterator, I: AsRef` — the8472, Mar 11 '17 at 14:22

score 5 · Accepted Answer · edited May 23 '17 at 11:54

Is it possible to convert Vec<PathBuf> to &[&Path] without memory allocations?

No, as answered by How do I write a function that takes both owned and non-owned string collections?; a PathBuf and a Path have different memory layouts (the answer uses String and str; the concepts are the same).

how should I change f signature to accept slices with Path and PathBuf?

Again as suggested in How do I write a function that takes both owned and non-owned string collections?, use AsRef:

use std::path::{Path, PathBuf};

fn f<P>(paths: &[P])
    where P: AsRef<Path>
{}

fn main() {
    let a = vec![PathBuf::from("/tmp/a.txt")];
    let b = vec![Path::new("/tmp/b.txt")];

    f(&a);
    f(&b);
}

This requires no additional heap allocation.

Peter Hall · Answer 2 · 2017-03-11T14:16:16.117

0

To pass around a slice, you have to also have the original data held somewhere. To have a &[&Path], then this needs to be pointing into something like a Vec<&Path>. But you don't have one of those, you have a Vec<PathBuf>.

To get this to work with your existing signatures, you can make a temporary Vec<&Path> and then take a slice of it.

fn f(paths: &[&Path]) {
}

fn main() {
    let a: Vec<PathBuf> = vec![PathBuf::from("/tmp/a.txt"), PathBuf::from("/tmp/b.txt")];
    let paths: Vec<&Path> = a.iter().map(PathBuf::as_path).collect();
    f(&paths[..]);

}

Even though this creates a new Vec, this is just a couple of pointers on the stack - it doesn't have to actually copy any of the paths.

edited Mar 11 '17 at 14:16

answered Mar 11 '17 at 14:04

Peter Hall

53,120
14
139
204

Size not matter from my point of view, but allocation on heap may require a lot of time. I thought about changing slice to iterator (almost like your code, but without `collect`), but still can not force compiler to accept my code, because of Sized magic. – user1244932 Mar 11 '17 at 14:11
There are no new heap allocations here. Only references to the existing paths. – Peter Hall Mar 11 '17 at 14:12
`let paths: Vec<&Path>` so you tell that `Vec` allocate place on stack? – user1244932 Mar 11 '17 at 14:28
It's allocating on the heap. But it's only allocating pointers, not the actual data. The compiler will likely optimise away the extra jumps too. – Peter Hall Mar 11 '17 at 14:31
1

*There are no new heap allocations here* — **there is absolutely heap allocation taking place**. The entire vector is allocated on the heap! I do not believe there will be any optimizations here because vectors are a run time concept, so the compiler cannot optimize through them. – Shepmaster Mar 11 '17 at 14:48
@Shepmaster Yes, that's fair. But there is no other way to lay out the memory of an unknown size of data, in order to create the requested slice. – Peter Hall Mar 11 '17 at 16:28

score 0 · Answer 3 · answered Mar 11 '17 at 14:58

It's not possible to just cast without any allocations, because their layout in memory is different.

Vec<PathBuf> stores the data inline, and [&Path] stores pointers to the data (it's roughly similar to Vec<&PathBuf>).

You need to create a new vector to hold the pointers. If the size is known at compile time, you could use a stack-allocated array for it. Otherwise map+collect is needed.

Vec to &[&Path] without allocations?

3 Answers3