lifetime of function pointer is for<'a, '_> while it should be for<'r>

Question

Sometimes I struggle with lifetimes. I'm still learning and I don't know what's happening here:

use std::future::Future;
use futures::future::{BoxFuture, FutureExt};

struct M{}
struct Client{}

impl Client {
    async fn send_and_expect<'a>(
        &'a mut self,
        m: &M
    ) -> std::result::Result<(), ()> {
        Ok(())
    }
    
    pub fn connection_retrier<'a, T>(
        f: fn(&'a mut Self, &M) -> T,
        f_self: &'a mut Self,
        f_m: &'a M,
    )-> BoxFuture<'a, std::result::Result<(),()>>
    where
        T: Future<Output = std::result::Result<(), ()>> + 'a
    {
        async move {
            Client::send_and_expect(f_self, f_m).await
        }.boxed()
    }
    
    async fn send_with_retry<'a>(&'a mut self) -> std::result::Result<(), ()> {
        let m = M{};
        Client::connection_retrier(
                    Client::send_and_expect, 
                    &mut *self, &m).await
    }
}

Error:

   Compiling playground v0.0.1 (/playground)
error[E0308]: mismatched types
  --> src/lib.rs:31:21
   |
11 |     ) -> std::result::Result<(), ()> {
   |          --------------------------- the `Output` of this `async fn`'s found opaque type
...
31 |                     Client::send_and_expect, 
   |                     ^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected fn pointer `for<'r> fn(&mut Client, &'r M) -> impl futures::Future`
              found fn pointer `for<'a, '_> fn(&'a mut Client, &M) -> impl futures::Future

Playground

I'm now completely confused about why in for<'r> fn(&mut Client, &'r M) -> impl futures::Future, &mut Client has no lifetime. And what does _ means in for<'a, '_> fn(&'a mut Client, &M) -> impl futures::Future`?

I'm very intersted in learning what's happening here.

Answer 1

Cleaning up the code

In order to see this code from the compiler's perspective, let's change all the lifetime parameters in this example to be explicit and have distinct names, expand the async fn sugar, and then look at how the error changes.

The above example is equivalent to the following after lifetime inference and desugaring.

// name the second lifetime, and expand the async fn sugar.
fn send_and_expect<'a, 'b>(&'a mut self, m: &'b M) -> impl Future<Item=Result<(), ()>> + 'a + 'b 
{ ... }

// rename 'a to 'c to avoid ambiguous lifetime names
pub fn connection_retrier<'c, T>(
        f: for<'d> fn(&'c mut Self, &'d M) -> T, // name the implicit higher-ranked lifetime here
        f_self: &'c mut Self,
        f_m: &'c M,
    )-> BoxFuture<'c, std::result::Result<(), ()>>
where T: Future<Output = std::result::Result<(), ()>> + 'c 
{ ... }

// rename 'a to 'e to avoid ambiguous lifetime names
async fn send_with_retry<'e>(&'e mut self) -> std::result::Result<(), ()> {

After making this change, the error becomes:

Compiling playground v0.0.1 (/playground)
error[E0308]: mismatched types
  --> src/lib.rs:31:21
   |
11 |     ) -> std::result::Result<(), ()> {
   |          --------------------------- the `Output` of this `async fn`'s found opaque type
...
31 |                     Client::send_and_expect, 
   |                     ^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected fn pointer `for<'d> fn(&mut Client, &'d M) -> impl futures::Future`
              found fn pointer `for<'a, 'b> fn(&'a mut Client, &'b M) -> impl futures::Future`

This should clarify your question about '_: it's just the name the compiler gave the inferred second lifetime parameter of send_and_expect. As for the missing lifetime on &mut Client, you can see that it's still missing here. For reasons I do not completely understand, and in ways that depend on the exact error message given, the compiler will sometimes omit concrete lifetimes when printing the type of references, but make no mistake, the lifetime of that reference is 'c.

Solving the error

Onto the actual problem. The signature of f indicates that connection_retrier is expecting a function which (1) takes a reference of lifetime &'c and (2) a reference of any other lifetime, and (3) returns a Future type which will remain valid as long as 'c does, as specified by your where bound.

When we pass send_and_expect to connection_retrier, in order for the signatures to match the compiler is coercing it to the type for<'d> send_and_expect::<'c, 'd>. But that type doesn't meet condition (3) above! Unlike a regular function, the default behavior of an async function is to capture all input lifetimes in its return type, so the return type of for<'d> send_and_expect::<'c, 'd> is in fact impl Future<Item=Result<(), ()>> + 'c + 'd, as you can tell by looking at send_and_expect's expanded signature.

Since this type borrows from the two lifetimes 'c and 'd, and there is no constraint that 'd: 'c (read: 'd outlives 'c), it may not remain valid for the entirety of the lifetime 'c, if 'd ends first. It is this mismatch that results in the rather cryptic lifetime error you received. There are two ways you can solve this problem, depending on your preferred semantics. You can either:

• Remove the higher-ranked bound from f entirely, and specify exactly the lifetimes you will be calling it with in connection_retrier (both &'c.)

  pub fn connection_retrier<'c, T>(
          f: fn(&'c mut Self, &'c M) -> T
          f_self: &'c mut Self,
          f_m: &'c M,
      ) -> BoxFuture<'c, std::result::Result<(), ()>>
  where T: Future<Output = std::result::Result<(), ()>> + 'c 
  { ... }
  

• Keep the signature of connection_retrier the same and specify that the future returned by send_and_expect only borrows from its first argument. To do this, you will need to drop the async fn sugar on the signature and wrap the body in an async move block.

  fn send_and_expect<'a, 'b>(&'a mut self, m: &'b M) -> impl Future<Output=Result<(), ()>> + 'a 
  { async move { ... } }
  

Note that you cannot solve this by writing the type of f as for<'d: 'c> fn(&'c mut Self, &'d M) -> T, as bounds are currently not permitted for universally quantified lifetimes.