How to get user confirmation for Location and the geo-coordinates in Dialogflow?

Question

I am making app for google assistant using dialog flow. My app uses a webhook function deployed on firebase. Main question is how to get user location - I need to pass the coordinates to a URL. I am able to prompt the message for location permission, but how to confirm and get coordinates.

Do I need to add any intent for confirmation? I am asking for location permission in default welcome intent.

The code is below:

'use strict';

process.env.DEBUG = 'actions-on-google:*';
const App = require('actions-on-google').DialogflowApp;

const functions = require('firebase-functions');



//----global variables-----
const http = require('https');
var body = "";
var body2="";
var address="";
var latitude="";
var longitude="";
var lati="";
var long="";


//------ADDRESS_ACTION to tell the current address based on current coordinates----
const ADDRESS_ACTION = 'address_action';

//-----DIRECTION_INTENT to output the map guidence through basic card------
const DIRECTION_INTENT = 'direction_action';

//--------during welcome intent, ask the permiisions------
const DIR_PERMISS_CONF= 'intent.welcome';

exports.addressMaker = functions.https.onRequest((request, response) => {
  const app = new App({request, response});
  console.log('Request headers: ' + JSON.stringify(request.headers));
  console.log('Request body: ' + JSON.stringify(request.body));


  function welcomeIntentDirection (app) 
  {
    app.askForPermission("Hi! Welcome to !To show address/direction",app.SupportedPermissions.DEVICE_PRECISE_LOCATION);
    if (app.isPermissionGranted()) 
    {
     // app.tell('You said ' + app.getRawInput());
    }
    else
    {
     // app.tell('You said ' + app.getRawInput());
    }

  }



function makeDirection(app)
{

  //-------here we are extraction what user said, we will extract the place name, it is for "direction to PLACE_NAME"-------

  var stringRequest=JSON.stringify(request.body);
  var jsonRequest=JSON.parse(stringRequest);
  var jsonSpeech=jsonRequest.originalRequest.data.inputs[0].rawInputs[0].query;
  var arr=jsonSpeech.split("to");
  var placeName=arr[1].trim();

  //------Current Location--------------
  let deviceCoordinates = app.getDeviceLocation().coordinates;
  latitude=deviceCoordinates.latitude;
  longitude=deviceCoordinates.longitude;



//-----now send this place name to api to get coordinates-----
 var req2= http.get("https://api.url-with-PARAMETER-placeName",function(res){

  res.on("data", function(chunk2){ body2 += chunk2;  });
  res.on('end', function()
  {
    if (res.statusCode === 200) 
    {
      var data2 = JSON.parse(body2);

      try
      {
          lati=data2.geometry.lat;
          long=data2.geometry.lng;
      }catch(e){app.tell("Invalid or non-existent address");}    
    }
  });
});


//-------------------Here BUILDING the CARD (Rich RESPONSE)-----------------------
  app.ask(app.buildRichResponse()
    // Create a basic card and add it to the rich response
    .addSimpleResponse('Your directions are here:')
    .addBasicCard(app.buildBasicCard()

      .addButton('Start Guidence', 'https://www.google.com/maps/dir/?api=1&origin='+latitude+','+longitude+'&destination='+lati+','+long+'')
      .setImage('https://png.pngtree.com/element_origin_min_pic/16/11/30/a535f05d9d512610e25a036b50da036f.jpg', 'Image alternate text')
      .setImageDisplay('CROPPED')
  )
);

}


  function makeAddress (app) 
      {
        let deviceCoordinates = app.getDeviceLocation().coordinates;
        latitude=deviceCoordinates.latitude;
        longitude=deviceCoordinates.longitude;
        var req=http.get("https://api.url/reverse?coords="+latitude+","+longitude+"", 
        function(res)
        {
          res.on("data", function(chunk){ body += chunk;  });

          res.on('end', function()
          {
            if (res.statusCode === 200) 
            {
                try 
                {
                  var data = JSON.parse(body);
                  address=data.words;
                  app.tell('Alright, your address is '+address);
                } catch (e) { }
            }
            else 
            {
              app.tell("Unable to contact to server +"+res.statusCode);
            }

          });

        });
      }

  // d. build an action map, which maps intent names to functions
  let actionMap = new Map();
  actionMap.set(DIR_PERMISS_CONF,welcomeIntentDirection);
  actionMap.set(ADDRESS_ACTION, makeAddress);
  actionMap.set(DIRECTION_INTENT, makeDirection);



app.handleRequest(actionMap);
});

Answer 1

Yes, you need an Intent to handle the response from the user. The model of Dialogflow is that all responses from the user need to be handled via an intent.

In this case, it would be your "confirmation" Intent.

You don't show a screen shot of the Intent (if you have further problems, please update the question to include it), but it needs to handle the Event actions_intent_PERMISSION. When handling this Event, you don't need any training phrases.

One that I have (with an Intent name and Action of result.location) looks like this:

As a side note, in your handler, you're trying to get the user name, but this name isn't available unless you've asked for permission for it. You didn't ask for permission. If you wanted to, you could ask for permission at the same time you asked for their location with something like

app.askForPermissions("To show your address", [
  app.SupportedPermissions.NAME,
  app.SupportedPermissions.DEVICE_PRECISE_LOCATION
]);

Update based on your screen shots.

It isn't clear what your conversation flow is meant to be, so I'm making a few assumptions.

But based on your code, you're asking for permission as a result of the Welcome intent. The user then gives permission, and you need an intent that accepts that reply.

I don't see an intent that accepts that reply.

In order to accept the reply, you need an Intent that handles the Event actions_intent_PERMISSION, as I illustrated in my screen shot above.

You also seem to be creating Contexts, although it isn't clear why.

Update based on the code you posted and your additional comments.

There are several issues with the code which may be causing some of the problems you're experiencing, and some other issues which may cause you problems in the future. To answer some of your questions in the comments and some unasked questions:

I have enabled Small Talk

This isn't hurting anything - but it probably isn't helping anything either. I would suggest you disable it until you get the core part of your conversation working.

I can get the coordinates

Good - so that Intent is working.

But...

I see that you're storing the coordinates in a global variable. That works in this case (but not quite, see the next question of yours), but won't scale if more than one person is using the action since the values could be mixed up in between user sessions.

In other words: If you use global variables in your webhook, every person has access to these values. This is VERY BAD.

What you want is a way to store user information in between calls to your webhook. There are a number of ways to do this that are covered elsewhere and can be rather involved. In general, however, you can store the values in either a context or in app.data and they will be returned to you as part of the same conversation.

I don't get a destination address when I ask the first time and When I ask a second time, it has the destination address

Your function makeDirection() contains an asynchronous operation to get the lat/lon of the destination address, however you're sending back the reply outside of that asynchronous block.

So what happens is that you start the http call, and then the code continues and you send the reply to the user with the empty destination. At some later point, the callback completes and the destination is set, but too late for the first call. But when you call it the second time, it goes through the same routine (starts the async call, and then immediately sends something back before the call completes), but the values from the previous run are set because you had set them in global variables.

To fix this, you're going to need to call app.ask() as part of your async callback. This might look something like this:

http.get("https://api.url-with-PARAMETER-placeName",function(res){

  res.on("data", function(chunk2){ body2 += chunk2;  });
  res.on('end', function(){
    if (res.statusCode === 200){
      var data2 = JSON.parse(body2);

      try{
        lati=data2.geometry.lat;
        long=data2.geometry.lng;
        app.ask( 
          app.buildRichResponse()
            .addThingsToRichResponse()
        );
      } catch(e){
        app.tell("Invalid or non-existent address");
      }
    }
  });
});

Issues with getting what the user said about the destination

Your code where you're trying to get the user's destination is... strange. Getting the rawInputs is legal, but trying to break it apart to get specific values isn't best practice, especially since the user might not say exactly what you think they're going to say. For example, what if they said "I'm heading towards somewhere".

Better is to have your direction_action Intent take a variety of different phrases and assign the "important" part of that (the destination) to a parameter. Then, in your webhook, you can read just the value of the parameter and pass that to the service to determine the location.

You also may wish to take a look at the recently announced Place Helper, which is similar to the device precise location helper, but also gets an address that the user specifies.