To detect the click on the overflow menu have such code: 
@Override
public boolean onMenuOpened(int featureId, Menu menu) {
    if(featureId == AppCompatDelegate.FEATURE_SUPPORT_ACTION_BAR && menu != null){
        //overflow menu clicked, put code here...
    }
    return super.onMenuOpened(featureId, menu);
}
@Override
public void onPanelClosed(int featureId, Menu menu) {
    ...
}
To detect click on menu items, in case you have a menu like that : 
<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android">
    <item android:id="@+id/menu2" android:alphabeticShortcut="b"
        android:title="Menu No. 2" android:orderInCategory="2">
        <menu >
        <group android:id="@+id/group2" android:checkableBehavior="single">
            <item android:id="@+id/submenu1" android:title="SubMenu No. 1" />
            <item android:id="@+id/submenu2" android:title="SubMenu No. 2" />
        </group>  
        </menu>
    </item>
</menu>
You should be able to detect the click in 
onOptionsItemSelected
@Override
public boolean onOptionsItemSelected(MenuItem item) {
    Log.w("ANDROID MENU TUTORIAL:", "onOptionsItemSelected(MenuItem item)");
    // Handle item selection
    switch (item.getItemId()) {
    case R.id.menu2:
        Toast.makeText(this, "Clicked: Menu No. 2", Toast.LENGTH_SHORT).show();
        return true;   
        ...
}