I did some test on caching. This is what I found:
You have to clear the cache for every route that lead to your action.
If you have 3 routes that lead to the exact same action in your controller, you will have one cache for each route.
Let's say, I have this route config:
routes.MapRoute(
                name: "config1",
                url: "c/{id}",
                defaults: new { controller = "myController", action = "myAction", id = UrlParameter.Optional }
                );
            routes.MapRoute(
                name: "Defaultuser",
                url: "u/{user}/{controller}/{action}/{id}",
                defaults: new { controller = "Accueil", action = "Index", user = 0, id = UrlParameter.Optional }
            );
            routes.MapRoute(
                name: "Default",
                url: "{controller}/{action}/{id}",
                defaults: new { controller = "Accueil", action = "Index", id = UrlParameter.Optional }
            );
Then, these 3 routes lead to myAction in myController with the param myParam:
- http://example.com/c/myParam
- http://example.com/myController/myAction/myParam
- http://example.com/u/0/myController/myAction/myParam
If my action is as follow
public class SiteController : ControllerCommon
    {
        [OutputCache(Duration = 86400, VaryByParam = "id")]
        public ActionResult Cabinet(string id)
        {
             return View();
}
}
I will have one cache for each route (in this case 3). Therefore, I will have to invalidate every route.
Like this
private void InvalidateCache(string id)
        {
            var urlToRemove = Url.Action("myAction", "myController", new { id});
            //this will always clear the cache as the route config will create the path
            Response.RemoveOutputCacheItem(urlToRemove);
            Response.RemoveOutputCacheItem(string.Format("/myController/myAction/{0}", id));
            Response.RemoveOutputCacheItem(string.Format("/u/0/myController/myAction/{0}", id));
        }