Yes this is possible but it requires a bit of hacking. It requires that you copy around some function handles.
Using the example provided in the question I will show how to wrap the function openvar in a user defined function that checks the size of the input variable and then allows the user to cancel any open operation for variables that are too large.
Additionally, this should work when the user double clicks a variable in the Workspace pane of the Matlab IDE.
We need to do three things.
- Get a handle to the original 
openvar function 
- Define the wrapper function that calls 
openvar 
- Redirect the original 
openvar name to our new function. 
Example Function
function openVarWrapper(x, vector)
    maxVarSize = 10000;
    %declare the global variable
    persistent openVarHandle; 
    %if the variable is empty then make the link to the original openvar
    if isempty(openVarHandle)
        openVarHandle = @openvar;
    end
    %no variable name passed, call was to setup connection
    if narargin==0
        return;
    end
    %get a copy of the original variable to check its size
    tmpVar = evalin('base', x);        
    %if the variable is big and the user doesn't click yes then return
    if prod( size( tmpVar)) > maxVarSize
        resp = questdlg(sprintf('Variable %s is very large, open anyway?', x));
        if ~strcmp(resp, 'Yes')
            return;
        end
    end
    if ischar(x) && ~isempty(openVarHandle);
        openVarHandle(x);
     end
 end
Once this function is defined then you simply need to execute a script that
- Clears any variables named 
openvar 
- run the 
openVarWrapper script to setup the connection 
- point the original 
openVar to openVarWrapper 
Example Script:
clear openvar;
openVarWrapper;
openvar = @openVarWrapper;
Finally when you want to clean everything up you can simply call:
clear openvar;