In IE versions 8-11 You can use document.documentMode.  Valid values are 5, 7 (compatibility mode), 8, 9, 10, and 11 (Edge).  
- Setting compatibility mode in the console changes the value directly. 
- Loading a page with a - <meta http-equivtag changes the value
 
- Adding a site to compatibility mode in "Tools -> Compatibility View
settings" changes the value to 7. 
https://msdn.microsoft.com/en-us/library/jj676915(v=vs.85).aspx
Examples
For example if I load this page in IE11 I get documentMode of 11.
<!doctype HTML>
<body>
<p>Hello World!<p>
</body>
This page loaded in IE11 sets documentMode to 9.
<html>
<head>
<meta http-equiv="x-ua-compatible" content="IE=9"/>
</head>
<body>
<p>Hello World!<p>
</body>
</html>