Per updated/clarified question (regarding this being for an error page)..
While javascript could still manage this, this is best suited for server-side.
I don't know what your OnError method looks like, or if you're using application.cfm. Here's how I'd accomplish this:
Instead of straight redirecting to /error.cfm or whatever the error page is, I'd:
<cfif not isDefined("url.inWindow")>
<cflocation url="/error.cfm" addtoken="no" />
<cfelse>
<cflocation url="/error.cfm?inWindow=1" addtoken="no" />
</cfif>
And within the error file, I'd do some <cfif>s based on the presence, of url.inWindow.
Javascript, and especially jQuery, have methods of doing this too, you could remove header and footer content and use unwrap(), to pull the content of the content area, as demonstrated here: JS - Remove a tag without deleting content.
I don't use CFWINDOW so am unfortunately unfamiliar with the nuances, and Railo supports it a bit differently
Cold Fusion
The best ColdFusion route to achieve this may just be passing a url variable via the source link.
<cfwindow ... source="/demo/6/windowContentA.cfm?inWindow=1">
And then, within the document.
<cfif isDefined("url.inWindow")>The page is within a window</cfif>
Javascript
The way to do this in javascript has to do with either (and this is the part I'm unable to test :/ without access to ACF.) checking either if the window is the top element (but that might not work).
<a href="javascript:void(0)" onclick="alert((self==top ? 'in a popup' : 'not in a popup'));">Test link</a>
Or (if that doesn't work, checking if an element exists within the page as well.
<a href="javascript:void(0)" onclick="alert(( document.getElementById('cf_window1') ? 'in a popup' : 'not in a popup'));">Test link</a>
Because CFWindow created an element named cf_window1, I used that for testing, but you should be able to use any element name. You could create an element specifically for this purpose if you wanted.
Although this how I managed to test...through some wizardry that doesn't matter, you could use similar javascript within a script tag to do something besides alert.
<script>
if(self==top) {
// The page is outside of a frame.
}
</script>
Or the look-for-element route
<script>
if(document.getElementById('cf_window1')) {
// The page is within a frame.
}
</script>
Neither route should be considered very secure. It's for convenience, and display, but you shouldn't consider them impenetrable, because they aren't.