Yes there is reason not to: orientation is unrelated to whether the device uses touch, it uses viewport width and height to esablish what the orientation is. (orientation: landscape) will apply when the viewport is wider than it is tall and (orientation: portrait) will apply when the viewport is taller than it is wide. So in effect, any browser on any device that supports media queries fully will utilise orientation. 
Check out the jsfiddle form this stackoverflow answer and you'll see the orientation media query taking effect when you resize your browser window.
As an addendum, you should be aware of this quote from the modernizr docs:
The Modernizr.touch test only indicates if the browser supports touch
  events, which does not necessarily reflect a touchscreen device. For
  example, Palm Pre / WebOS (touch) phones do not support touch events
  and thus fail this test. Additionally, Chrome (desktop) used to lie
  about its support on this, but that has since been rectified.
  Modernizr also tests for Multitouch Support via a media query, which
  is how Firefox 4 exposes that for Windows 7 tablets.
From http://modernizr.com/docs/#features-misc
Having said that, it may still be suitable for what you're doing. That's up to you