Perusing the release notes for Fx Quantum 57, specifically Quantum CSS notes, a number of differences between Gecko and Stylo are listed, and a few hacks come to mind.
Here's one:
You can use @supports with a calc(0s) expression in conjunction with @-moz-document to test for Stylo — Gecko does not support <time> values in calc() expressions but Stylo does:
@-moz-document url-prefix() {
  @supports (animation: calc(0s)) {
    /* Stylo */
  }
}
Here's a proof-of-concept:
body::before {
  content: 'Not Fx';
}
@-moz-document url-prefix() {
  body::before {
    content: 'Fx legacy';
  }
  @supports (animation: calc(0s)) {
    body::before {
      content: 'Fx Quantum';
    }
  }
}
 
 
Note that Fx Quantum 59 and 60 disallowed the use of @-moz-document in public-facing documents, but version 61 restores functionality for the @-moz-document url-prefix() hack, allowing this to work as intended. Version 60 is an ESR release, however, so if you need to target that version, you'll need to change the @-moz-document at-rule to a media query:
@media (-moz-device-pixel-ratio) {
  @supports (animation: calc(0s)) {
    /* Stylo */
  }
}
Targeting only legacy versions of Firefox is a little tricky — if you're only interested in versions that support @supports, which is Fx 22 and up, @supports not (animation: calc(0s)) is all you need:
@-moz-document url-prefix() {
  @supports not (animation: calc(0s)) {
    /* Gecko */
  }
}
... but if you need to support even older versions, you'll need to make use of the cascade, as demonstrated in the proof-of-concept above.