What are the benefits of switching from Rule and /. to OptionsPattern[] and OptionValue in a large application?

Question

Old habits die hard, and I realise that I have been using opts___Rule pattern-matching and constructs like thisoption /. {opts} /. Options[myfunction] in the very large package that I'm currently developing. Sal Manango's "Mathematica Cookbook" reminds me that the post-version-6 way of doing this is opts:OptionsPattern[] and OptionValue[thisoption]. The package requires version 8 anyway, but I had just never changed the way I wrote this kind of code over the years.

Is it worth refactoring all that from my pre-version-6 way of doing things? Are there performance or other benefits?

Regards

Verbeia

EDIT: Summary

A lot of good points were made in response to this question, so thank you (and plus one, of course) all. To summarise, yes, I should refactor to use OptionsPattern and OptionValue. (NB: OptionsPattern not OptionPattern as I had it before!) There are a number of reasons why:

1. It's a touch faster (@Sasha)
2. It better handles functions where the arguments must be in HoldForm (@Leonid)
3. OptionsPattern automatically checks that you are passing a valid option to that function (FilterRules would still be needed if you are passing to a different function (@Leonid)
4. It handles RuleDelayed (:>) much better (@rcollyer)
5. It handles nested lists of rules without using Flatten (@Andrew)
6. It is a bit easier to assign multiple local variables using OptionValue /@ list instead of having multiple calls to someoptions /. {opts} /. Options[thisfunction] (came up in comments between @rcollyer and me)

EDIT: 25 July I initially thought that the one time using the /. syntax might still make sense is if you are deliberately extracting a default option from another function, not the one actually being called. It turns out that this is handled by using the form of OptionsPattern[] with a list of heads inside it, for example: OptionsPattern[{myLineGraph, DateListPlot, myDateTicks, GraphNotesGrid}] (see the "More information" section in the documentation). I only worked this out recently.

Answer 1

It seems like relying on pattern-matcher yields faster execution than by using PatternTest as the latter entails invocation of the evaluator. Anyway, my timings indicate that some speed-ups can be achieved, but I do not think they are so critical as to prompt re-factoring.

In[7]:= f[x__, opts : OptionsPattern[NIntegrate]] := {x, 
  OptionValue[WorkingPrecision]}

In[8]:= f2[x__, opts___?OptionQ] := {x, 
  WorkingPrecision /. {opts} /. Options[NIntegrate]}

In[9]:= AbsoluteTiming[Do[f[1, 2, PrecisionGoal -> 17], {10^6}];]

Out[9]= {5.0885088, Null}

In[10]:= AbsoluteTiming[Do[f2[1, 2, PrecisionGoal -> 17], {10^6}];]

Out[10]= {8.0908090, Null}

In[11]:= f[1, 2, PrecisionGoal -> 17]

Out[11]= {1, 2, MachinePrecision}

In[12]:= f2[1, 2, PrecisionGoal -> 17]

Out[12]= {1, 2, MachinePrecision}

Answer 2

A little known (but frequently useful) fact is that options are allowed to appear in nested lists:

In[1]:= MatchQ[{{a -> b}, c -> d}, OptionsPattern[]]

Out[1]= True

The options handling functions such as FilterRules know about this:

In[2]:= FilterRules[{{PlotRange -> 3}, PlotStyle -> Blue, 
  MaxIterations -> 5}, Options[Plot]]

Out[2]= {PlotRange -> 3, PlotStyle -> RGBColor[0, 0, 1]}

OptionValue takes it into account:

In[3]:= OptionValue[{{a -> b}, c -> d}, a]

Out[3]= b

But ReplaceAll (/.) doesn't take this into account of course:

In[4]:= a /. {{a -> b}, c -> d}

During evaluation of In[4]:= ReplaceAll::rmix: Elements of {{a->b},c->d} are a mixture of lists and nonlists. >>

Out[4]= a /. {{a -> b}, c -> d}

So, if you use OptionsPattern, you should probably also use OptionValue to ensure that you can consume the set of options the user passes in.

On the other hand, if you use ReplaceAll (/.), you should stick to opts___Rule for the same reason.

Note that opts___Rule is also a little bit too forgiving in certain (admittedly obscure) cases:

Not a valid option:

In[5]:= MatchQ[Unevaluated[Rule[a]], OptionsPattern[]]

Out[5]= False

But ___Rule lets it through:

In[6]:= MatchQ[Unevaluated[Rule[a]], ___Rule]

Out[6]= True

Update: As rcollyer pointed out, another more serious problem with ___Rule is that it misses options specified with RuleDelayed (:>). You can work around it (see rcollyer's answer), but it's another good reason to use OptionValue.

Answer 3

While several answers have stressed different aspects of old vs. new way of using options, I'd like to make a few additional observations. The newer constructs OptionValue - OptionsPattern provide more safety than OptionQ, since OptionValue inspects a list of global Options to make sure that the passed option is known to the function. The older OptionQ seems however easier to understand since it is based only on the standard pattern-matching and isn't directly related to any of the global properties. Whether or not you want this extra safety provided by any of these constructs is up to you, but my guess is that most people find it useful, especially for larger projects.

One reason why these type checks are really useful is that often options are passed as parameters by functions in a chain-like manner, filtered, etc., so without such checks some of the pattern-matching errors would be very hard to catch since they would be causing harm "far away" from the place of their origin.

In terms of the core language, the OptionValue - OptionsPattern constructs are an addition to the pattern-matcher, and perhaps the most "magical" of all its features. It was not necessary semantically, as long as one is willing to consider options as a special case of rules. Moreover, OptionValue connects the pattern-matching to Options[symbol] - a global property. So, if one insists on language purity, rules as in opts___?OptionQ seem easier to understand - one does not need anything except the standard rule-substitution semantics to understand this:

f[a_, b_, opts___?OptionQ] := Print[someOption/.Flatten[{opts}]/.Options[f]]

(I remind that the OptionQ predicate was designed specifically to recognize options in the older versions of Mathematica), while this:

f[a_, b_, opts:OptionsPattern[]] := Print[OptionValue[someOption]]

looks quite magical. It becomes a bit clearer when you use Trace and see that the short form of OptionValue evaluates to a longer form, but the fact that it automaticaly determines the enclosing function name is still remarkable.

There are a few more consequences of OptionsPattern being a part of the pattern language. One is the speed improvements discussed by @Sasha. However, speed issues are often over-emphasized (this is not to detract from his observations), and I expect this to be especially true for functions with options, since these tend to be the higher-level functions, which will likely have non-trivial body, where most of the computation time will be spent.

Another rather interesting difference is when one needs to pass options to a function which holds its arguments. Consider a following example:

ClearAll[f, ff, fff, a, b, c, d];
Options[f] = Options[ff] = {a -> 0, c -> 0};
SetAttributes[{f, ff}, HoldAll];
f[x_, y_, opts___?OptionQ] :=
   {{"Parameters:", {HoldForm[x], HoldForm[y]}}, {" options: ", {opts}}};
ff[x_, y_, opts : OptionsPattern[]] :=
   {{"Parameters:", {HoldForm[x], HoldForm[y]}}, {" options: ", {opts}}};

This is ok:

In[199]:= f[Print["*"],Print["**"],a->b,c->d]
Out[199]= {{Parameters:,{Print[*],Print[**]}},{ options: ,{a->b,c->d}}}

But here our OptionQ-based function leaks evaluation as a part of pattern-matching process:

In[200]:= f[Print["*"],Print["**"],Print["***"],a->b,c->d]
During evaluation of In[200]:= ***
Out[200]= f[Print[*],Print[**],Print[***],a->b,c->d]

This is not completely trivial. What happens is that the pattern-matcher, to establish a fact of match or non-match, must evaluate the third Print, as a part of evaluation of OptionQ, since OptionQ does not hold arguments. To avoid the evaluation leak, one needs to use Function[opt,OptionQ[Unevaluated[opt]],HoldAll] in place of OptionQ. With OptionsPattern we don't have this problem, since the fact of the match can be established purely syntactically:

In[201]:= ff[Print["*"],Print["**"],a->b,c->d]
Out[201]= {{Parameters:,{Print[*],Print[**]}},{ options: ,{a->b,c->d}}}

In[202]:= ff[Print["*"],Print["**"],Print["***"],a->b,c->d]
Out[202]= ff[Print[*],Print[**],Print[***],a->b,c->d]

So, to summarize: I think choosing one method over another is largely a matter of taste - each one can be used productively, and also each one can be abused. I am more inclined to use the newer way, since it provides more safety, but I do not exclude that there exist some corner cases when it will surprise you - while the older method is semantically easier to understand. This is something similar to C-C++ comparison (if this is an appropriate one): automation and (possibly) safety vs. simplicity and purity. My two cents.

Answer 4

Your code itself has a subtle, but fixable flaw. The pattern opts___Rule will not match options of the form a :> b, so if you ever need to use it, you'll have to update your code. The immediate fix is to replace opts___Rule with opts:(___Rule | ___RuleDelayed) which requires more typing than OptionsPattern[]. But, for the lazy among us, OptionValue[...] requires more typing than the short form of ReplaceAll. However, I think it makes for cleaner reading code.

I find the use of OptionsPattern[] and OptionValue to be easier to read and instantly comprehend what is being done. The older form of opts___ ... and ReplaceAll was much more difficult to comprehend on a first pass read through. Add to that, the clear timing advantages, and I'd go with updating your code.