5

Suppose I have a hypotetical function I'd like to approximate:

def f(x):
    return a * x ** 2 + b * x + c

Where a, b and c are the values I don't know.

And I have certain points where the function output is known, i.e.

x = [-1, 2, 5, 100]
y = [123, 456, 789, 1255]

(actually there are way more values)

I'd like to get a, b and c while minimizing the squared error (and additionally get that squared error).

What is the way to do that in Python?

There should be existing solutions in scipy, numpy or anywhere like that.

kirillbobyrev
  • 1,619
  • 2
  • 14
  • 28
  • what about http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.leastsq.html – Tom83B Feb 23 '16 at 16:19
  • 1
    See this question: [python numpy/scipy curve fitting](https://stackoverflow.com/questions/19165259/python-numpy-scipy-curve-fitting). Not a great question but a great answer. – Peter Wood Feb 23 '16 at 16:19

1 Answers1

6

Since the function you're trying to fit is a polynomial, you can use numpy.polyfit

>>> numpy.polyfit(x, y, 2) # The 2 signifies a polynomial of degree 2
array([  -1.04978546,  115.16698544,  236.16191491])

This means that the best fit was y ~ -1.05 x2 + 115.157x + 236.16.

For a general function, the more you know about it (e.g., is it convex, differentiable, twice differentiable, etc.), the better you can do with scipy.optimize.minimize. E.g., if you hardly know anything about it, you can use it specifying to use the Nelder-Mead method. Other methods there (refer to the documentation) can make use of the Jacobian and the Hessian, if they are defined, and you can calculate them.

Personally, I find that using it with Nelder-Mead (requiring almost no parameters) gives me adequate results for my needs.

Example

Suppose you're trying to fit y = kx with k as the parameter to optimize. You'd write a function

x = ...
y = ...

def ss(k):
    # use numpy.linalg.norm to find the sum-of-squares error between y and kx

Then you'd use scipy.optimize.minimize on the function ss.

Ami Tavory
  • 74,578
  • 11
  • 141
  • 185
  • Yes, generally it's convex and differntiable. – kirillbobyrev Feb 23 '16 at 16:37
  • Thank you! Generally I don't have anything more sophisticated than what I mentioned. I know there's polyfit you mentioned, but what if I have a function, which is exactly y=kx, I.e. I know there's no second parameter, which appears in y=kx+b? The polyfit assumes that there are always n + 1 features in a polynomial of n degree. – kirillbobyrev Feb 23 '16 at 16:44
  • @omtcyf0 See further update, with your last one as an example. – Ami Tavory Feb 23 '16 at 16:47
  • Aha, okay, I see, thank you! So am I correct to assume that If I know exactly how the function looks like (I.e. For example that there are no entries of k degree and so on) Nelder-Mead and minimize is what I'm looking for? The task I'm having is to calculate parameters in an existing physics formula, so I know exactly how it looks like. – kirillbobyrev Feb 23 '16 at 16:48
  • Well, Nelder-Mead + minimize is what I'd personally use. Truth be told, though, if you're willing to take the time to calculate all partial derivatives and stuff, you can use one of the faster-convergence options there. I've never done that it practice (my Convex Optimization prof. must be rolling in his grave). – Ami Tavory Feb 23 '16 at 16:52
  • Got ya! Thank you very much for the answers! – kirillbobyrev Feb 23 '16 at 16:54