Constrained least-squares estimation in Python

Question

I'm trying to perform a constrained least-squares estimation using Scipy such that all of the coefficients are in the range (0,1) and sum to 1 (this functionality is implemented in Matlab's LSQLIN function).

Does anybody have tips for setting up this calculation using Python/Scipy. I believe I should be using scipy.optimize.fmin_slsqp(), but am not entirely sure what parameters I should be passing to it.[1]

Many thanks for the help, Nick

[1] The one example in the documentation for fmin_slsqp is a bit difficult for me to parse without the referenced text -- and I'm new to using Scipy.

Do you require using scipy? `cvxpy` is a convex optimization library made for this type of problems. — Geoffrey Negiar, Dec 03 '19 at 05:03

score 7 · Answer 1 · edited May 23 '17 at 12:09

7

scipy-optimize-leastsq-with-bound-constraints on SO givesleastsq_bounds, which is leastsq with bound constraints such as 0 <= x_i <= 1. The constraint that they sum to 1 can be added in the same way.
(I've found leastsq_bounds / MINPACK to be good on synthetic test functions in 5d, 10d, 20d; how many variables do you have ?)

edited May 23 '17 at 12:09

Community

1
1

answered Mar 31 '12 at 12:47

denis

21,378
10
65
88

score 1 · Answer 2 · answered Feb 15 '12 at 15:51

1

Have a look at this tutorial, it seems pretty clear.

answered Feb 15 '12 at 15:51

ev-br

24,968
9
65
78

score 0 · Answer 3 · answered Feb 11 '16 at 16:47

0

Since MATLAB's lsqlin is a bounded linear least squares solver, you would want to check out scipy.optimize.lsq_linear.

answered Feb 11 '16 at 16:47

Milind R

676
1
8
20

Yes, but how do I add linear constraint expressions? The documentation only seems to allow the specification of bounds on the variables. – Realhermit Mar 08 '18 at 00:05

score 0 · Answer 4 · answered Oct 06 '22 at 13:41

Non-negative least squares optimization using scipy.optimize.nnls is a robust way of doing it. Note that, if the coefficients are constrained to be positive and sum to unity, they are automatically limited to interval [0,1], that is one need not additionally constrain them from above.

scipy.optimize.nnls automatically makes variables positive using Lawson and Hanson algorithm, whereas the sum constraint can be taken care of as discussed in this thread and this one.

Scipy nnls uses an old fortran backend, which is apparently widely used in equivalent implementations of nnls by other software.

Constrained least-squares estimation in Python

4 Answers4

Linked