Check if pyomo model and generated LP file format is valid and catch error/exception

Question

I have a pyomo ConcreteModel() which I solve repeatedly within another stochastic optimization process whereas one or more parameters are changed on the model.

The basic process can be described as follows:

# model is created as a pyomo.ConcreteModel()
for i in range(0, 10):    
    # change some parameter on the model
    opt = SolverFactory('gurobi', solver_io='lp')
    # how can I check here if the changed model/lp-file is valid?
    results = opt.solve(model)

Now I get an error for some cases where the model and LP file (see gist) seems to contain NaN values:

ERROR: Solver (gurobi) returned non-zero return code (1)
ERROR: Solver log: Academic license - for non-commercial use only Error
    reading LP format file /tmp/tmp8agg07az.pyomo.lp at line 1453 Unrecognized
    constraint RHS or sense Neighboring tokens: " <= nan c_u_x1371_: +1 x434
    <= nan "

    Unable to read file Traceback (most recent call last):
      File "<stdin>", line 5, in <module> File
      "/home/cord/.anaconda3/lib/python3.6/site-
      packages/pyomo/solvers/plugins/solvers/GUROBI_RUN.py", line 61, in
      gurobi_run
        model = read(model_file)
      File "gurobi.pxi", line 2652, in gurobipy.read
      (../../src/python/gurobipy.c:127968) File "gurobi.pxi", line 72, in
      gurobipy.gurobi.read (../../src/python/gurobipy.c:125753)
    gurobipy.GurobiError: Unable to read model Freed default Gurobi
    environment

Of course, the first idea would be to prevent setting these NaN-values. But I don't know why they occur anyhow and want to figure out when the model breaks due to a wrong structure caused by NaNs.

I know that I can catch the solver status and termination criterion from the SolverFactory() object. But the error obviously occurs somewhere before the solving process due to the invalid changed values.

How can I can catch these kinds of errors for different solvers before solving i. e. check if the model/lp-file is valid before applying a solver? Is there some method e.g. check_model() which delivers True or False if the model is (not) valid or something similar?

Thanks in advance!

Yes, it is better to catch the error at the source as you point out. Pyomo seems to have a [BuildCheck](https://pyomo.readthedocs.io/en/latest/working_abstractmodels/BuildAction.html) method for this. Another way is to open the file and search for `nan` (e.g., [here](https://stackoverflow.com/questions/4940032/how-to-search-for-a-string-in-text-files)), but this will be very slow for large models. I think the best option is to understand why `nan`s occur (e.g., division by zero?) and prevent them.. — Ioannis, Jan 02 '19 at 23:42

Qi Chen · Accepted Answer · 2019-01-03T10:25:42.497

1

If you know that the error is taking place when the parameter values are being changed, then you could test to see whether the sum of all relevant parameter values is a valid number. After all, NaN + 3 = NaN.

Since you are getting NaN, I am going to guess that you are importing parameter values using Pandas from an Excel spreadsheet? There is a way to convert all the NaNs to a default number.

Code example for parameter check:

>>> from pyomo.environ import *
>>> m = ConcreteModel()
>>> m.p1 = Param(initialize=1)
>>> m.p2 = Param(initialize=2)
>>> for p in m.component_data_objects(ctype=Param):
...     print(p.name)
... 
p1
p2
>>> import numpy
>>> m.p3 = Param(initialize=numpy.nan)
>>> import math
>>> math.isnan(value(sum(m.component_data_objects(ctype=Param))))
True

Indexed, Mutable Parameters:

>>> from pyomo.environ import *
>>> m = ConcreteModel()
>>> m.i = RangeSet(2)
>>> m.p = Param(m.i, initialize={1: 1, 2:2}, mutable=True)
>>> import math
>>> import numpy
>>> math.isnan(value(sum(m.component_data_objects(ctype=Param))))
False
>>> m.p[1] = numpy.nan
>>> math.isnan(value(sum(m.component_data_objects(ctype=Param))))
True

edited Jan 03 '19 at 10:25

answered Jan 02 '19 at 19:28

Qi Chen

1,648
1
10
19

I am looking for some generic solution because my pyomo models differ in their structure. Nevertheless, thanks for your answer! – Cord Kaldemeyer Jan 02 '19 at 20:18
I edited the answer to add a code check for whether any of the `Param` values attached to the model are `NaN`. This won't work if you directly inject a `NaN` into a constraint expression, though. – Qi Chen Jan 03 '19 at 02:08
I am working with mutable Params which I change on the model subsequently. Your approach works for scalar Parameters using math.isnan (https://docs.python.org/3/library/math.html#math.isnan). But does it also hold for arbitrary model structures e.g. when the parameters have a 2-dimensional index, ...? Can it be extented e.g. by using any() so that it is capable of checking all model structures? Anyway, I like the idea of extracting all data and checking if I know the model structure.. – Cord Kaldemeyer Jan 03 '19 at 08:08
But don't you agree that it would be nice to have something like this included within pyomo? – Cord Kaldemeyer Jan 03 '19 at 08:10
@CordKaldemeyer You can certainly use `any` instead of `sum`. I would consider that an improvement. This code works for arbitrary model structures. I agree--it would be nice to have an organized set of tools like this within Pyomo. Pull requests are welcome. I can claim responsibility for some of the (undocumented) code in pyomo.util: https://github.com/Pyomo/pyomo/tree/master/pyomo/util. – Qi Chen Jan 03 '19 at 10:32
Thanks for your help. As the answer is generic or can be adapted so, I have accepted. Unfortunately, I do not have time to provide a PR at the moment. But I think the developers a now aware and maybe someone else will push a PR based on your code into the code base. Thanks a lot! – Cord Kaldemeyer Jan 03 '19 at 13:54

Check if pyomo model and generated LP file format is valid and catch error/exception

1 Answers1