highspy API improvements and benchmark

[rlloretb]

I am exploring highspy as an opensource alternative to gurobipy.
Which is the current roadmap to improve highspy?
For instance, in Gurobi it is possible to natively use dicts (tupledicts) to create and access variables efficiently, in addition to the quicksum function.
Also, are there any benchmarks for highspy model building time for large models (10 - 100k variables/constraints)?

[jajhall]

In short, we can't see us developing the modelling features of highspy much. It's just there to allow simple illustrative examples to be presented when introducing HiGHS.

That said, we're aware of some limitations when accessing from Python the data generated by the HiGHS C++ code.

[rlloretb]

I see. Is this because most HiGHS users are comfortable with working with C++ directly, or because you guys externalize this to third-party projects such as JuMP or pyomo?

[jajhall]

Essentially, yes. We also have very little Python expertise in the HiGHS team, so we couldn't do much if we wanted to.

There are plenty of users working through the C, C#, Julia, Fortran and JavaScript interfaces to HiGHS

[zhouxinsheng]

Essentially, yes. We also have very little Python expertise in the HiGHS team, so we couldn't do much if we wanted to.

There are plenty of users working through the C, C#, Julia, Fortran and JavaScript interfaces to HiGHS

By the way, I have another some questions in using current highspy version:
1, Is any function to get problem type (LP, QP, MIP)?
2, How to add quadratic term to objective or constraint, and how to get quadratic info in objective and constraint?
3, How to get a constraint's slack value in solution?
4, is there any difference between h.run() and h.solve()?
5, how to convert HighsSparseMatrix to scipy.sparse.csr_matrix? I have to calculate inverted B for getBasisInverseCol and getBasisInverseRow not implemented in python api.
6, is h.getIis() is the conflict analyse function, can't find its usage in web?

[jajhall]

1. There is no function to get problem type (LP, QP, MIP). The modeller will know this
2. HiGHS can only solve QPs, so quadratic terms can only appear in the objective. The can be passed as part of the original model, or via highs_passHessian or highs_passHessianPointers
3. Using getSolution, the values of slack variables are the constraint activities in solution.row_value
4. solve() is only in highspy and has some features to facilitate threading in Python and handle keyboard interrupts
5. I don't know how to convert HighsSparseMatrix to scipy.sparse.csr_matrix. Interesting that you want to use the equivalent of getBasisInverseCol and getBasisInverseRow in highspy. We didn't add them to highspy, as we expected the use of such sophisticated methods not to occur in Python. We can add them in due course.
6. Sorry, h.getIis() was added prematurely. We should remove it until implementation of IIS is complete.

[zhouxinsheng]

Thanks! I'm writing python code on power grid unit commitment and economic dispatch tasks in which large scale (million variables and constraints) MIP or LP models are built and solved.
In my study, models are modified frequently after built, sometimes, problem type is changed, so I want to know the current problem type through some get_problem_type() function, now I do this by checking the h.getLp().integrality_ list, LP is returned if no kInteger exists, otherwise MIP.
Sometimes, I also need to dive a little deeper into solution, getBasisInverseCol and getBasisInverseRow is used to study how the basic variables will change in respond to constraint right hand disturbance, this cannot be obtained through sensitivity analysis. Now I get the B matrix through extracting the basic col and row of h.getLp().a_matrix_, then calculate the inverse of B.
And, as discussed in #1996, the IIS feature is valuable to tell us why the model is infeasible, so we can revise it, especially in large scale problem with complex constraints.

[metab0t]

@rlloretb
You can take a look at PyOptInterface if you are interested in high-performant modelling in Python. It supports HiGHS out of the box.

It has built-in support for tupledict and quicksum like gurobipy for variables and constraints.

[rlloretb]

Thanks for the reference. I see that callbacks are not supported for HiGHS. Is there any plan for supporting them?

[metab0t]

I can add callback support for HiGHS like Gurobi and COPT with some effort if you are interested.

However, I rarely use callbacks in my daily research, so it will need more tests by users.

[rlloretb]

Great. I am happy to help with testing.