18

I would like to start using Python for modelling and solving optimization problems. I would like to use both single-objective problems and multi-objective problems with a multidimensional objective space. For the multiobjective problems I'd like to use a metaheuristic, something like multiobjective evolutionary algorithms (like NSGA-2) for solving it.

Now my question is, which Python package for OR is suitable for doing this? Can I for example use something like:

  • Pyomo
  • Pulp
  • Pyopt

I'd appreciate every comment and I'd be quite thankful for your help.

Update: Here is a more detailed desciption of what I intend to do. Basically I have a multiobjective optimization problem (mixed-integer linear program) with 2 objectives and I would like to compare three methods in different sceanrios with varying complexity:

  1. Weighted sum approach solved by an exaxt algorithm (e.g. using a commerical solver like CPLEX)
  2. Weightes sum approach solved by a single-objetice metaheuristic (like conventional evolutionary algorithms or particle swarm optimization)
  3. Real multiobjectice optimization with a metaheuristic (like NSGA-2 or MOPSO)

I'd like to do this all in Python, as I read here in the forum that Python is strongly used in the OR community. Which packages would you advice me to use?

Additional note: With real multiobjective optimization I mean, not to use a weighted sum approach (and thus convert the objective space into a one-dimensional space) but to have a multidimensional objective space and try to find the Pareto optimal solutions (e.g. with NSGA-2 which is a 'real' multiobjective optimization metaheuristic)

PeterBe
  • 1,632
  • 2
  • 15
  • 30
  • If you don't find that the answers by @hasson or me adequately address your concern, I suggest you follow my suggestion to edit the question to provide explicit details as to what you do and don't want. – Mark L. Stone Aug 14 '20 at 16:24
  • @MarkL.Stone Thanks Mark for your comments. I just did what you told me. – PeterBe Aug 17 '20 at 14:59
  • I suggest you explain what "Real multiobjective optimization" is. – Mark L. Stone Aug 17 '20 at 16:05
  • Hi Mark, thanks for your comment. I am sorry for imprecise term. With real multiobjective optimization I mean, not to use a weighted sum approach (and thus convert the objective space into a one-dimensional space) but to have a multidimensional objective space and try to find the Pareto optimal solutions (e.g. with NSGA-2 which is a 'real' multiobjective optimization metaheuristic). – PeterBe Aug 17 '20 at 16:22
  • 2
    I suggest you edit this enhanced explanation into the question. – Mark L. Stone Aug 17 '20 at 16:39
  • @MarkL.Stone: Thanks Mark for your comment. I just did that. Hopefully someone can answer my question and give me good advice – PeterBe Aug 18 '20 at 08:16
  • Thanks for updating the question. I have updated my answer according to the new scope. – dhasson Aug 19 '20 at 23:56

5 Answers5

16

If you use packages like PyOMO, PuLP or pyOpt, you'd have to implement all the operations for multiobjective optimization - e.g. to find nondominated solutions or the different mutation operators - that could take some time. An alternative is using DEAP for that, it's a Python framework for evolutionary algorithm and they have NSGA-II implemented. It's quite customizable and you can also easily interact with other Python libraries in the routines (e.g. for mutation and crossover operations). A second library is jMetalPy, which has a broad scope with more multiobjective optimization algorithms implemented (DEAP is focused on evolutionary algorithms).

A second alternative is to model some objectives as a budget constraint and use pyomo, pulp, etc, with a varying parameter for that constraint's bound. In the end you'll have found a set of optimal solutions and will be able approximate the nondominated (Pareto) front. There are also some LP- and MIP-specific multiobjective optimization algorithms in the literature. See for example this this GitHub project which is compatible with Julia

Other alternatives, like taking a linear combination of objectives, are contained in Mark's answer.

To answer the updated question: OP wants to compare three methods for multiobjective mixed-integer linear program with 2 objectives, in different scenarios with varying complexity, using Python:

  1. Weighted sum approach solved by an exact algorithm
  2. Weighted sum approach solved by a single-objetive metaheuristic
  3. Multiobjective optimization with a metaheuristic (like NSGA-2 or MOPSO), having a multidimensional objective space and trying to find the Pareto optimal solutions.

I recommend the following for each scenario:

For the weighted sum approach, use PyOMO. This way you'll dominate a Python module that allows you to interact with either Gurobi, CPLEX, GLPK, CBC, Mosek, BARON, among other solvers, allowing to be more tool-agnostic than if you worked with a specific software's API. Moreover, there's GAMS/PYOMO which allows users to solve GAMS models using solvers within the PyOMO modeling system. This can be useful as you stated having used GAMS in the past.

For scenarios 2. and 3., you can use jMetalPy which has several kinds of algorithms implemented for single-objective (Evolution Strategy, Genetic Algorithm, Local Search, Simulated annealing) and many more for multi-objective: 8 Evolutionary Algorithms (GDE3, HYPE, IBEA, MOCell, MOEA/D, NSGA-II, NSGA-III, SPEA2) and 2 PSO Algorithms (OMOPSO, SMPSO). This way, you'll learn only one library that can get you a whole variety of algorithms and tests available.

dhasson
  • 1,667
  • 1
  • 9
  • 21
  • Thanks dhasson for your answer. My main goal is to use Python for optimization as I heard that it is strongly used in industry. So basically I would like to use a general python package for optimization and (later) use multiobjective optimization approaches. So I would like to also use normal (one-dimensional) solvers like CPLEX for the optimization. – PeterBe Aug 12 '20 at 13:00
  • Yes of course, Python is widely used in industry and you can use it to learn and implement that stuff.. Nonetheless, if you need to create a high-performant system in the future, you might want to reimplement the Python solution/prototype, or parts of it, in a low level language like C++. This will depend on several factors. – dhasson Aug 12 '20 at 13:30
  • Thanks dhasson for your answer. But for me there is still the question if I should use Pyomo, Pulp, Pyopt or Deap (as you suggested). I would like to do normal optimizaiton and multiobjective optimization. I also found jmetalPy (https://github.com/vOptSolver/vOptSolver). I know jMetal from Java. So which of those is the most widely used optimization package for Python when it comes to normal and multiobjective optimization? – PeterBe Aug 12 '20 at 16:02
  • That could depend on the specific use case you have in mind - do you want to solve problems that can be modelled on them (e.g. linear, integer, quadratic constraints) or not necessarily? The basic ideas on Pyomo an Pulp are quite similar, but the implementations and some functionalities differ. As a reference, counting StackOverflow tags it seems that Pyomo is used more than PuLP (around 530 with tag ‘pulp’ vs 780 for ‘pyomo’). For single-objective optimization you could use Pyomo or Pulp or even CPLEX and Gurobi's APIs mentioned in Mark's answer (if you have a license for that software). – dhasson Aug 12 '20 at 18:25
  • The jMetalPy Github repository has 16 contributors, 180 stars and around 70 forks, while the DEAP one has 57 contributors, 3800 stars and 800 forks. So it's likely DEAP is used more than jMetalPy in general (expected result, as it's counting evolutionary algorithms and multi-objective users). Note that jMetalPy has more multi-objective algorithms implemented, as can be seen in DEAP's and jMetalPy's algorithm lists and according to this 2019 comparison by jMetalPy authors. – dhasson Aug 12 '20 at 18:43
  • Thanks dhasson for your detailed comments and your help. In my specific use case I would like to solve a mixed-integer linear optimization problem with 2 objectives. So you would advice me to use Pyomo for the single objetive problems and DEAP for the multiobjetive problems if I understood correctly? – PeterBe Aug 13 '20 at 09:05
  • I also have a follow up question: Can I use single-objective metaheuristics (like EA and PSO) in Pyomo or Pulp? – PeterBe Aug 13 '20 at 10:27
  • No, for that you need to use an adequate metaheuristics implementation as I said before (e.g. Deap or jMetalpy). Pulp and pyomo are mainly modeling interfaces that interact with mathamatical optimization solvers like CBC, CLP, CPLEX, Gurobi, among others. – dhasson Aug 13 '20 at 13:19
  • Thanks for your answer and help dhasson. You said that Pulp and pyomo are mainly modeling interfaces. So can I use them to generally model the optimization problem and then use a metaheuristic package (like Deap or jMetalpy) to solve the optimization problem without modelling it again? The strinking point here is that I assume that each of the packages (Pyomo, Pulp, Deap etc.) have different syntax and conventions for modelling optimization problem. I'd like to model it only once as learning different modelling syntaxes might be too timeconsuming. – PeterBe Aug 13 '20 at 14:11
  • No, can't do that, you'll have to learn different syntaxes – dhasson Aug 13 '20 at 14:14
  • Thanks for your comment dhasson. That is really a pitty and quite bad for me. Can you think about a way how I can only model it once and then let different solvers (or optimization algorithms) solve it? If not, can you tell me - just based on your own experience - if Deap is more similar to Pulp, Pyomo or Pyopt? – PeterBe Aug 13 '20 at 14:57
  • Can I implement an evolutionary algorithm on my own in Pulp, Pyomo or Pyopt? And is Deap similar to one of those packages? I'd appreciate every input and would be quite thankful if you could share your experience. – PeterBe Aug 14 '20 at 14:37
  • Without a concrete idea and use in mind, I don't think deap is similar to a relevant portion of any of those modeling packages. Why would you want to implement an evolutionary algo in pulp/pyomo/pyopt? Those libraries work with specific structures and software design in mind, not necessarily useful or easily compatible for coding genetic programs or swarm optimization. Please refer to the different projects' documentation and repositories to understand this further and better. – dhasson Aug 14 '20 at 19:18
  • Thanks dhasson for your comments and help. I updated my original question to give you a little bit more background information about what I am inteding to do. – PeterBe Aug 17 '20 at 14:58
  • @PeterBe, would you try using an algebraic modeling language like GAMS? It really does have a dozen solvers involved commercial and open-source like CPLEX, Gurobi, CBC, etc. Also, its language has capability to implement some useful algorithms like eps-constraint, etc to solve the multi-objective optimization problems efficiently. As it is a commercial software, you need a valid license. – A.Omidi Aug 17 '20 at 15:36
  • Thanks Omidi for your answer. Bascially I have been using GAMS for 2 years and I would like to move away from it for several reasons the main being that it is not a general purpose programming language and if you want to use the results of the optimization for something else (like simulations in my case) it is really inconvenient to link GAMS to a programming language like Java (especially the data exchange and the debuggin drove me nuts). Further, as far as I know you can't use evolutionary algorithms or other metaheuristics with GAMS. And I doubt that you can really do multiobjective opt – PeterBe Aug 17 '20 at 16:25
  • @PeterBe, GAMS really has many low-level API's like Java, Python, etc to exchange the data and some manipulating in the optimization model. Also, it comes with an excellent facility to connect with other programs like DB or Excel which is frequently used in the industry. I think the biggest limitation of using GAMS is its separate license for each solver. I really do not know that GAMS has the capability to implement some algorithms like GA but, MOP can be solved by implementing useful methods such as goal programming, lexicograph, etc. – A.Omidi Aug 18 '20 at 13:35
  • Thanks Omidi for your answer. As stated before, I used GAMS together with JAVA and it was quite unconvient for me to switch all the time between the two languages and editors. Further, the debugging drove me nuts because GAMS uses a special data exchange format (.gdx files) that I did not like at all. My personal impression is that GAMS is not really a 'moder' language (altough it has some nice features). Bottom line is that I definetely want to move away from it. The other big reason for that is that I can't use metaheuristics (I once asked this in the GAMS forum and nobody answered). – PeterBe Aug 18 '20 at 13:43
  • 1
    @PeterBe, Ok, That's right. If you are interested in Python, one of the best choices (I think) is Pyomo. As you have experienced in the GAMS language, it has near syntax with it. Another reason is, it can be connected to many of the different solvers by NEOS interface in which solves the broad range of the optimization problems. As per it is python-based, you might combine it with lots of useful packages to implement what you are trying to do. – A.Omidi Aug 18 '20 at 19:14
  • @dhasson: Thanks for your updated new answer dhasson. I have some follow up questions. 1) You first mentioned that you would rather use DEAP instead of jMetalPy as you can also use multiobjective optimization there. Why did you change your inital opinion? 2) Can I use metaheuristic solvers in Pyomo? I'd appreciate further comments – PeterBe Aug 20 '20 at 08:34
  • as mentioned before in the comments here, jmetalpy has more algorithms implemented. As in the new scope of the question you apparently want to be able to use the largest quantity of heuristics using the minimum number of frameworks possible, and mention particle swarm optimization (which isn't available in deap), I adapted the answer to the new information. 2) no you can't, otherwise I would have mentioned only pyomo as an answer to the whole question.
    • – dhasson Aug 20 '20 at 14:02
  • Thanks dhasson for your comments and great effort. Maybe one last question (afterward I will accept your answer): Would your answer change if I did not want to do the metaehristic multiobjective optimization and for the metaheuristic singleobjective optimization evolutionary algorithms would be enough (no PSO etc.)? For all 3 mentionded scenarions your adivce would be to use Pyomo and jMetalPy. What about the 'reduced' scope? Would you still go with them? – PeterBe Aug 21 '20 at 08:11
  • 1
    I would stay with pyomo and jmetalpy, as it makes it simpler to extend for other available metaheuristics (e.g. simulated annealing) if I wanted to extend the scope in the future. Moreover, if I decide to work with other metaheuristics in the future (for another project), I'd know a framework with a larger variety of algorithms available (with respect to deap). – dhasson Aug 21 '20 at 14:53
  • Thanks dhasson for your great help. I accepted your answer and awarded a bounty to you. – PeterBe Aug 21 '20 at 15:58
  • You're welcome, good luck with the learning! – dhasson Aug 21 '20 at 16:05