I just tried to install 0.5.0, but the installation instructions are not working. They break because make is pointed to the installation folder instead of the source folder when looking for SCIP headers, i.e. at

if [ -e $(SCIPOPTDIR)/include/scip/scip.h ] ; \

while scip.h is in

/path/to/source/scipoptsuite-5.0.0/scip/src/scip/scip.h

I didn't wanted to submit a pull request because I don't know the best way to proceed here. Should there be two environment variables one pointing to the build directory and another pointing to the source directory? is there maybe a better approach?

@leethargo I guess I'll need some help here regarding the versioning of CSIP.
Should we go to CSIP 0.3.7 or CSIP 0.4.0 ?

The interface of CSIP didn't change

Hi all,
I come up with an error when building SCIP with "make SHARED=true GMP=false READLINE=false ZLIB=false scipoptlib" before building CSIP, as:
9801a7b319bb:scip-4.0.0 baohuaw$ make SHARED=true GMP=false READLINE=false ZLIB=false scipoptlib
make/make.project:477: ZIMPL was deactived because of missing GMP support.
make: *** No rule to make target `scipoptlib'. Stop.
Can you please support me?
Thx

Instead of asking the users (who run the makefile) to supply the locations for the SCIP headers and the scipoptlib.so shared library, the Makefile should look for the environment variable SCIPOPTDIR and created the needed links based on that path.

This is also how PySCIPOpt's setup.py is working now.

For MathProgBase, we need to set the variable types after the variables were added with loadproblem!. This can probably be done with SCIPchgVarType in the problem stage.

I don't remember why we (or I) decided to use CONST expressions to represent exponents of POW expressions.

So, for instance, to represent z^2 where z is the third variable of some problem, the code is like:

    int nops = 3; 
    CSIP_OP ops[] = {VARIDX, CONST, POW};
    int children[] = {2, 0, 0, 1};
    int begin[] = {0, 1, 2, 4};
    double values[] = {2.0};

see test.c::test_nlp

However, that CONST expression is never used when creating the expression tree, since when creating the POW expression the code looks directly into values:

        case SCIP_EXPR_REALPOWER:
            assert(2 == begin[i + 1] - begin[i]);
            {
                double exponent;
                // the second child is the exponent which is a const
                exponent = values[children[begin[children[begin[i] + 1]]]];
                printf("Seeing a power with exponent %g (nchild %d)\n", exponent, begin[i+1] - begin[i]);
                SCIP_in_CSIP(SCIPexprCreate(SCIPblkmem(scip), &exprs[i],
                                            ops[i], exprs[children[begin[i]]], exponent));
            }
            break;

This of course produces a memory leak, since the created CONST expression is never used, hence, never freed

Separated from #3

Some results (solution, runtime etc.) are reset after freeTransform is called.

With #24 we do that implicitely, whenever necessary. We should at least warn the users about this behavior in the README. Maybe it's possible to warn them when specific functions are called. See also #23.

CSIP fails of compile when I run Pkg.build("SCIP"). The output is cryptic/difficult to debug:

INFO: Building SCIP
INFO: Attempting to Create directory /home/adowling2/.julia/v0.4/SCIP/deps/downloads
INFO: Directory /home/adowling2/.julia/v0.4/SCIP/deps/downloads already created
INFO: Downloading file https://github.com/SCIP-Interfaces/CSIP/archive/v0.3.5.zip
INFO: Done downloading file https://github.com/SCIP-Interfaces/CSIP/archive/v0.3.5.zip
INFO: Attempting to Create directory /home/adowling2/.julia/v0.4/SCIP/deps/src
INFO: Directory /home/adowling2/.julia/v0.4/SCIP/deps/src already created
INFO: Attempting to Create directory /home/adowling2/.julia/v0.4/SCIP/deps
INFO: Directory /home/adowling2/.julia/v0.4/SCIP/deps already created
INFO: Path /home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5 already created
INFO: Changing Directory to /home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5
gcc -std=c99 -Wall -pedantic -I/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/lib/include -I/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/include -c /home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/src/csip.c -L/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/lib -Wl,-rpath,/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/lib -lscipopt -fPIC -o /home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/src/csip.o
/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/src/csip.c: In function ‘CSIPheurAddSolution’:
/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/src/csip.c:1467:5: warning: passing argument 7 of ‘SCIPtrySolFree’ makes integer from pointer without a cast [enabled by default]
     SCIP_in_CSIP(SCIPtrySolFree(scip, &sol, FALSE, TRUE, TRUE, TRUE, &stored));
     ^
In file included from /home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/src/csip.c:3:0:
/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/lib/include/scip/scip.h:17947:14: note: expected ‘unsigned int’ but argument is of type ‘unsigned int *’
 SCIP_RETCODE SCIPtrySolFree(
              ^
/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/src/csip.c:1467:5: error: too few arguments to function ‘SCIPtrySolFree’
     SCIP_in_CSIP(SCIPtrySolFree(scip, &sol, FALSE, TRUE, TRUE, TRUE, &stored));
     ^
In file included from /home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/src/csip.c:3:0:
/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/lib/include/scip/scip.h:17947:14: note: declared here
 SCIP_RETCODE SCIPtrySolFree(
              ^
make: *** [/home/adowling2/.julia/v0.4/SCIP/deps/src/CSIP-0.3.5/lib/libcsip.so] Error 1
================================[ ERROR: SCIP ]=================================

LoadError: failed process: Process(`make`, ProcessExited(2)) [2]
while loading /home/adowling2/.julia/v0.4/SCIP/deps/build.jl, in expression starting on line 34

================================================================================

I have compiled SCIP per the instructions using

make IPOPT=true SHARED=true GMP=false READLINE=false ZLIB=false scipoptlib

and have confirmed it works with make test. I have also set SCIPOPTDIR. Any suggestions on further debugging this problem?

This doesn't work properly, with the current implementation.

I noticed while testing the setting of objective sense. After the solve is finished, we still have the transformed problem, it seems. When setting another objective sense and calling solve again, nothing happens (the old solution is still considered optimal).

My workaround here was to call SCIPfreeTransform() before changing the objective sense. But this would need to be done before any change of the model (add var, add cons, change bound...) and looks horrible.

At the moment, only adding linear constraints is implemented, which is what all the solvers can do.

We should also support adding quadratic and SOS1/SOS2 constraints, because it's easy.

I was running valgrind on the tests and it found various leaks within CSIPaddLazyCallback:

==8578== 29 bytes in 1 blocks are still reachable in loss record 1 of 12
==8578==    at 0x4C28C20: malloc (vg_replace_malloc.c:296)
==8578==    by 0x6098989: strdup (strdup.c:42)
==8578==    by 0x573C7D1: addMemlistEntry (memory.c:159)
==8578==    by 0x573CCC3: BMSallocMemory_call (memory.c:379)
==8578==    by 0x405160: CSIPaddLazyCallback (in /home/rs/src/CSIP/test/test)
==8578==    by 0x4024B2: test_lazy (in /home/rs/src/CSIP/test/test)
==8578==    by 0x40380F: all_tests (in /home/rs/src/CSIP/test/test)
==8578==    by 0x403984: main (in /home/rs/src/CSIP/test/test)
[...]
==8578== 40 bytes in 1 blocks are still reachable in loss record 5 of 12
==8578==    at 0x4C28C20: malloc (vg_replace_malloc.c:296)
==8578==    by 0x573CC76: BMSallocMemory_call (memory.c:370)
==8578==    by 0x405160: CSIPaddLazyCallback (in /home/rs/src/CSIP/test/test)
==8578==    by 0x4024B2: test_lazy (in /home/rs/src/CSIP/test/test)
==8578==    by 0x40380F: all_tests (in /home/rs/src/CSIP/test/test)
==8578==    by 0x403984: main (in /home/rs/src/CSIP/test/test)
==8578== 
==8578== 40 bytes in 1 blocks are still reachable in loss record 6 of 12
==8578==    at 0x4C28C20: malloc (vg_replace_malloc.c:296)
==8578==    by 0x573C784: addMemlistEntry (memory.c:154)
==8578==    by 0x573CCC3: BMSallocMemory_call (memory.c:379)
==8578==    by 0x405160: CSIPaddLazyCallback (in /home/rs/src/CSIP/test/test)
==8578==    by 0x4024B2: test_lazy (in /home/rs/src/CSIP/test/test)
==8578==    by 0x40380F: all_tests (in /home/rs/src/CSIP/test/test)
==8578==    by 0x403984: main (in /home/rs/src/CSIP/test/test)
==8578== 
[...]

At first I thought that we had to SCIPfreeMemory all the CONSHDLRDATA*. But when I tried that, I got a "double free" error. Do we need to free something that is stored within this callback data?

In Makefile, GNU style flag -Wl,-rpath=$(CSIPLIBDIR) is not working on Mac. Instead, it should be -Wl,-rpath,$(CSIPLIBDIR).

Currently, the lazy constraint callback is called both in CONSENFOLP, CONSENFOPS and CONSCHECK. In the last case, no cut is actually added to the problem.

This means that the same solution is handled several times by the callback, potentially wasting time. See also the related issue in SCIP.jl.

We should definitely set a smaller check priority for the lazy constraint callback.
In addition, we could try to add the cuts even from CONSCHECK. Here we'd have to check the STAGE first.

SCIP supports fairly general nonlinear constraints given as expression graph.

The expression graph could be input via the adjacency matrix and another vector for the symbolic meaning of the nodes.

Up to now, we called SCIPfreeTransform directly after SCIPsolve, but stored (copied) some important results, such as best solution, objective value and status.

We now want to expose more results (such as solving time, see #18 in SCIP.jl) and this approach is no longer viable.

First, we should remove the call to SCIPfreeTransform in the solve, but introduce it in all modifying methods.

Then (optionally) clean up some of the copied results that we no longer need.

There should be a conda recipe for CSIP, because there's already one for scipoptlib, so the dependency could be covered implicitely.

If libscipopt.so was compiled with IPOPT=true, then our build will fail, because of missing linker flags. I don't know whether we should attempt to detect the dependencies of the SCIP library (such as readline, gmp and zlib) or add build variables of our own?

Supporting IPOPT would really be nice, but requiring it (by default) is not.

We want to use CSIP in SCIP.jl and implement the LinearQuadraticModel interface of MathProgBase. The current functionality of CSIP is not sufficient and needs to be extended.

• allow callbacks to stop solving with status "user limit" (d95e449)

MathProgBase supports MIP-warmstarts with partial solutions, where undefined variables are given NaN values.

In the upcoming SCIP release, new methods SCIPcreatePartialSol etc. are introduced. We should then always create partial solutions. If they happen to be complete, SCIP will just add them.

I'm interested in using CSIP (from Julia with SCIP.jl) to solve nonlinear optimizations involving trig functions, but it looks like CSIP doesn't currently support those expression types. I'm willing to put in some effort to add them (I'm not familiar with CSIP, but it doesn't look too bad), but I wanted to first make sure that such a change would be welcome. Is there a reason that trig functions were excluded?

See SCIP.jl/#49.

In CSIPsetNonlinearObj, an auxiliary variable is added, to be used as the objective function in SCIP (which must be linear), with an inequality constraint that bounds this variable with the objective function.
If (feasible) initial values are given for all proper variables, none is given for this aux. var., and a value of 0 will most likely violate this inequality.

so what are the requirements? we want to delete variables and constraints? and then reuse those indices?

To give an initial solution, the user might want to give values for (a subset of) the variables.
These should be checked at the beginning of the solve.

I prefer to not change the signature of CSIPaddVar and instead provide a new function CSIPsetInitialValues, analogous to the bounds.

Also, I'm not sure whether SCIP already supports "partial" solutions, but I believe it's still WIP. In that case, I don't want to implement anything on the CSIP side now, but just try to pass the (0-filled?) values as a full solution candidate.

See also the discussion about heuristic callbacks in #3.