-
Notifications
You must be signed in to change notification settings - Fork 138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CVode returns wrong yout if h becomes zero #82
Labels
Comments
dweindl
added a commit
to dweindl/AMICI
that referenced
this issue
Feb 17, 2025
Due to LLNL/sundials#82, we can get exceptions from `CVodeSolver::getSens`, escaping `FinalStateStorer::~FinalStateStorer`, that will result in crashes. This is fixed here. The issue is semi-producible with [Boehm_JProteomeRes2014](https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/tree/15b4f78aff1121870537ad3e1a6b902cb8f73930/Benchmark-Models/Boehm_JProteomeRes2014): Progamm output: ``` 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVode:WARNING] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3482: Internal t = 0 and h = 0 are such that t + h = t on the next step. The solver will continue anyway. (99) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVodeGetDky:BAD_T] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3783: Illegal value for t.t = 2.5 is not between tcur - hu = 0 and tcur = 0. (-25) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVodeGetDky:BAD_T] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3783: Illegal value for t.t = 5 is not between tcur - hu = 0 and tcur = 0. (-25) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVode:ILL_INPUT] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3328: Trouble interpolating at tout = 5. tout too far back in direction of integration (-22) 2025-02-17 14:44:33.988 - amici.swig_wrappers - ERROR - [model1_data1][FORWARD_FAILURE] AMICI forward simulation failed at t = 5: AMICI failed to integrate the forward problem terminate called after throwing an instance of 'amici::CvodeException' what(): CVODE routine CVodeGetSens failed with error code -25. Thread 1 "python" received signal SIGABRT, Aborted. ``` Stacktrace: ``` AMICI-dev#10 0x00007ffff70811f1 in _Unwind_RaiseException (exc=0x6d5ff10) at ../../../src/libgcc/unwind.inc:136 AMICI-dev#11 0x00007ffff48bb384 in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x7fff5c6d6aa8 <typeinfo for amici::CvodeException>, dest=0x7fff5c5476c4 <amici::CvodeException::~CvodeException()>) at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:93 AMICI-dev#12 0x00007fff5c5bdb64 in amici::CVodeSolver::getSens (this=0x6d130d0) at python/sdist/amici/src/solver_cvodes.cpp:663 AMICI-dev#13 0x00007fffbc5709e8 in amici::Solver::getStateSensitivity (this=0x6d130d0, t=2.5) at python/sdist/amici/src/solver.cpp:1340 AMICI-dev#14 0x00007fffbc570750 in amici::Solver::writeSolution (this=0x6d130d0, t=0x6d26db0, x=..., dx=..., sx=..., xQ=...) at python/sdist/amici/src/solver.cpp:1303 AMICI-dev#15 0x00007fffbc5d376d in amici::ForwardProblem::getSimulationState (this=0x6d26c90) at python/sdist/amici/src/forwardproblem.cpp:421 AMICI-dev#16 0x00007fffbc5d45df in amici::FinalStateStorer::~FinalStateStorer (this=0x7fffffffb308, __in_chrg=<optimized out>) at include/amici/forwardproblem.h:450 AMICI-dev#17 0x00007fffbc5d2285 in amici::ForwardProblem::workForwardProblem (this=0x6d26c90) at python/sdist/amici/src/forwardproblem.cpp:203 AMICI-dev#18 0x00007fffbc54fbf9 in amici::runAmiciSimulation (solver=..., edata=0x5078010, model=..., rethrow=false) at python/sdist/amici/src/amici.cpp:108 ```
github-merge-queue bot
pushed a commit
to AMICI-dev/AMICI
that referenced
this issue
Feb 17, 2025
Due to LLNL/sundials#82, we can get exceptions from `CVodeSolver::getSens`, escaping `FinalStateStorer::~FinalStateStorer`, that will result in crashes. This is fixed here. The issue is semi-producible with [Boehm_JProteomeRes2014](https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/tree/15b4f78aff1121870537ad3e1a6b902cb8f73930/Benchmark-Models/Boehm_JProteomeRes2014): Progamm output: ``` 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVode:WARNING] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3482: Internal t = 0 and h = 0 are such that t + h = t on the next step. The solver will continue anyway. (99) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVodeGetDky:BAD_T] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3783: Illegal value for t.t = 2.5 is not between tcur - hu = 0 and tcur = 0. (-25) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVodeGetDky:BAD_T] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3783: Illegal value for t.t = 5 is not between tcur - hu = 0 and tcur = 0. (-25) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVode:ILL_INPUT] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3328: Trouble interpolating at tout = 5. tout too far back in direction of integration (-22) 2025-02-17 14:44:33.988 - amici.swig_wrappers - ERROR - [model1_data1][FORWARD_FAILURE] AMICI forward simulation failed at t = 5: AMICI failed to integrate the forward problem terminate called after throwing an instance of 'amici::CvodeException' what(): CVODE routine CVodeGetSens failed with error code -25. Thread 1 "python" received signal SIGABRT, Aborted. ``` Stacktrace: ``` #10 0x00007ffff70811f1 in _Unwind_RaiseException (exc=0x6d5ff10) at ../../../src/libgcc/unwind.inc:136 #11 0x00007ffff48bb384 in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x7fff5c6d6aa8 <typeinfo for amici::CvodeException>, dest=0x7fff5c5476c4 <amici::CvodeException::~CvodeException()>) at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:93 #12 0x00007fff5c5bdb64 in amici::CVodeSolver::getSens (this=0x6d130d0) at python/sdist/amici/src/solver_cvodes.cpp:663 #13 0x00007fffbc5709e8 in amici::Solver::getStateSensitivity (this=0x6d130d0, t=2.5) at python/sdist/amici/src/solver.cpp:1340 #14 0x00007fffbc570750 in amici::Solver::writeSolution (this=0x6d130d0, t=0x6d26db0, x=..., dx=..., sx=..., xQ=...) at python/sdist/amici/src/solver.cpp:1303 #15 0x00007fffbc5d376d in amici::ForwardProblem::getSimulationState (this=0x6d26c90) at python/sdist/amici/src/forwardproblem.cpp:421 #16 0x00007fffbc5d45df in amici::FinalStateStorer::~FinalStateStorer (this=0x7fffffffb308, __in_chrg=<optimized out>) at include/amici/forwardproblem.h:450 #17 0x00007fffbc5d2285 in amici::ForwardProblem::workForwardProblem (this=0x6d26c90) at python/sdist/amici/src/forwardproblem.cpp:203 #18 0x00007fffbc54fbf9 in amici::runAmiciSimulation (solver=..., edata=0x5078010, model=..., rethrow=false) at python/sdist/amici/src/amici.cpp:108 ```
dweindl
added a commit
to AMICI-dev/AMICI
that referenced
this issue
Feb 18, 2025
Due to LLNL/sundials#82, we can get exceptions from `CVodeSolver::getSens`, escaping `FinalStateStorer::~FinalStateStorer`, that will result in crashes. This is fixed here. The issue is semi-producible with [Boehm_JProteomeRes2014](https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/tree/15b4f78aff1121870537ad3e1a6b902cb8f73930/Benchmark-Models/Boehm_JProteomeRes2014) with forward sensitivities: Program output: ``` 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVode:WARNING] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3482: Internal t = 0 and h = 0 are such that t + h = t on the next step. The solver will continue anyway. (99) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVodeGetDky:BAD_T] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3783: Illegal value for t.t = 2.5 is not between tcur - hu = 0 and tcur = 0. (-25) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVodeGetDky:BAD_T] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3783: Illegal value for t.t = 5 is not between tcur - hu = 0 and tcur = 0. (-25) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVode:ILL_INPUT] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3328: Trouble interpolating at tout = 5. tout too far back in direction of integration (-22) 2025-02-17 14:44:33.988 - amici.swig_wrappers - ERROR - [model1_data1][FORWARD_FAILURE] AMICI forward simulation failed at t = 5: AMICI failed to integrate the forward problem terminate called after throwing an instance of 'amici::CvodeException' what(): CVODE routine CVodeGetSens failed with error code -25. Thread 1 "python" received signal SIGABRT, Aborted. ``` Stack trace: ``` #10 0x00007ffff70811f1 in _Unwind_RaiseException (exc=0x6d5ff10) at ../../../src/libgcc/unwind.inc:136 #11 0x00007ffff48bb384 in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x7fff5c6d6aa8 <typeinfo for amici::CvodeException>, dest=0x7fff5c5476c4 <amici::CvodeException::~CvodeException()>) at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:93 #12 0x00007fff5c5bdb64 in amici::CVodeSolver::getSens (this=0x6d130d0) at python/sdist/amici/src/solver_cvodes.cpp:663 #13 0x00007fffbc5709e8 in amici::Solver::getStateSensitivity (this=0x6d130d0, t=2.5) at python/sdist/amici/src/solver.cpp:1340 #14 0x00007fffbc570750 in amici::Solver::writeSolution (this=0x6d130d0, t=0x6d26db0, x=..., dx=..., sx=..., xQ=...) at python/sdist/amici/src/solver.cpp:1303 #15 0x00007fffbc5d376d in amici::ForwardProblem::getSimulationState (this=0x6d26c90) at python/sdist/amici/src/forwardproblem.cpp:421 #16 0x00007fffbc5d45df in amici::FinalStateStorer::~FinalStateStorer (this=0x7fffffffb308, __in_chrg=<optimized out>) at include/amici/forwardproblem.h:450 #17 0x00007fffbc5d2285 in amici::ForwardProblem::workForwardProblem (this=0x6d26c90) at python/sdist/amici/src/forwardproblem.cpp:203 #18 0x00007fffbc54fbf9 in amici::runAmiciSimulation (solver=..., edata=0x5078010, model=..., rethrow=false) at python/sdist/amici/src/amici.cpp:108 ```
dweindl
added a commit
to dweindl/AMICI
that referenced
this issue
Feb 18, 2025
…dev#2647) Due to LLNL/sundials#82, we can get exceptions from `CVodeSolver::getSens`, escaping `FinalStateStorer::~FinalStateStorer`, that will result in crashes. This is fixed here. The issue is semi-producible with [Boehm_JProteomeRes2014](https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab/tree/15b4f78aff1121870537ad3e1a6b902cb8f73930/Benchmark-Models/Boehm_JProteomeRes2014) with forward sensitivities: Program output: ``` 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVode:WARNING] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3482: Internal t = 0 and h = 0 are such that t + h = t on the next step. The solver will continue anyway. (99) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVodeGetDky:BAD_T] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3783: Illegal value for t.t = 2.5 is not between tcur - hu = 0 and tcur = 0. (-25) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVodeGetDky:BAD_T] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3783: Illegal value for t.t = 5 is not between tcur - hu = 0 and tcur = 0. (-25) 2025-02-17 14:44:33.988 - amici.swig_wrappers - DEBUG - [model1_data1][cvodes:CVode:ILL_INPUT] python/sdist/amici/ThirdParty/sundials/src/cvodes/cvodes.c:3328: Trouble interpolating at tout = 5. tout too far back in direction of integration (-22) 2025-02-17 14:44:33.988 - amici.swig_wrappers - ERROR - [model1_data1][FORWARD_FAILURE] AMICI forward simulation failed at t = 5: AMICI failed to integrate the forward problem terminate called after throwing an instance of 'amici::CvodeException' what(): CVODE routine CVodeGetSens failed with error code -25. Thread 1 "python" received signal SIGABRT, Aborted. ``` Stack trace: ``` AMICI-dev#10 0x00007ffff70811f1 in _Unwind_RaiseException (exc=0x6d5ff10) at ../../../src/libgcc/unwind.inc:136 AMICI-dev#11 0x00007ffff48bb384 in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x7fff5c6d6aa8 <typeinfo for amici::CvodeException>, dest=0x7fff5c5476c4 <amici::CvodeException::~CvodeException()>) at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:93 AMICI-dev#12 0x00007fff5c5bdb64 in amici::CVodeSolver::getSens (this=0x6d130d0) at python/sdist/amici/src/solver_cvodes.cpp:663 AMICI-dev#13 0x00007fffbc5709e8 in amici::Solver::getStateSensitivity (this=0x6d130d0, t=2.5) at python/sdist/amici/src/solver.cpp:1340 AMICI-dev#14 0x00007fffbc570750 in amici::Solver::writeSolution (this=0x6d130d0, t=0x6d26db0, x=..., dx=..., sx=..., xQ=...) at python/sdist/amici/src/solver.cpp:1303 AMICI-dev#15 0x00007fffbc5d376d in amici::ForwardProblem::getSimulationState (this=0x6d26c90) at python/sdist/amici/src/forwardproblem.cpp:421 AMICI-dev#16 0x00007fffbc5d45df in amici::FinalStateStorer::~FinalStateStorer (this=0x7fffffffb308, __in_chrg=<optimized out>) at include/amici/forwardproblem.h:450 AMICI-dev#17 0x00007fffbc5d2285 in amici::ForwardProblem::workForwardProblem (this=0x6d26c90) at python/sdist/amici/src/forwardproblem.cpp:203 AMICI-dev#18 0x00007fffbc54fbf9 in amici::runAmiciSimulation (solver=..., edata=0x5078010, model=..., rethrow=false) at python/sdist/amici/src/amici.cpp:108 ```
This issue still persists at least with SUNDIALS 7.1.1. I can reproduce it, but unfortunately, I cannot provide a self-contained example. I'm happy to provide more information if needed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, I stumbled across a problem in
CVode
which seems to yield a wrongyout
ifh
becomes zero.After
CVode
:Internal t = 0 and h = 0 are such that t + h = t on the next step. The solver will continue anyway.
andCVodeGetDky
:Illegal value for t.t = 900 is not between tcur - hu = 0 and tcur = 0
,CVode
still returnsCV_SUCCESS
, butyout
is not the correct value fortout
, but for the time whenh
became zero.The problem seems to be the multiplication with
cv_mem->cv_h
for checking whethertout
was reached and subsequently ignoring the return value ofCVodeGetDky
(CV_BAD_T
), here:sundials/src/cvodes/cvodes.c
Lines 3298 to 3305 in a9bc74f
Similar checks exist in
CVodeF
here:sundials/src/cvodes/cvodea.c
Lines 560 to 561 in a9bc74f
and here:
sundials/src/cvodes/cvodea.c
Lines 472 to 473 in a9bc74f
CVodeF
leads to the same warnings, but returnsCV_BAD_T
. That is at least notCV_SUCCESS
, but it is somewhat misleading, asCVodeGetDky
should not have been called in the first place.This happens in a more complex application and I am not sure I can easily provide a minimal reproducible example.
The text was updated successfully, but these errors were encountered: