Klaus,
If this works it will be game changing for us! I will test tomorrow and let you know.
I’m not sure I follow what you say about the memory leak. Are you saying that even with this new patch, I may get a leak ?
Which files in the ZIP file should I update ?
Thank you,
Simon Mazzucca
Senior Developer
1345 Avenue of the Americas, 3rd Floor
New York, NY 10105
( Main 646.840.4950
( Direct 646.840.4967
* [hidden email]
From: Klaus Spanderen-2 [via QuantLib] [mailto:ml-node+[hidden email]]
Sent: Sunday, June 15, 2014 10:17 AM
To: Simon Mazzucca
Subject: Re: Fwd: pure virtual function call
Hi Simon,
I think I've made some progress on this topic. In order to be able to debug it I've implemented the InflationCPIBondTest::testCleanPrice unit test in Scala... and this program shows the same behavior as you've outlined:
1. pure virtual function call with an unpatched QuantLib
2. spurious dead locks when using the patch
Indeed due to the cyclic reference structure (Inflation Index referring the CPI term-structure and via versa) this problem exposes a possible dead lock situation within the patched observer pattern. After this has been fixed the CPI bond test case is now running smoothly for hours w/o any "pure virtual function calls" or dead locks.
The latest version of the patch is available here
http://hpc-quantlib.de/src/observer.zip
I'd would be nice if you can test it. BTW, due to the cyclic references in the QL inflation code it is quite easy to create a memory leak even with Scala. I had to manually break the cyclic reference, s. line 141 in the CPIBond example
https://github.com/lballabio/quantlib/pull/106
best regards
Klaus
On Friday, June 06, 2014 08:47:47 AM smazzucca wrote:
> Klaus,
>
> It will be very complicated to write up a sample to reproduce the error... I
> will if it comes to it, but can you see if this looks right in the meantime
> ? Or maybe it can be simplified in such a way that the critical code is
> used less ?
>
> This is how we create CpiBond objects. This is done thousands of times:
>
> private CPIBond CreateIndexLinkedBond()
> {
> ZeroIndexTermCurve.LoadCurveValues(InstrumentCalendar,
> DayCounter, ScenarioDate, false);
> ZeroIndexTermCurve.AdjustCurve(_CurrentMaturityDate);
>
> Period zeroCouponSwapObservationPeriod = new
> Period(ZERO_COUPON_SWAP_OBSERVATION_LAG-1, TimeUnit.Months); Date
> observationLagDate =
> ScenarioDate.Subtract(zeroCouponSwapObservationPeriod);
>
> DateGeneration.Rule backwardDateGeneration =
> DateGeneration.Rule.Backward;
>
> bool interpolated = false;
> ZeroInflationTermStructureHandle cpiTS = new
> ZeroInflationTermStructureHandle(); ZeroInflationIndex inflationIndex = new
> USCPI(interpolated, cpiTS);
>
> inflationIndex.addFixing(observationLagDate,
> CurrentReferenceIndexlevel);
>
> //Zero coupon inflation index swap
> Datum[] zeroCouponInflationIndexSwapData =
> ZeroIndexTermCurve.ToDatumArray(); ZeroHelperVector zeroCouponSwapHelpers =
> ZeroIndexTermCurve.ToZeroHelperVector( inflationIndex,
> zeroCouponSwapObservationPeriod,
> InstrumentCalendar,
> DayCounter);
>
> double baseZeroRate = zeroCouponInflationIndexSwapData[0].rate /
> 100; BackwardFlat backwardFlat = new BackwardFlat();
> double accuracy = 1.0e-12;
>
> ZeroInflationTermStructure piecewiseZeroInflation = new
> PiecewiseZeroInflation( ScenarioDate,
> InstrumentCalendar,
> DayCounter,
> zeroCouponSwapObservationPeriod,
> QuantLib.Frequency.Monthly,
> interpolated,
> baseZeroRate,
> _DiscountCurve,
> zeroCouponSwapHelpers,
> accuracy,
> backwardFlat);
>
> cpiTS = new
> ZeroInflationTermStructureHandle(piecewiseZeroInflation); inflationIndex =
> new USCPI(interpolated, cpiTS);
>
> List<double> fixedRatesList = new List<double>() { 1, CouponRate
> / 100 }; DoubleVector fixedRates = new DoubleVector(fixedRatesList);
>
> //Always use actual365fixed ISDA convention for this
> DayCounter fixedDayCount = new Actual365Fixed();
> CPI.InterpolationType observationInterpolation =
> CPI.InterpolationType.Linear; Period couponFrequenceyPeriod = new
> Period(Frequency);
>
> Schedule fixedSchedule = new Schedule(
> IssueDate,
> MaturityDate,
> couponFrequenceyPeriod,
> InstrumentCalendar,
> _BusinessDayConvention,
> _BusinessDayConvention,
> backwardDateGeneration,
> _EndOfMonth);
>
> CPIBond bond = new CPIBond(
> (uint)SettlementDays,
> NotionalValue,
> GrowthOnly,
> BaseIndex,
> ObservationPeriod,
> inflationIndex,
> observationInterpolation,
> fixedSchedule,
> fixedRates,
> fixedDayCount,
> _BusinessDayConvention,
> IssueDate
> );
>
> return bond;
> }
>
>
> First we create the CpiBond with CreateIndexLinkedBond() then load the
> pricing engine:
>
> private void LoadPriceEngine()
> {
> PricingEngine engine = new
> DiscountingBondEngine(_DiscountCurve);
>
> _Bond.setPricingEngine(engine);
>
> if (_CleanPrice == 0)
> _CleanPrice = _Bond.cleanPrice();
> }
>
>
> Methods called from CreateIndexLinkedBond():
>
> public void LoadCurveValues(Calendar calendar, DayCounter
> dayCounter, Date scenarioDate, bool inludeFirstCurvePoint) {
> List<CurveValue> curveValues = GetAsCurveValues();
>
> _DayCounter = dayCounter;
> _Calendar = calendar;
> //Curves must be initialized with reference date
> double firstValue = (curveValues.Count > 0) ?
> curveValues[0].Value : 0.0; CurveDates = new List<Date>() { scenarioDate };
> CurveValues = new List<double>() { firstValue };
> int startIndex = inludeFirstCurvePoint ? 0 : 1;
>
> for (int i = startIndex; i < curveValues.Count(); i++)
> {
> Period months = new Period(curveValues[i].Months,
> TimeUnit.Months); Date itemDate =
> _Calendar.adjust(scenarioDate.Add(months)); CurveDates.Add(itemDate);
> CurveValues.Add(curveValues[i].Value);
> }
> }
>
> /// <summary>
> /// With good data, this won't be an issue, but defensively we must
> /// ensure that the Maturity Date is included in the curve
> /// </summary>
> public void AdjustCurve(Date maturityDate)
> {
> if (CurveDates.Count > 0)
> {
> Date lastDate = CurveDates[CurveDates.Count - 1];
> double lastValue = CurveValues[CurveValues.Count - 1];
>
> if (maturityDate.ToDateTime() > lastDate.ToDateTime())
> {
> CurveDates.Add(maturityDate);
> CurveValues.Add(lastValue);
> }
> }
> }
>
> Thank you,
> Simon
>
> From: Klaus Spanderen-2 [via QuantLib]
> [mailto:[hidden email]] Sent: Thursday, June 05,
> 2014 5:03 PM
> To: Simon Mazzucca
> Subject: Re: Fwd: pure virtual function call
>
>
> Hi Simon,
>
>
>
> thanks for the error report. Case 3 shows that it is actually the observer
> pattern, which is creating the problems for you. (I bet, if you comment out
> the update() methd in case 1 it will also work).
>
>
>
> Unclear now is why the patch does not fix it. Based on your description I
> guess in case 2 your application is facing a dead lock. If I had an exmple
> program then I'd remove boost::mutex::scoped_lock in observable.cpp or I'd
> replace them by boost::recursive_mutex::scoped_lock (and also replace
> boost::mutex by boost::recursive_mutex)
>
>
>
> thanks and regards
>
> Klaus
>
> On Thursday, June 05, 2014 08:02:33 AM smazzucca wrote:
> > OK, I think I have something we can work with.
> >
> >
> >
> > I have 3 sets of files:
> >
> > 1) SWIG wrapper for QL 1.3
> >
> > 2) SWIG wrapper for QL 1.4 (patched)
> >
> > 3) SWIG wrapper for QL 1.4 (patched, but with update() method commented
> > out)
> >
> >
> >
> > This is what I get with each:
> >
> >
> >
> > 1) This is my control scenario where I get the "Pure Virtual Function
> > Call"
> >
> > error
> >
> >
> >
> > 2) Contrary to what I said yesterday, I don't actually get the PVFC error,
> >
> > but it's something similar. While the PVFC issue takes down the whole
> >
> > process, this /freezes/ the process. The good news is that in VS I can
> >
> > actually see a bit of what's going on.
> >
> >
> >
> > The "GC Finalizer Thread" is trying to dispose a CPIBond object:
> >
> >
> >
> > ~CPIBond()
> >
> > {
> >
> > Dispose();
> >
> > }
> >
> >
> >
> > Which in turn calls:
> >
> >
> >
> > public override void Dispose()
> >
> > {
> >
> > lock (this)
> >
> > {
> >
> > if (swigCPtr.Handle != IntPtr.Zero)
> >
> > {
> >
> > if (swigCMemOwn)
> >
> > {
> >
> > swigCMemOwn = false;
> >
> > *NQuantLibcPINVOKE.delete_CPIBond(swigCPtr);*
> >
> > }
> >
> > swigCPtr = new HandleRef(null, IntPtr.Zero);
> >
> > }
> >
> > GC.SuppressFinalize(this);
> >
> > base.Dispose();
> >
> > }
> >
> > }
> >
> >
> >
> > The above (from the C# wrapper) gets stuck on the bold line while calling
> >
> > the method below (in the C++ wrapper):
> >
> >
> >
> > SWIGEXPORT void SWIGSTDCALL CSharp_delete_CPIBond(void * jarg1) {
> >
> > CPIBondPtr *arg1 = (CPIBondPtr *) 0 ;
> >
> >
> >
> > arg1 = (CPIBondPtr *)jarg1;
> >
> > {
> >
> > try {
> >
> > delete arg1;
> >
> > } catch (std::out_of_range& e) {
> >
> > {
> >
> > SWIG_CSharpException(SWIG_IndexError, const_cast<char*>(e.what()));
> >
> > return ;
> >
> > };
> >
> > } catch (std::exception& e) {
> >
> > {
> >
> > SWIG_CSharpException(SWIG_RuntimeError,
> >
> > const_cast<char*>(e.what())); return ;
> >
> > };
> >
> > } catch (...) {
> >
> > {
> >
> > SWIG_CSharpException(SWIG_UnknownError, "unknown error"); return ;
> >
> > };
> >
> > }
> >
> > }
> >
> > }
> >
> >
> >
> > 3) Lastly, when I run the version with commented out update(), it all
> >
> > appears to work. I have not checked the numbers themselves, but at least I
> >
> > don't get any errors. I think the only problem is that it appears a bit
> >
> > slow, but I have not actually timed it.
> >
> >
> >
> > As far as converting your test app to C#, I am not sure we should do that
> >
> > yet. The issue might have something to do with how we use QL. I will see
> > if
> >
> > I can isolate my issue in a small app I can send you.
> >
> >
> >
> > Hopefully the above will help you understand what's going on.
> >
> >
> >
> > Thank you,
> >
> > Simon
> >
> >
> >
> >
> >
> >
> >
> > --
> >
> > View this message in context:
> >
> > http://quantlib.10058.n7.nabble.com/pure-virtual-function-call-tp15342p153
> > 9
> >
> > 4.html Sent from the quantlib-users mailing list archive at Nabble.com.
> >
> >
> >
> > --------------------------------------------------------------------------
> > --
> >
> > -- Learn Graph Databases - Download FREE O'Reilly Book
> >
> > "Graph Databases" is the definitive new guide to graph databases and their
> >
> > applications. Written by three acclaimed leaders in the field,
> >
> > this first edition is now available. Download your free book today!
> >
> > http://p.sf.net/sfu/NeoTech
> >
> > _______________________________________________
> >
> > QuantLib-users mailing list
> >
> > [hidden email]</user/SendEmail.jtp?type=node&node=15399&i=0>
> >
> > https://lists.sourceforge.net/lists/listinfo/quantlib-users
>
> ----------------------------------------------------------------------------
> -- Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and their
> applications. Written by three acclaimed leaders in the field,
> this first edition is now available. Download your free book today!
> http://p.sf.net/sfu/NeoTech
> _______________________________________________
> QuantLib-users mailing list
> [hidden email]</user/SendEmail.jtp?type=node&node=15399&i=1>
> https://lists.sourceforge.net/lists/listinfo/quantlib-users
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://quantlib.10058.n7.nabble.com/pure-virtual-function-call-tp15342p1539
> 9.html To unsubscribe from pure virtual function call, click
> here<http://quantlib.10058.n7.nabble.com/template/NamlServlet.jtp?macro=uns
> ubscribe_by_code&node=15342&code=c21henp1Y2NhQGNvbmNlcHRvbmVsbGMuY29tfDE1MzQ
> yfC0xNjkyNzM2OTc1>.
> NAML<http://quantlib.10058.n7.nabble.com/template/NamlServlet.jtp?macro=mac
> ro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.temp
> late.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-inst
> ant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
>
> This e-mail, including its contents and attachments, if any, is confidential
> and is intended only for the addressee. If you are not the intended
> recipient, you are hereby notified that any use, dissemination or
> distribution of this communication is strictly forbidden. If you received
> this e-mail in error please immediately notify the sender and delete or
> destroy this and all copies of this message and all attachments. Any
> unauthorized disclosure, use, distribution, or reproduction of this message
> or any attachments is prohibited and may be unlawful.
>
>
>
>
> --
> View this message in context:
> http://quantlib.10058.n7.nabble.com/pure-virtual-function-call-tp15342p1541
> 3.html Sent from the quantlib-users mailing list archive at Nabble.com.
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users
Locked
