Problem with ZeroInflationIndex and fixings

[Ballabio Gerardo-4]

Hi all,
I found a bug, or better, a missing piece of implementation in class QuantLib::InflationIndex and/or ZeroInflationIndex (the latter is derived from the former; there's also YoYInflationIndex, it probably shares the same problem but I haven't looked at it yet).

I'm tracking the R01020x svn branch, so if this has already been fixed in trunk, please forgive me AND backport the patch.

As you know, inflation indices are published monthly, then inflation revaluation coefficients are defined for every day of the month, usually by interpolation. So, for each month, one should add the fixing for only one date (let's say the first day of the month) and then be able to retrieve the index value (interpolated or not) for all days.

Reading the code, I found that InflationIndex manages that by specializing the Index::addFixing virtual method: the specialized version takes the fixing from its argument and replicates it for all days in the month. Then ZeroInflationIndex specializes Index::fixing to add the interpolation logic.
 
The problem is that addFixing isn't the only way to add fixings. There are also two addFixings methods with different signatures (one retrieves fixings from a TimeSeries, the other from separate sequences of dates and values). As a matter of fact, those are the ONLY methods accessible from the QuantLibXL interface (via functions qlIndexAddFixings and qlIndexAddFixings2). So, basically, you can't manage fixings of inflation indices reasonably in QuantLibXL (you need replicate all fixings manually in your worksheet).

The obvious solution would be to declare the two addFixings methods as virtual and specialize them as well. But unfortunately, one of the two is a template method, and C++ doesn't allow those to be virtual (because templates must be instantiated at compile time, but virtual function calls are only resolved at run time, thus the compiler can't know the template arguments in advance). Browsing Google search results I found there's a technique called "type erasure" that's supposed to work around that, but I don't know whether that could be dropped into the QuantLib code and what consequences it might have (in particular, I suspect that it could conflict with QuantLibAddin's type resolution scheme).

However I believe there's another solution: to modify the ZeroInflationIndex::fixing method so that it only requires fixings on the first day of each month. Here's my proposed patch:

--- inflationindex.cpp  (revision 18491)
+++ inflationindex.cpp  (working copy)
@@ -80,20 +80,19 @@
     Rate ZeroInflationIndex::fixing(const Date& aFixingDate,
                                     bool /*forecastTodaysFixing*/) const {
         if (!needsForecast(aFixingDate)) {
+            std::pair<Date,Date> lim = inflationPeriod(aFixingDate, frequency_);
             const TimeSeries<Real>& ts = timeSeries();
-            Real pastFixing = ts[aFixingDate];
+            Real pastFixing = ts[lim.first];
             QL_REQUIRE(pastFixing != Null<Real>(),
-                       "Missing " << name() << " fixing for " << aFixingDate);
+                       "Missing " << name() << " fixing for " << lim.first);
             Real theFixing = pastFixing;
             if (interpolated_) {
-                // fixings stored flat & for every day
-                std::pair<Date,Date> lim =
-                    inflationPeriod(aFixingDate, frequency_);
+                // fixings stored on first day of every period
                 if (aFixingDate == lim.first) {
                     // we don't actually need the next fixing
                     theFixing = pastFixing;
                 } else {
-                    Date fixingDate2 = aFixingDate + Period(frequency_);
+                    Date fixingDate2 = lim.first + Period(frequency_);
                     Real pastFixing2 = ts[fixingDate2];
                     QL_REQUIRE(pastFixing2 != Null<Real>(),
                                "Missing " << name() << " fixing for " << fixingDate2);

There might be other methods that need a similar patching. I intend to look for them in the next days. But before doing that, I'd like to know whether you think that this is the right way to go.

Thank you
 Gerardo


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head></head>
<body>
<div style="font-family:Calibri;font-size:10px">
Banca Profilo S.p.A.
Corso Italia, 49 - 20122 Milano - Tel. [phone], Fax [phone]
Capitale Sociale Euro 136.794.106,00 i.v.
Iscrizione al Registro Imprese di Milano, C.F. e P.IVA 09108700155 - [hidden email]
Iscritta all’Albo delle Banche e dei Gruppi bancari
Aderente al Fondo Interbancario di Tutela dei depositi
Aderente al Conciliatore Bancario Finanziario e all’Arbitro Bancario Finanziario
Appartenente al Gruppo bancario Banca Profilo e soggetta all’attività di direzione e coordinamento di Arepo BP S.p.A.


DISCLAIMER:
The information transmitted may contain confidential and/or privileged material.
Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon,
this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender and delete the material from any computer.
</div>
</body>
</html>


------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and
their applications. This 200-page book is written by three acclaimed
leaders in the field. The early access version is available now.
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users

[Luigi Ballabio]

Hi,
    the patch looks good to me (especially since I was kind of bugged
by the need to store flat fixings).  Does the test suite still run
correctly? And while you're at it, may you check whether you can
replace lim.first + Period(frequency_); in the last patched line by
lim.second? It should be the same, and it looks somewhat more natural
to me.

Thanks,
    Luigi


On Mon, May 13, 2013 at 4:41 PM, Ballabio Gerardo
<[hidden email]> wrote:

> Hi all,
> I found a bug, or better, a missing piece of implementation in class QuantLib::InflationIndex and/or ZeroInflationIndex (the latter is derived from the former; there's also YoYInflationIndex, it probably shares the same problem but I haven't looked at it yet).
>
> I'm tracking the R01020x svn branch, so if this has already been fixed in trunk, please forgive me AND backport the patch.
>
> As you know, inflation indices are published monthly, then inflation revaluation coefficients are defined for every day of the month, usually by interpolation. So, for each month, one should add the fixing for only one date (let's say the first day of the month) and then be able to retrieve the index value (interpolated or not) for all days.
>
> Reading the code, I found that InflationIndex manages that by specializing the Index::addFixing virtual method: the specialized version takes the fixing from its argument and replicates it for all days in the month. Then ZeroInflationIndex specializes Index::fixing to add the interpolation logic.
>
> The problem is that addFixing isn't the only way to add fixings. There are also two addFixings methods with different signatures (one retrieves fixings from a TimeSeries, the other from separate sequences of dates and values). As a matter of fact, those are the ONLY methods accessible from the QuantLibXL interface (via functions qlIndexAddFixings and qlIndexAddFixings2). So, basically, you can't manage fixings of inflation indices reasonably in QuantLibXL (you need replicate all fixings manually in your worksheet).
>
> The obvious solution would be to declare the two addFixings methods as virtual and specialize them as well. But unfortunately, one of the two is a template method, and C++ doesn't allow those to be virtual (because templates must be instantiated at compile time, but virtual function calls are only resolved at run time, thus the compiler can't know the template arguments in advance). Browsing Google search results I found there's a technique called "type erasure" that's supposed to work around that, but I don't know whether that could be dropped into the QuantLib code and what consequences it might have (in particular, I suspect that it could conflict with QuantLibAddin's type resolution scheme).
>
> However I believe there's another solution: to modify the ZeroInflationIndex::fixing method so that it only requires fixings on the first day of each month. Here's my proposed patch:
>
> --- inflationindex.cpp  (revision 18491)
> +++ inflationindex.cpp  (working copy)
> @@ -80,20 +80,19 @@
>      Rate ZeroInflationIndex::fixing(const Date& aFixingDate,
>                                      bool /*forecastTodaysFixing*/) const {
>          if (!needsForecast(aFixingDate)) {
> +            std::pair<Date,Date> lim = inflationPeriod(aFixingDate, frequency_);
>              const TimeSeries<Real>& ts = timeSeries();
> -            Real pastFixing = ts[aFixingDate];
> +            Real pastFixing = ts[lim.first];
>              QL_REQUIRE(pastFixing != Null<Real>(),
> -                       "Missing " << name() << " fixing for " << aFixingDate);
> +                       "Missing " << name() << " fixing for " << lim.first);
>              Real theFixing = pastFixing;
>              if (interpolated_) {
> -                // fixings stored flat & for every day
> -                std::pair<Date,Date> lim =
> -                    inflationPeriod(aFixingDate, frequency_);
> +                // fixings stored on first day of every period
>                  if (aFixingDate == lim.first) {
>                      // we don't actually need the next fixing
>                      theFixing = pastFixing;
>                  } else {
> -                    Date fixingDate2 = aFixingDate + Period(frequency_);
> +                    Date fixingDate2 = lim.first + Period(frequency_);
>                      Real pastFixing2 = ts[fixingDate2];
>                      QL_REQUIRE(pastFixing2 != Null<Real>(),
>                                 "Missing " << name() << " fixing for " << fixingDate2);
>
> There might be other methods that need a similar patching. I intend to look for them in the next days. But before doing that, I'd like to know whether you think that this is the right way to go.
>
> Thank you
>  Gerardo
>
>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml">
> <head></head>
> <body>
> <div style="font-family:Calibri;font-size:10px">
> Banca Profilo S.p.A.
> Corso Italia, 49 - 20122 Milano - Tel. [phone], Fax [phone]
> Capitale Sociale Euro 136.794.106,00 i.v.
> Iscrizione al Registro Imprese di Milano, C.F. e P.IVA 09108700155 - [hidden email]
> Iscritta all’Albo delle Banche e dei Gruppi bancari
> Aderente al Fondo Interbancario di Tutela dei depositi
> Aderente al Conciliatore Bancario Finanziario e all’Arbitro Bancario Finanziario
> Appartenente al Gruppo bancario Banca Profilo e soggetta all’attività di direzione e coordinamento di Arepo BP S.p.A.
>
>
> DISCLAIMER:
> The information transmitted may contain confidential and/or privileged material.
> Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon,
> this information by persons or entities other than the intended recipient is prohibited.
> If you received this in error, please contact the sender and delete the material from any computer.
> </div>
> </body>
> </html>
>
>
> ------------------------------------------------------------------------------
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and
> their applications. This 200-page book is written by three acclaimed
> leaders in the field. The early access version is available now.
> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
> _______________________________________________
> QuantLib-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/quantlib-users

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users

[Ballabio Gerardo-4]

Da: Luigi Ballabio [mailto:[hidden email]]
> the patch looks good to me (especially since I was kind of bugged by the need to store flat fixings).  Does the test suite still run correctly?

Test suite? Which test suite?!? ...Ok, found ;-)

1>Tests completed in 18 m 3 s
1>Test suite "Master Test Suite" passed with:
1>  1744 assertions out of 1744 passed
1>  501 test cases out of 501 passed

> And while you're at it, may you check whether you can replace lim.first + Period(frequency_); in the last patched line by lim.second? It should be the same, and it looks somewhat more natural to me.

Done, with lim.second+1 (lim.second is the last day of the period). Updated patch follows:

===================================================================
--- inflationindex.cpp  (revision 18491)
+++ inflationindex.cpp  (working copy)
@@ -80,23 +80,21 @@
     Rate ZeroInflationIndex::fixing(const Date& aFixingDate,
                                     bool /*forecastTodaysFixing*/) const {
         if (!needsForecast(aFixingDate)) {
+            std::pair<Date,Date> lim = inflationPeriod(aFixingDate, frequency_);
             const TimeSeries<Real>& ts = timeSeries();
-            Real pastFixing = ts[aFixingDate];
+            Real pastFixing = ts[lim.first];
             QL_REQUIRE(pastFixing != Null<Real>(),
-                       "Missing " << name() << " fixing for " << aFixingDate);
+                       "Missing " << name() << " fixing for " << lim.first);
             Real theFixing = pastFixing;
             if (interpolated_) {
-                // fixings stored flat & for every day
-                std::pair<Date,Date> lim =
-                    inflationPeriod(aFixingDate, frequency_);
+                // fixings stored on first day of every period
                 if (aFixingDate == lim.first) {
                     // we don't actually need the next fixing
                     theFixing = pastFixing;
                 } else {
-                    Date fixingDate2 = aFixingDate + Period(frequency_);
-                    Real pastFixing2 = ts[fixingDate2];
+                    Real pastFixing2 = ts[lim.second+1];
                     QL_REQUIRE(pastFixing2 != Null<Real>(),
-                               "Missing " << name() << " fixing for " << fixingDate2);
+                               "Missing " << name() << " fixing for " << lim.second+1);
                     // now linearly interpolate
                     Real daysInPeriod = lim.second+1 - lim.first;
                     theFixing = pastFixing
===================================================================

On a related note, I was made aware of this issue (scroll down to the first message of the thread):

http://quantlib.10058.n7.nabble.com/Fwd-Inflation-Index-Interpolation-yoy-volatility-structure-td13888.html

Having checked the term sheet of an inflation-linked bond that I have at hand, I'd agree with Peter that Murex is doing the right thing and QuantLib isn't, i.e., the interpolation must be done over the month *after* the observation lag (in the example, November rather than August).

However, I suspect that there isn't a straightforward fix. Because as this is done now, for an observation date of 28 November, you must apply the lag and thus ask for the fixing on 28 August. But what if the observation date is 31 May? You can't have the fixing on 31 February.

Thus, in order to interpolate correctly, the lag must be applied *in* the method that retrieves the fixings. But because it would be quite confusing to add the fixing on 1 January and then find that there is no fixing on 1 January but there is one on 1 April, I'd rather not override the fixing() method at all and provide a new method laggedFixing() instead. Then PiecewiseZeroInflationCurve and PiecewiseYoYInflationCurve would need to be modified to call the latter method instead of the former.

What do you think?

Gerardo


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head></head>
<body>
<div style="font-family:Calibri;font-size:10px">
Banca Profilo S.p.A.
Corso Italia, 49 - 20122 Milano - Tel. [phone], Fax [phone]
Capitale Sociale Euro 136.794.106,00 i.v.
Iscrizione al Registro Imprese di Milano, C.F. e P.IVA 09108700155 - [hidden email]
Iscritta all’Albo delle Banche e dei Gruppi bancari
Aderente al Fondo Interbancario di Tutela dei depositi
Aderente al Conciliatore Bancario Finanziario e all’Arbitro Bancario Finanziario
Appartenente al Gruppo bancario Banca Profilo e soggetta all’attività di direzione e coordinamento di Arepo BP S.p.A.


DISCLAIMER:
The information transmitted may contain confidential and/or privileged material.
Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon,
this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender and delete the material from any computer.
</div>
</body>
</html>


------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users