Decoration of Handles in ObjectHandlerXL

[eric ehlers]

Hi All

Problems have arisen with the ObjectHandler feature of decorating
object handles e.g. "my_object" -> "my_object_~xxxxx" and in CVS I've
disabled this feature.  Garbage collection continues to function as
before but the user is now responsible for guaranteeing that his
handles are unique.

If this causes problems please let me know.  Please see below if you
require further detail.

Many Thanks,
Eric


The user supplies a stub "my_object" and gets back "my_object_~xxxxx"
where xxxxx is a unique key used to link back to the calling cell for
purposes of garbage collection.    xxxxx gets incremented every time
the object is reconstructed.

Suppose the user wants to pass that object to another function. If he
just references the cell containing the object e.g. A3 then all is
well.  But suppose a reference to the calling cell isn't convenient.
The user knows he called his object "my_object" and wants to use that
handle but can't because the object is now called "my_object_~xxxxx".

The example presented to me is where we might use one of several
different workbooks to create rate helpers e.g. "1W", "1M", ... We
want to bootstrap the yield curve without worrying about which book
was used to create the helpers so we want to refer to the rate helpers
by their raw handles not by referencing the cell in which the rate
helper object resides.

In CVS I've commented out the old behavior and replaced it with a
tentative implementation of the new approach.  The "~xxxxx" is
retained internally for garbage collection which continues to function
as before but the user doesn't see this and is now responsible for
ensuring that his handles are unique (as has always been the case when
GC is disabled).  Longer term I would make some further refinements to
the new code I just want to make sure this change won't cause problems
for anyone.



It's been pointed out to me that there's a problem with
ObjectHandler's feature of taking a user-supplied handle e.g.
"my_object" and decorating it to "my_object_~xxxxx" to force
uniqueness and faciliate garbage collection.

I propose to suppress this decoration,

Some further detail below.

Thanks,
Eric

[eric ehlers]

Hello,

Plamen pointed out to me that the change below breaks dependency
tracking where the user supplies the handle as a literal string
(rather than a cell reference).  So I rolled back that change,
apologies for the confusion.

I propose instead a solution which preserves existing behavior while
additionally allowing the user to refer to objects by the handle stub:

- user creates object and supplies stub "my_object"
- OH suffixes stub with unique key returning "my_object~_xxxxx" to calling cell
- user may later ask for either "my_object" or "my_object~_xxxxx" and
OH is intelligent enough to always retrieve "my_object~_xxxxx"

If the user refers to an object by its stub then dependency tracking
is not enforced and the user is responsible for ensuring that the
referenced object is refreshed before it is requested.

Also the handle stub is not guaranteed to be unique.  When the user
refers to an object by the stub then OH checks whether multiple
objects exist with the same identifier and if so an exception is
thrown.

Please get back to me with any thoughts or concerns.  I hope to
proceed shortly with the implementation.

Regards,
Eric

On 4/20/06, eric ehlers <[hidden email]> wrote:

> Hi All
>
> Problems have arisen with the ObjectHandler feature of decorating
> object handles e.g. "my_object" -> "my_object_~xxxxx" and in CVS I've
> disabled this feature.  Garbage collection continues to function as
> before but the user is now responsible for guaranteeing that his
> handles are unique.
>
> If this causes problems please let me know.  Please see below if you
> require further detail.
>
> Many Thanks,
> Eric
>
>
> The user supplies a stub "my_object" and gets back "my_object_~xxxxx"
> where xxxxx is a unique key used to link back to the calling cell for
> purposes of garbage collection.    xxxxx gets incremented every time
> the object is reconstructed.
>
> Suppose the user wants to pass that object to another function. If he
> just references the cell containing the object e.g. A3 then all is
> well.  But suppose a reference to the calling cell isn't convenient.
> The user knows he called his object "my_object" and wants to use that
> handle but can't because the object is now called "my_object_~xxxxx".
>
> The example presented to me is where we might use one of several
> different workbooks to create rate helpers e.g. "1W", "1M", ... We
> want to bootstrap the yield curve without worrying about which book
> was used to create the helpers so we want to refer to the rate helpers
> by their raw handles not by referencing the cell in which the rate
> helper object resides.
>
> In CVS I've commented out the old behavior and replaced it with a
> tentative implementation of the new approach.  The "~xxxxx" is
> retained internally for garbage collection which continues to function
> as before but the user doesn't see this and is now responsible for
> ensuring that his handles are unique (as has always been the case when
> GC is disabled).  Longer term I would make some further refinements to
> the new code I just want to make sure this change won't cause problems
> for anyone.

[Toyin Akin]

Hi Eric,

I don't like the idea of the user being able to use the stub key if a unique
key is generated for him/her and this new key provides no chances of
mispricing on the sheet (assuming that all input parameters are correct and
the user knows the instrument they are pricing).

As your library grows and many more objects are added, using the stub only
approach makes it difficult to track unique objects.

Why not stick with the "my_object~_xxxxx" key?

Also using the stub key only where the object may have already have been
created (a yieldcurve and a book object with the same name) would cause
errors on the spreadsheet that will be very difficult to track down.

The result will simply display #value, but for the user, how do you debug
this?

Visiting dependant cells may not help, because the previous object with the
same name may have been created within another pricing calculation
(worksheet/workbook same Excel session).

What are the main issues in not using the computed key?

Also, the "stub" key is static and Excel will not detect a change within
it's calculation dependency list. One will have to resort to a global (F9)
spreadsheet computation in order to ensure any new changes to the key is
picked up (that's assuming the previously created object is not part of this
updating too!!)

Just my 2 cents worth...

Toy out.

>From: "eric ehlers" <[hidden email]>
>To: "QuantLib developers" <[hidden email]>
>Subject: [Quantlib-dev] Re: Decoration of Handles in ObjectHandlerXL
>Date: Mon, 24 Apr 2006 08:54:40 +0200
>
>Hello,
>
>Plamen pointed out to me that the change below breaks dependency
>tracking where the user supplies the handle as a literal string
>(rather than a cell reference).  So I rolled back that change,
>apologies for the confusion.
>
>I propose instead a solution which preserves existing behavior while
>additionally allowing the user to refer to objects by the handle stub:
>
>- user creates object and supplies stub "my_object"
>- OH suffixes stub with unique key returning "my_object~_xxxxx" to calling
>cell
>- user may later ask for either "my_object" or "my_object~_xxxxx" and
>OH is intelligent enough to always retrieve "my_object~_xxxxx"
>
>If the user refers to an object by its stub then dependency tracking
>is not enforced and the user is responsible for ensuring that the
>referenced object is refreshed before it is requested.
>
>Also the handle stub is not guaranteed to be unique.  When the user
>refers to an object by the stub then OH checks whether multiple
>objects exist with the same identifier and if so an exception is
>thrown.
>
>Please get back to me with any thoughts or concerns.  I hope to
>proceed shortly with the implementation.
>
>Regards,
>Eric
>
>On 4/20/06, eric ehlers <[hidden email]> wrote:
> > Hi All
> >
> > Problems have arisen with the ObjectHandler feature of decorating
> > object handles e.g. "my_object" -> "my_object_~xxxxx" and in CVS I've
> > disabled this feature.  Garbage collection continues to function as
> > before but the user is now responsible for guaranteeing that his
> > handles are unique.
> >
> > If this causes problems please let me know.  Please see below if you
> > require further detail.
> >
> > Many Thanks,
> > Eric
> >
> >
> > The user supplies a stub "my_object" and gets back "my_object_~xxxxx"
> > where xxxxx is a unique key used to link back to the calling cell for
> > purposes of garbage collection.    xxxxx gets incremented every time
> > the object is reconstructed.
> >
> > Suppose the user wants to pass that object to another function. If he
> > just references the cell containing the object e.g. A3 then all is
> > well.  But suppose a reference to the calling cell isn't convenient.
> > The user knows he called his object "my_object" and wants to use that
> > handle but can't because the object is now called "my_object_~xxxxx".
> >
> > The example presented to me is where we might use one of several
> > different workbooks to create rate helpers e.g. "1W", "1M", ... We
> > want to bootstrap the yield curve without worrying about which book
> > was used to create the helpers so we want to refer to the rate helpers
> > by their raw handles not by referencing the cell in which the rate
> > helper object resides.
> >
> > In CVS I've commented out the old behavior and replaced it with a
> > tentative implementation of the new approach.  The "~xxxxx" is
> > retained internally for garbage collection which continues to function
> > as before but the user doesn't see this and is now responsible for
> > ensuring that his handles are unique (as has always been the case when
> > GC is disabled).  Longer term I would make some further refinements to
> > the new code I just want to make sure this change won't cause problems
> > for anyone.
>
>
>-------------------------------------------------------
>Using Tomcat but need to do more? Need to support web services, security?
>Get stuff done quickly with pre-integrated technology to make your job
>easier
>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
><a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid0709&bid&3057&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid0709&bid&3057&dat1642
>_______________________________________________
>Quantlib-dev mailing list
>[hidden email]
>https://lists.sourceforge.net/lists/listinfo/quantlib-dev

[eric ehlers]

Hi Toy,

[Toyin Akin]

Hi,

Your scenario of yieldCurve updating... Not bad.

I didn't realise that you had an Excel environment where the construction of
the yieldcurve takes place at such a granular level, thus allowing the
updating of the curve without the reconstruction of it. Most implementations
I have seen just recreates the curve and has one function that takes all
market inputs (ie - FinancialCad, MBRM etc...)

Okay I agree that you need the user key for this scenario. But I'd certainly
place the issues that you have mentioned regarding the risks of using just
the user key within your documentation
(in BOLD!!!). This would be for advanced users of Excel who know all about
Excel's calculation dependencies.

>Since Excel 2003, changes in A1 would still force a recalc of A2.
>When the inputs to A1 change, Excel is smart enough to realize that
>the dependents of A1 need to be recalculated, even though the physical
>contents of A1 are unchanged.
>
>That's with cell references.  The problem arises when you retrieve an
>object via a raw string "my_object".  In this case you have no
>guarantee that you're getting the latest version of "my_object".

I would put this in the doc also.

Toy out.

>From: "eric ehlers" <[hidden email]>
>To: "Toyin Akin" <[hidden email]>
>CC: [hidden email]
>Subject: Re: [Quantlib-dev] Re: Decoration of Handles in ObjectHandlerXL
>Date: Mon, 24 Apr 2006 13:08:37 +0200
>
>Hi Toy,
>
>From your message it's not clear to me whether you're responding to my
>original proposal (20 April) or the revised one (24 April).
>
>Let me reiterate that under the revised proposal, the existing
>behavior of OH would be retained.  The user would additionally have
>the option of retrieving objects via handle stubs.
>
>On 4/24/06, Toyin Akin <[hidden email]> wrote:
> > I don't like the idea of the user being able to use the stub key if a
>unique
> > key is generated for him/her and this new key provides no chances of
> > mispricing on the sheet (assuming that all input parameters are correct
>and
> > the user knows the instrument they are pricing).
>
>Under the revised proposal, OH supports retrieval of objects both by
>the handle stub ("my_object") and the full handle
>("my_object~_xxxxx").  Normally objects would be retrieved by their
>full handle but if required the user has the option of retrieving an
>object by its stub - in which case the user must respect the caveat
>that Excel no longer forces the object to be refreshed before it is
>referenced.
>
>If you feel that allowing this alternative behavior is dangerous then
>we could consider making the behavior configurable - you could build
>OH such that retrieval of objects via the handle stub is not supported
>(i.e. preserve the current behavior).
>
> > As your library grows and many more objects are added, using the stub
>only
> > approach makes it difficult to track unique objects.
>
>Under the proposed approach OH throws an exception if a user attempts
>to retrieve an object via its handle stub and two objects exists with
>the same stub.
>
> > Why not stick with the "my_object~_xxxxx" key?
>
>For normal use yes.
>
>We have a special case where we'd like to support retrieval of objects
>via the stub.  Consider three sheets which
>1) instantiate some rate helper objects (qlDepositRateHelper)
>2) update the rates in real time (qlSetQuote)
>3) bootstrap the yield curve (qlPiecewiseFlatForward)
>
>Now imagine that we have multiple versions of #2 depending on where we
>want to source the rates feed.  When we bootstrap the yield curve, we
>don't want to bother about which sheet is updating the rates.  Here's
>what we do:
>
>- Load sheet #1, refresh it, and close it.
>- The OH repository is now initialized with a collection of rate
>helper objects ("ON", "1W", "1M", etc.)
>- The sheet which created these objects (#1) is closed, so the objects
>will not be reconstructed again
>- Load sheets #2 and #3.  The curve is bootstrapped, the rates are
>updated in real time, and the state of the curve changes accordingly
>(thanks to QL's behind-the-scenes use of the Observer/Observable
>pattern)
>- Note that no further deconstruction/reconstruction of objects is
>taking place - we are merely changing the state of the rate helper and
>yield curve objects, which were constructed once
>
>If we have to use full handles, then sheet #3 needs to know which
>version of sheet #2 is in use.  This could change from one session to
>the next.  Maintaining this link is not impossible but we find it more
>convenient to allow objects to be referenced via handle stubs.  Sheets
>#2 and #3 simply trust that rate helper objects ("ON", "1W", "1M",
>etc.) exist without needing to know their full handles.  Therefore
>sheet #3 doesn't need to link to sheet #2 and we can switch to a
>different version of sheet #2 without editing sheet #3.
>
> > Also using the stub key only where the object may have already have been
> > created (a yieldcurve and a book object with the same name) would cause
> > errors on the spreadsheet that will be very difficult to track down.
> >
> > The result will simply display #value, but for the user, how do you
>debug
> > this?
> >
> > Visiting dependant cells may not help, because the previous object with
>the
> > same name may have been created within another pricing calculation
> > (worksheet/workbook same Excel session).
>
>Suppose the user tries to create two objects, and supplies the same
>stub, "my_object", to both constructors.  This will succeed, exactly
>as it does today - the first object will be created e.g. as
>"my_object~_00001" and the second as "my_object~_00002".If the user
>requests "my_object~_00001" or "my_object~_00002" he gets the correct
>object back.  If he requests "my_object", OH sees that two objects
>exist with that handle stub and throws an exception.
>
> > What are the main issues in not using the computed key?
>
>As discussed - normally we use the computed key.  But when we want to
>reference an object and don't want to keep track of where in the Excel
>session it resides - we reference it via the handle stub.
>
> > Also, the "stub" key is static and Excel will not detect a change within
> > it's calculation dependency list. One will have to resort to a global
>(F9)
> > spreadsheet computation in order to ensure any new changes to the key is
> > picked up (that's assuming the previously created object is not part of
>this
> > updating too!!)
>
>I was under the same impression, until Plamen clarified things for me.
>
>Consider two cells, A1 which constructs an object, and A2 which
>references the object in A1.
>
>Under the design proposed on 20 April, A1 would hold value "my_object".
>
>Since Excel 2003, changes in A1 would still force a recalc of A2.
>When the inputs to A1 change, Excel is smart enough to realize that
>the dependents of A1 need to be recalculated, even though the physical
>contents of A1 are unchanged.
>
>That's with cell references.  The problem arises when you retrieve an
>object via a raw string "my_object".  In this case you have no
>guarantee that you're getting the latest version of "my_object".
>
>Anyway under the revised design, the existing behavior of OH is
>preserved, and cell A1 contains "my_object~_xxxxx".  Normally A2 holds
>a reference to A1.  But in the special case described above we also
>have the option for A2 to refer to the raw string ""my_object".  In
>this case the user must ensure that a refresh of A1 is followed by a
>refresh of A2.  As mentioned in our environment we accomplish this by
>putting A1 on a separate sheet and closing the sheet after the object
>is instantiated.
>
> > Just my 2 cents worth...
>
>I very much welcome your feedback.
>
>Regards,
>Eric

[eric ehlers]

Hi All,

For the handle retrieval I committed a revised change which I hope
will suit everybody.  Existing behavior is the same, except:
- garbage collection is always on
- in addition to being able to retrieve an object via
"my_object_~xxxxx" you can now retrieve it via "my_object"
- as always, retrieving an object via a string rather than a reference
breaks dependency tracking, as per Toyin's request this caveat will
now be documented in *BOLD*

Any problems please let me know.

Regards,
Eric