ObjectHandler and QuantLibAddin II

[Penschke, Walter]

Hi Eric,

here my thougts on your last posting:

- I am still not clear about interface.hpp/interface.cpp:
I can see that it is a procedural wrapper for ObjectHandler and understand
that it is required
for environments like Excel. What is a bit striking to me is that it is
tightly bound to ObjectWidget
which is one specific sub-class of Object. Am I right that for any other
subclass of Object - and if
I understood your idea correctly, there will be lots - again such a
procedural wrapper is required?

If so, could it be an alternative to incorporate the actual class we are
dealing with "somehow" in
the signature of interface? Having had a look at the current code of
interface, I was thinking if one
could for example add the name of the class and then use a factory to create
an instance of the
class in the (now general) MAKE method of interface. Wrt to the UPDATE
method I am not sure
if we need to know about the concrete derived class at all. Could it work to
use straight forward
polymorphism here? May be I missed something, but if that would work we
would be fine with
one single interface class. Haven't yet found time to actually try it.

- Here is what I have done so far with SOAP and what design issues I
encountered:
I am using gSoap (http://gsoap2.sourceforge.net) and it seems to be ok. It
seems to be pretty fast
and works on both platforms which are relevant to me (Windows and Linux).
STL types are not
fully supported. It does work with lists but not with maps for example (at
least the version I am using,
which is only 2-3 month old.) The available documentation is pretty good and
big.

Up to now I did get working a small sample pricing server and a small Excel
based market data retrieval server.
I wrote an own little ExcelEngine for accessing Spreadsheets, which hides
all the nasty COM stuff away.

I am not too much into gSoap and therefore was using a procedural interface
approach (similar to your idea).
I.e. the communication between client and server is based on a set of
functions. Haven't yet checked if an
object based approach is supported and works.

My approach so far was to leave the client as dumb as possible. Within Excel
for example I don't need to
have QuantLib available. For my little sample stuff this is ok as I only did
have to provide a small VB-routine
which accesses the server via a thin client library.

At this stage I was thinking how to sensibly extend the service prototypes.
It seems to me that the principal
problem is the same as you are addressing with ObjectHandler/QuantLibAddin:
There is the need to wrap up
QuantLib functionality as a whole, via a set of functions.

Problem is that the number of thes functions can become pretty big. As I am
reluctant to wrap up all QL functionality
I was first thinking about reducing the granularity on the client side. The
question is: To what extend? One idea I have
in mind is to provide one big "master-method", which is fed with a variable
list of arguments. I this case I would need
a corresponding Dispatcher on the client side. Also, the load on the server
would be pretty high as it not only performs
pricing but also needs to create the objects from scratch for each request.

At this stage I came across your ObjectHandler/QuantLibAddin packages. The
idea of a local object store is nice as
it would reduce the load on the server. I also don't see a big problem with
having QL available on the client side as well.
Clearly you are going the other way of keeping the granularity on the client
side in sync with QL, which is natural when
QL also resides on the client.

The remaining problem of wrapping up the complete QL functionality is
however remaining.
For the SOAP-calls your idea with (XML-)serialisation seems to be the right
way. In contrast to Java this is, however,
not a neglectable task within C++ because in C++ no proper reflection
mechanism is available (Class.forName(...)).
Perhaps one could supply a big serialiser/deserialiser class. But this idea
also seems to be a bit clunky.

Alternatively there might be other SOAP implementations available, which
support automatic stub/skeleton-generation
from a "normal" C++ class. Perhaps one could create the required
communication code automatically???

Sorry that I have more questions than answers. I am also pretty much at the
beginning.
Please, let me know what you think about it.


wpe

[eric ehlers]

> Having had a look at the current code of interface, I was thinking if one
> could for example add the name of the class and then use a factory to create
> an instance of the class in the (now general) MAKE method of interface.

Yes, this is the right approach.  I've made the change in
ObjectHandler - the interface layer is now collapsed into a single
factory function.  It needs some refinement but it's definitely the
way forward.  I'm in the process of making the corresponding change in
QuantLibAddin (I already completed a quick test to confirm it will
work throughout the app).  Many thanks for this excellent suggestion,
I owe you a beer.

> Wrt to the UPDATE method I am not sure
> if we need to know about the concrete derived class at all. Could it work to use straight forward
> polymorphism here? May be I missed something, but if that would work we would be fine with
> one single interface class. Haven't yet found time to actually try it.

This isn't a generic update, i.e. it's not a simple overwrite of
values in Object.  The idea is to allow the client to invoke any
member function in the underlying object.  In the ObjectHandler
example, the relevant function happens to be called Foo->update()
(I'll change this to a less confusing name).  In QuantLibAddin there's
an example which uses this feature to call
QuantLib::VanillaOption->setPricingEngine().

I gather you weren't thinking of the Update method in those terms when
you suggested polymorphism(?), but just to follow through on that
idea:  to achieve this polymorphically, the interface in the base
Object class would need to receive some kind of code or other means of
telling the derived Object class which method in the underlying
QuantLib class to call.  It might be technically doable but it
wouldn't be implementing polymorphism so much as circumventing it and
I don't think it's the right approach.

I'm not certain there's actually a requirement for this feature to
call a member function in the underlying QuantLib object.  I
implemented it because it fit easily into the design, to let people
see the idea and to see if there's any interest.  But usually in a
spreadsheet if you want to change the state of an object you just
delete it and recreate it with new inputs.  There may be cases where
the member function thing could be used to improve performance but I
don't know how often that situation will arise and we might consider
scrapping this feature altogether (with the possibility of
resurrecting it later if required).

Incidentally I have similar thoughts about the so-called "low level
interrogation" feature which enables the client to get directly at the
underlying object - I included it in ObjectHandler to demonstrate the
possibility but I'm not sure the feature is useful or desirable - it
hasn't been needed so far for QuantLibAddin.

> Up to now I did get working a small sample pricing server and a small Excel
> based market data retrieval server.
> I wrote an own little ExcelEngine for accessing Spreadsheets, which hides
> all the nasty COM stuff away.

So the server calls in to Excel with updates in pseudo real-time?  Sounds neat.

> At this stage I was thinking how to sensibly extend the service prototypes.
> One idea I have
> in mind is to provide one big "master-method", which is fed with a variable
> list of arguments. I this case I would need
> a corresponding Dispatcher on the client side.

The ObjectHandler factory function could be the Dispatcher?  The
client could specify whether a request should be executed locally or
remotely, for remote requests the factory function serializes the
request and sends it to the server.

> Also, the load on the server would be pretty high as it not only performs
> pricing but also needs to create the objects from scratch for each request.

We could think about persistence.  The client has a handle for each
object in ObjectHandler - perhaps there would be the possibility for a
handle to refer to an object which is cached on the server.  This also
allows a single object to be shared by multiple clients.

> The remaining problem of wrapping up the complete QL functionality is
> however remaining.
> For the SOAP-calls your idea with (XML-)serialisation seems to be the right
> way. In contrast to Java this is, however,
> not a neglectable task within C++ because in C++ no proper reflection
> mechanism is available (Class.forName(...)).
> Perhaps one could supply a big serialiser/deserialiser class. But this idea
> also seems to be a bit clunky.

I've been assuming there will be (de)serialize member functions in the
base Object class, which derived classes override appropriately - all
of this using QuantLib-FpML/QuantLib-XML to do the real work.  I agree
that the implementation isn't trivial.

> Alternatively there might be other SOAP implementations available, which
> support automatic stub/skeleton-generation from a "normal" C++ class.
> Perhaps one could create the required communication code automatically???

Yes, in the worst case scenario where we need specific code to
(de)serialize each class, it would at least be nice to be able to
generate the code automatically.

> Sorry that I have more questions than answers.

Apology not accepted!  Your feedback is fantastic and has already led
to a fundamental improvement in the ObjectHandler design.  I'm very
glad to be having this discussion because we need to ensure that
ObjectHandler can be extended to support distributed computing and
it's best to get this right on day one.

> Please, let me know what you think about it.

My first thought is that we need more information.  You've anticipated
the design problems that we're likely to encounter, we need to
investigate available tools in more detail, maybe someone has already
solved some of these problems.  I'll see what info I can dig up and I
will keep you posted.

Longer term I think we should do some prototyping - identify the most
promising design, and then code up a small test for a few sample
classes - perhaps with ObjectHandler plugged in to your server - as a
proof of concept.

Regards,
Eric

[eric ehlers]

Hi Walter

> > Having had a look at the current code of interface, I was thinking if one
> > could for example add the name of the class and then use a factory to create
> > an instance of the class in the (now general) MAKE method of interface.

> Yes, this is the right approach.  I've made the change in
> ObjectHandler - the interface layer is now collapsed into a single
> factory function.  It needs some refinement but it's definitely the
> way forward.  I'm in the process of making the corresponding change in
> QuantLibAddin (I already completed a quick test to confirm it will
> work throughout the app).  Many thanks for this excellent suggestion,
> I owe you a beer.

OK, QuantLibAddin is now factory-enabled (and half its former size).
I need to do a lot of refinements before the release but the basic
change is in place, the design is a lot cleaner with the end user
interface working exactly as before.

Looking forward to your further feedback.

Tschuess,
Eric