Login  Register

Re: Sqrt of large correlation matrix

Posted by Etuka Onono-2 on Aug 01, 2016; 9:33am
URL: http://quantlib.414.s1.nabble.com/Sqrt-of-large-correlation-matrix-tp17596p17629.html

I don’t run calculations on large matrices, so take the comments below with some scepticism.  Assuming the functions are working as intended:

1) I don’t see that you gain anything from calling rankReducedSqrt with 100% retention.  Effectively, you keep the whole matrix this way.  Try experimenting with lower values (but large enough to give your exercise meaning).  If the eigenvalues in your matrix don’t decay rapidly in value, you may have to resign yourself to a lengthy calculation.
2) 4000 is a large matrix and QuantLib calculates all of the eigenvalues of the matrix, even when calculating the rank reduced square root matrices.  I can’t remember (or never knew) the efficiency of the Schur decomposition algorithm, but I am willing to bet that it is expensive.  You could try applying the Schur decomposition method in the algorithm and see if that is the bottleneck.  If it is, then methods that involve calculating the biggest eigenvalues that you are interested in only might be competitive.

Best regards,
Etuka



On 1 Aug 2016, at 11:04, [hidden email] wrote:

Send QuantLib-users mailing list submissions to
[hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/quantlib-users
or, via email, send a message with subject or body 'help' to
[hidden email]

You can reach the person managing the list at
[hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of QuantLib-users digest..."
Today's Topics:

  1. Re: Quantlib 1.8 Python SWIG on Linux/Python 3.5.1
     (Luigi Ballabio)
  2. Re: QuantLib SWIG C# - Calling Conventions (cdecl/stdcall)
     issue (Luigi Ballabio)
  3. Re: Sqrt of large correlation matrix (Luigi Ballabio)
  4. Re: What is the meaning of time variable in GBM sample paths
     (Luigi Ballabio)

From: Luigi Ballabio <[hidden email]>
Subject: Re: [Quantlib-users] Quantlib 1.8 Python SWIG on Linux/Python 3.5.1
Date: 1 August 2016 at 09:40:00 BST


Apologies for the delay. Yes, more RAM would help...

On Sat, Jul 16, 2016 at 4:10 PM Jay-Jay-D <[hidden email]> wrote:
I tried with your suggestion, but the outcome is the same.

I also tried in in the same Fedora but with out Anaconda, and same problem.

Maybe my machine is to small? It has only 2 GB RAM.



--
View this message in context: http://quantlib.10058.n7.nabble.com/Quantlib-1-8-Python-SWIG-on-Linux-Python-3-5-1-tp17463p17603.html
Sent from the quantlib-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports.http://sdm.link/zohodev2dev
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users



From: Luigi Ballabio <[hidden email]>
Subject: Re: [Quantlib-users] QuantLib SWIG C# - Calling Conventions (cdecl/stdcall) issue
Date: 1 August 2016 at 09:41:24 BST


Hello Fabrice,
    yes, adding the configurations would be nice, thanks.

Luigi


On Sat, Jul 2, 2016 at 12:07 PM Fabrice Lecuyer <[hidden email]> wrote:
Hi all,

 (email already sent as a non member, apologies if it creates a duplicate)

I’ve been building QuantLib & QuantLib SWIG “as is” with VS14 for use with C# and came across an initial error when running in debug, where visual studio breaks giving a PInvoke stack imbalance error, as described in this old thread:
I first tried the solution described in this thread (modifying the SWIG generated file to add calling convention = cdecl) and all seemed ok for a while.
The next problem appeared when QuantLib C++ was throwing an error, the C# wasn’t catching it and instead crashing with a memory imbalance error. Again, this is described in the thread below, but this time without a solution.
In order to catch the thrown C++ error, I’ve had to revert to the original SWIG generated C# source file, and instead build QuantLib C++ using the stdcall calling convention (default in VS14 proj files is cdecl).
Now everything is good, but I just wanted to ask around to other C# users if this is a problem they have had, if they solved it this way that I’m safe to assume I won’t find another issue of the kind down the line, or if I’m unnecessarily complicating things.
If this is indeed the best solution, would adding build configurations for stdcall be considered? Happy to provide the updated vcproj files if yes.
 
Regards,
Fabrice.
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users



From: Luigi Ballabio <[hidden email]>
Subject: Re: [Quantlib-users] Sqrt of large correlation matrix
Date: 1 August 2016 at 10:00:26 BST


Hello,
    apologies for the delay. Did you get any feedback on this?

Luigi


On Fri, Jul 15, 2016 at 10:29 AM ian_dfw <[hidden email]> wrote:
I am a risk manager and trying to run Monte Carolo sim for a large number of
issues. So far I have a 4000 x 4000 correlation matrix. With rankreducedSqrt
function with Spectral and 100% retention, it took 3 hrs to complete the
calculation.  Am I doing this the most efficient way? Is there anyway to
improve performance wise?  Thanks.  Following is the code:

rankReducedSqrt(*corr_mat, 4000, 1.0, SalvagingAlgorithm::Spectral);



--
View this message in context: http://quantlib.10058.n7.nabble.com/Sqrt-of-large-correlation-matrix-tp17596.html
Sent from the quantlib-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports.http://sdm.link/zohodev2dev
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users



From: Luigi Ballabio <[hidden email]>
Subject: Re: [Quantlib-users] What is the meaning of time variable in GBM sample paths
Date: 1 August 2016 at 11:04:14 BST


It's the length of the paths as a fraction of year, so 100/255.0 should work.  I haven't checked for extreme values, but when I try your code, extract 10000 sample paths and collect the end points, I get a standard deviation between 5.4 and 5.5 for 255 time steps and length = 1, and a std dev around 2.9 for 100 time steps and length = 100/255.0.

Luigi


On Mon, Jun 27, 2016 at 2:31 PM <[hidden email]> wrote:
I use the code (taken from https://mhittesdorf.wordpress.com/2013/12/29/introducing-quantlib-modeling-asset-prices-with-geometric-brownian-motion/) below to generate GBM sample paths. The code is for a sample path of one year. I would however like to make sample paths for 100 trading days. So I adjusted time_steps to 100, but I don’t know what to do with the length variable. When I set it to 100/255, the extreme values of the sample paths are much bigger and more frequent then when I just leave it at 1 (which to me seems counterintuative). So what is the meaning of the length variable and to which value should I set it? Thank you.
 
 
Real starting_price = 20.16; //closing price for INTC on 12/7/2012
Real mu = .2312; //INTC one year historical annual return
Volatility hv = 0.2116; //INTC one year historical volatility
Size time_steps = 255; //trading days in a year (U.S.)
Time length = 1; //one year
 
//instantiate Geometric Brownian Motion (GBM) stochastic process
const boost::shared_ptr<StochasticProcess>& gbm = boost::shared_ptr<StochasticProcess> (new GeometricBrownianMotionProcess(starting_price, mu, hv));
 
//generate a sequence of normally distributed random numbers from a
//uniform distribution using Box-Muller transformation
BigInteger seed = SeedGenerator::instance().get();
MersenneTwisterUniformRng mersenneRng(seed);
BoxMullerGaussianRng<MersenneTwisterUniformRng> boxMullerRng(mersenneRng);
RandomSequenceGenerator<BoxMullerGaussianRng<MersenneTwisterUniformRng>> gsg(time_steps, boxMullerRng);
 
//generate simulated path of stock price using GBM stochastic process
PathGenerator<RandomSequenceGenerator<BoxMullerGaussianRng<MersenneTwisterUniformRng>> > gbmPathGenerator(gbm, length, time_steps, gsg, false);
const Path& samplePath = gbmPathGenerator.next().value;
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users


------------------------------------------------------------------------------
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users


------------------------------------------------------------------------------

_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users