Friday, November 28, 2008

Not so much increase in parallel performance

A few days ago I was celebrating the dramatic improvement of computing
performance using "snow" package in R.
(http://somerandomwalksofmind.blogspot.com/2008/11/my-first-parallel-program.html).
But later when I posted the question on R mailing list
(http://www.r-project.org/mail.html), some people pointed out that in
fact I did not such more-than-doubling improvement in performance.

The trick, according to Stefan Evert
(http://www.nabble.com/More-than-doubling-performance-with-snow-td20654005.html),
is to look at the elapsed time in the system.time() output. In that
case the boost in speed is not so large. For example:

> library(snow)
>
> cc <- makePVMcluster(2)
>
> n.size <- 1000
>
> temp <- NULL
> for(i in 1:10){
+ x <- list(matrix(rnorm(n.size^2),n.size))
+ temp <- c(temp,x)
+ }
>
> system.time(t.1 <- clusterApply(cc,temp,"solve"))
user system elapsed
2.980 0.548 21.909
> system.time(t.2 <- lapply(temp,"solve"))
user system elapsed
24.290 0.636 25.058

So to further gain increase in execution in speed using snow I may
have to have a computer with many cores (much more than my current duo
core laptop).

No comments: