[ale] OT: Multi-core Utilization

[Jim Lynch]

On 03/08/2013 11:33 AM, Jeff Hubbs wrote:
> My *practical* experience has a hole in it when it comes to developing 
> software to efficiently use multiple cores in a machine.
>
> If I'm writing code in the likes of C++, Python, or Fortran 
> (acknowledging that I've got a range of programming paradigms there) 
> and let's say that I'm subtracting two 2-D arrays of floating point 
> numbers from one another element-wise, how is it that the operation 
> gets blown across multiple CPU cores in an efficient way, if at all?  
> Bear in mind that if this is done in Fortran, it's done in a pair of 
> nested do-loops so unless the compiler is really smart, that becomes a 
> serial operation.
When I worked for Cray Research, the Portland Group had a FORTRAN 
compiler that handled multiple processors using compiler directives to 
separate tasks and define regions.   I took a class from them because a 
number of our customers were electing to use their compiler. That was 
before CRI had a decent multi-processing compiler.  They're still in 
business and since they've had a lot of years understanding that 
paradigm, they've probably got a good solution.  That's only if you have 
the cash to spend.

I've written, debugged an cussed my share of MP programs, mostly in 
FORTRAN.  The C and C++ compilers weren't as efficient, due to the more 
complex nature of those compilers.  FORTRAN lent itself well to 
implementing MP techniques.  I should clarify.  You can get good 
efficiency out of either compiler, if you write your code in a manner 
that lends itself to analysis.  It's easy in C to write a loop that 
can't be automatically parallelized.  It's harder to do in FORTRAN.  
What I'm saying is that the typical FORTRAN scientific program doing 
vector or matrix math is much more likely to lend itself to 
parallelization than a typical C program.  FORTRAN programmers like 
arrays, C programmers like pointers.


Jim.