These instances could run on mobile, Windows, Mac, Linux, etc.

The system needs to be able to perform highly parallel computing with speed & efficiency, ideally take advantage of SSE, and ideally support CUDA/OpenCL.

I have considered the following:

Java – it is memory hungry and doesn’t run on Mac (not officially, anyway)
.Net – memory hungry; limited in platform scope; no native SSE
Delphi – not 64 bit; limited platform scope
C/C++ – not directly supported by Mac; complex to code; however it is ubiquitous
Objective-C – supported by Mac; appears to be supported elsewhere; works by passing messages, which is per my design requirements; don’t know much about it

Here is my run down of some options (in no particular order):

C/C++

If all you care about is performance (and nothing else), these will provide. Direct access to system level constructs, such as processor affinity and inline assembly can certainly have an impact on performance. However there a 2 main drawbacks to the C/C++ option. Firstly neither have a well defined memory model, so the memory model you are developing against is that of the CPU you are running the system on (if you don’t know what a memory model is how it applies to concurrent programming, then you shouldn’t be doing it). This ties you very tightly to a single platform. The second is the lack of a garbage collector, manual memory management is tricky (but doable) in the simple cases, with C++ a number of supporting utilities simplify the problem (e.g. auto pointers/smart pointers). When writing concurrent code it is an order of magnitude harder as the rules for should release a certain piece of memory become very hard to define. E.g. when a object is passed onto a different thread, who’s responsible for releasing it? If using C++ it’s important to ensure that you are using thread safe versions of the classes used to help manage memory. E.g. the boost smart pointer only supports the use of methods declared as “const” across different threads (e.g. dereferencing is OK), however non-const methods (e.g. assignment) are not thread safe.

Java

Java would be my recommendation, it has support on all of the platforms you mentioned (including mobile, e.g JavaME and Android) as well as CUDA support. Java has a well defined memory model that is consistent across platforms, a robust and mature JIT and optimiser, and number of good and improving garbage collectors. Most general purpose applications written in Java will run just as fast as their C/C++ counterparts. While it is a bit of memory hog, if you are doing HPC work you are most likely going to throw some decent hardware at the problem. Given that you can address 100s of GB on commodity hardware, the memory problem is less of an issue than it used to be. Mobile is the only real area where memory usage is constrained and the specialist runtime environments perform better in this respect (see above). The only main drawback for Java (from an HPC perspective) is the lack of pass by copy complex types (i.e. structs in C#) so all complex objects have to be heap allocated putting pressure on the GC. Escape analysis is supposed to help with this somewhat, however it is difficult to get it do work well in very general cases (note that it has jumped in and out of various revisions of the JDK recently).

Worth mentioning the broad language support (Scala and Groovy++ have pretty good performance profiles) and there are a number of message passing concurrent frameworks (actors, akka, kilim).

C#/.Net

Probably the most complete from a language perspective, especially if you include things like F# for approaching things in a functional manner. However you are generally pushed toward an MS platform if you want the best performance (and the mobile story here is not great either). Mono produces a pretty good platform feature wise (caveat: I’m a contributor) however they are still playing catch up with performance, e.g. an accurate, compacting collector has recently been added and is still a bit experimental.

Google Go

Very new, but interesting from the perspective that it is natively compiled (no jit overhead), but still has a strong runtime, memory model, garbage collector and direct support for CSP (concurrent sequential processing) features in the languages (channels). Probably has some ways to go, the development of a new GC has gotten under way but not surfaced yet and there is probably a lot more they can get out of the compiler.

Haskell and other functional languages

Not a lot of experience here, but some of the functional constructs such as immutability and persistent collections can be quite useful for building robust concurrent applications.