Peak Performance vs Sustained Performance
Conditions which inhibit achieving a high, relatively stable, level of sustained performance:
- Data access patterns may change over the algorithm execution time. Most message passing algorithms are tuned such that the data space and computation space are divided as evenly as possible among the processors.
- Changing the data access patterns may result in a communication bottlenecks
- Dynamic Load balancing is the process of adjusting data space/computation space division on the fly in order to adjust to changing conditions
- Tera’s claim is that their latency hiding feature insulates them from performance dependencies on memory access patterns