Technological advances in high performance computing – processing speed, memory systems, storage and networking – are creating exciting new opportunities that move well beyond improving the precision of simulation models. The use of extreme computing in real-time applications with high velocity data and live analytics is within reach. The availability of fast growing social and sensor networks raise new possibilities in monitoring, assessing and predicting environmental, social and economic incidents as they happen. Add in grand challenges in data fusion, analysis and visualization, and extreme computing hardware has an increasingly essential role in enabling efficient processing workflows for huge heterogeneous data streams.

VESTEC will create the software solutions needed to realise this vision, to connect big data streams and extreme computing for urgent decision making in various fields with high impact for the European community. VESTEC will build a flexible toolchain to combine multiple data sources, efficiently extract essential features, enable flexible scheduling and interactive supercomputing, and realise 3D visualization environments for interactive explorations by stakeholders and decision makers. VESTEC will develop and evaluate methods and interfaces to integrate high-performance data analytics processes into running simulations and real-time data environments. Interactive ensemble management will launch new simulations for new data, building up statistically more and more accurate pictures of emerging, time-critical phenomena. Innovative data compression approaches, based on topological feature extraction and data sampling, will result in considerable reductions in storage and processing demands by discarding domain-irrelevant data. In-situ rendering will provide images for quick and accurate perception of findings.

Three emerging use cases will demonstrate the immense benefit for urgent decision making: wildfire monitoring and forecasting; analysis of risk associated with mosquito-borne diseases; and the effects of space weather on technical supply chains. VESTEC brings together experts in extreme computing, massively parallel simulation codes, data analytics, and visualization to address the challenges holistically.