SC24 BoF: Building a Global Data Streaming Community: lessons learnt, potential tools, challenges and opportunities – the European and US ecosystems address the latest paradigm shift towards global data streaming in large data systems
Abstract
Data Streaming is a proven, widely used technique. It’s not commonly used in HPC yet, but it can support critical applications with high societal and scientific impact. This BoF examines use cases like Destination Earth, Large Hadron Collider, Square Kilometre Array, Exa-AToW, and Earth systems digital twins by NASA, and discusses how to make streaming of measurement/sensor or simulation data useful in the context of such large, data-oriented initiatives, which all critically rely on HPC infrastructures.
The aim is starting a global HPC Data Streaming Community, to examine lessons learnt, adapt data streaming approaches and drive integration with HPC systems.
Session leaders
- Sai Narasimhamurthy, ETP4HPC and ParTec AG
- Mike Little, NASA
- Thomas Geenen, ECMWF
- Hans-Christian Hoppe, ParTec
- Damien Gratadour, Observatoire de Paris
- Laurent Morin, INRIA