Towards a European Solar Data Centre (WP5)

Given the number of solar infrastructures operated and under development by the European institutions, WP5 focuses on activities of common interest to the Solar Physics community for the curation and dissemination of solar data, to boost its use and re-use and thus optimize its scientific exploitation in line with the spirit/essence of the European Open Science Cloud (EOSC) project. These common efforts can be understood as the first steps towards building up a European Solar Data Centre for EST. The activities of this work package respond to the demands on data management which is an essential theme throughout this SOLARNET proposal.

The European Solar Physics community operates and exploits a number of observing facilities dedicated to observe the Sun with high-resolution. They provide state-of-the-art imaging and spectro-polarimetric observations with excellent scientific quality. Yet, with the steady improvement of the observing capabilities, we are experiencing a significant increase in the data rates and data volumes generated at the observatories. In addition, with the new 4m-class European Solar Telescope (EST) on the horizon, the European Solar Physics community urgently requires efficient handling and calibration of large data volumes and equally important, the archiving and development of advanced data search tools that guarantee the use and re-use of data, maximising its scientific exploitation and sustainability.

WP5 will pursue the following activities: data calibration, data archiving and development of advanced data search techniques, the Solar Virtual Observatory (SVO) as a frame work for data dissemination, and high-level data products and visualization tools. The work package combines the competences of 16 partner institutions, including one industrial partner. The first activity implements automatic calibration pipelines for panoramic spectrometers and investigates several new approaches to accelerate and improve post processing of high resolution observations, with a view to EST. There will be an interface to WP7 on adaptive optics development in this part of the project. The second activity focuses on applying new approaches to data archiving and data mining including Automatic Test Equipment (ATE) application of deep learning methods for data retrieval. The third activity will carry SVO from prototype to production level. The fourth activity addresses the question of co-alignment of data from various sources and the development of basic and advanced tools for data analysis, focusing on specific scientific questions.

The activities in WP5 complement the other WPs to complete the life cycle in solar research: (1) Science drives the request of access (through WP9) to (a) the solar observatories (for which new observing techniques and instrumentation is developed in WP6, WP7 and WP8), and/or (b) to run theoretical models in supercomputing centres; (2) the observational data must be first calibrated (WP5), and (3) further value is added to the science-ready data when developing new high-level data tools (WP5); (4) the data must be curated for efficient archiving (WP5 and WP2, by defining standards in metadata); (5) data can be then disseminated to the scientific community via a SOLARNET Virtual Observatory (WP5) through the Virtual Access Programme (WP10), and via outreach (WP4) and networking activities (WP3) within the scientific community.

Diagram of the life cycle in solar physics research

WP5 brings together 14 research institutes and 1 SME specialised on Python software development. According with the objectives of WP5, the following lines of action are pursued:

List of activities of sub-WPs in WP5 based on data centre activities, deliverables and institutes engaged.