The most exciting recent astrophysical events have involved transient phenomena. Notable examples are the possible correlation of an astrophysical neutrino with a flaring gamma-ray source, and the observation of gamma rays and gravitational waves produced by a Kilonova. The potential very-high-energy gamma-ray emission from such events is of particular scientific interest, tracking extreme acceleration processes. However, the occurrence of such transients can often only be anticipated on a very short term and, hence, is not considered in traditional approaches to scheduling observations.
This project sets out to increase the coverage of transient and variable gamma-ray sources through dynamic scheduling of observations. To this end, time series data representing light curves of gamma-ray and further energy bands will be used for short-term forecasting of flares, e.g., based on outlier detection or recurrent neural networks. Combining these forecasts with contextual information, such as monitored weather conditions, source positions in the sky, and telescope permutations in the participating array, the observing schedule is optimised.
Any realisation of the above idea, however, has to cope with scalability and responsiveness challenges: Time series data are collected at high rates (several hundred Hertz for current telescope arrays) and some gamma-ray sources are known to be variable at minute timescales. Hence, low-latency data processing and efficient online decision making are of crucial importance for dynamic observation scheduling. This project aims at providing the respective conceptual and technological foundations, by answering the following research questions:
• What are models for online decision making that combine approaches for flare forecasting with contextual information for optimal observation scheduling? This includes questions related to the expressiveness needed for the decision mechanism as well as the temporal and spatial granularity considered in these models.
• How can the online decision making be expressed in a computational model that is based on streaming data? Here, important aspects are the required operator algebra and data correlation mechanisms. Also, the notions of state to be maintained during processing is to be clarified.
• How to optimise the latency of stream processing for online decision making? Directions to answer this question are (i) prefetching of contextual information for low-latency assessment of flare forecasts, (ii) state management for streaming operators, and (iii) approximate stream processing using data sketches.
In sum, the results of this project will be a grounding of dynamic observation scheduling in models for data stream processing along with algorithm for their efficient realisation.