Current cyberinfrastructure (CI) in wildlife monitoring is limited to normal angle videos with a limited field of view and has caused missing the recording of important events that occurred outside of the direction being filmed. Moreover, existing remote cameras only allow the recording of short videos for a few minutes and thus cannot document many hours of wildlife activity in the monitoring zone. This project proposes methods for panoramic video monitoring that capture 360 degree uninterrupted videos to document complete wildlife activities. The project will allow wildlife scientists to access high fidelity monitoring data in both the spatial and temporal domains. The abundant research data and metadata embedded in panoramic videos will enhance the productivity of biologists and ecologists.
To this end, we first propose camera computing strategies to maximally compress the video with negligible overhead. This would mitigate the overall need for storage. Second, we propose a networked storage scheme to address the intermittent nature of the network in the wild, where only partial video is transported while the remaining video is generated in the receiver. Finally, we will develop and validate the panoramic video monitoring in wildlife scenarios.
People
- PI Zhisheng Yan
- External Collaborator Klara Nahrstedt, UIUC
- Students Anh Nguyen, Shiqing Luo, Jun Yi, Bo Chen, Taslim Murad, Xinyu Hu, Md Islam
Publications
- Enhancing 360 Video Streaming through Salient Content in Head-Mounted Displays Anh Nguyen and Zhisheng Yan SJ (Sensors Journal), 23(8), 4016, 2023. [PDF]
- DAO: Dynamic Adaptive Offloading for Video Analytics Taslim Murad, Anh Nguyen, and Zhisheng Yan ACM MM (ACM International Conference on Multimedia), October 2022. [PDF]
- Dissecting Latency in 360 Video Camera Sensing Systems Zhisheng Yan and Jun Yi SJ (Sensors Journal), 22(16), 6001, 2022. [PDF]
- Context-aware Image Compression Optimization for Visual Analytics Offloading Bo Chen, Zhisheng Yan, and Klara Nahrstedt ACM MMSys, (ACM International Conference on Multimedia Systems), June, 2022. [PDF]
- Deep Contextualized Compressive Offloading for Images Bo Chen, Zhisheng Yan, Hongpeng Guo, Zhe Yang, Ahmed Ali-Eldin, Prashant Shenoy, and Klara Nahrstedt ACM SenSys - AIChallengeIoT (ACM SenSys Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things), November 2021. [PDF]
- An Analysis of Delay in Live 360 Video Streaming Systems Jun Yi, Md Reazul Islam, Shivang Aggarwal, Dimitrios Koutsonikolas, Y Charlie Hu, and Zhisheng Yan ACM MM (ACM International Conference on Multimedia), October 2020. [PDF]
- How to Evaluate Mobile 360 Video Streaming Systems? Shivang Aggarwal, Sibendu Paul, Pranab Dash, Nuka Saranya Illa, Y. Charlie Hu, Dimitrios Koutsonikolas, and Zhisheng Yan ACM HotMobile (ACM International Workshop on Mobile Computing Systems and Applications ), March 2020. [PDF]
Code and Data
- Zeus
Zeus is the first live 360 video streaming research prototype for human viewing. It uses publicly available hardware and software and can be used for benchmarking and research designs in 360 video systems.
[Paper] [GitHub Project]
- WildPano WildPano is the first 360 wildlife video dataset. It consists of 30 videos. The average video length was 2 minutes 4 seconds. A total of 29 species were recognized.
- PanoSalNet PanoSalNet includes a CNN-based panoramic saliency detection model for 360-degree video and a LSTM-based head movement prediction model for 360-degree video streaming systems.
Funding
This project is sponsored by the National Science Foundation CRII Program under award number 2151463.