notesum.ai

Published at November 29

Ten Ways in which Virtual Reality Differs from Video Streaming

cs.PF

cs.MM

cs.NI

Released Date: November 29, 2024

Authors: Gustavo de Veciana¹, Sonia Fahmy², George Kesidis³, Voicu Popescu²

Aff.: ¹The University of Texas at Austin; ²Purdue University; ³Penn State University

Arxiv: http://arxiv.org/pdf/2411.19730v1

Refer to caption

Aspect	Stored 2D Video Streaming	Networked Virtual Reality
(A) Application characteristics
1. Interaction	Mostly sequential viewing, with typically-infrequent viewing-position changes.	Users change location and orientation, and potentially interact with the virtual environment or other users in real time and in unpredictable ways; users may teleport.
2. Content dynamics	Mostly MPEG-encoded video (with potentially additional streams for encoded audio and subtitles).	Heterogeneous and may include (textured) polygon meshes or point clouds, animations, stored or live video, audio, and subtitles. Much of the content may be static and may have a long useful life.
3. Resource needs	Relatively simple due to predictable asynchronous interaction with each user.	Complex due to an individualized experience, heterogeneity of content, and synchronous interaction. When a set of users is close (in both the virtual environment and the physical world), can amortize computation, caching, and communication costs.
4. QoE requirement	QoE is a function of startup delay, stalls, frame quality and variability.	Requires $\sim$ 10 ms motion-to-photon latency to prevent cybersickness. QoE is a complex function of usability, 3D perception, response time to every action, delay with every teleportation, headset energy consumption and temperature, in addition to frame quality and variability.
(B) Rendering and adaptation
5. Rendering	Video displayed on client; no computation on an edge server is typically required.	Choice between (i) Rendering on client device (constrained by device capacity and does not leverage multi-client sharing) or (ii) rendering on an edge server (constrained by network latency and bandwidth), or (iii) partial rendering on both client and server(s).
6. Adaptation	Select one of a few available bitrates for each (typically 4-second) “chunk.”	Dynamically employ level-of-detail and visibility-based virtual environment complexity reduction based on the user field of view.
(C) Prefetching and caching
7. Prefetching	Sequentially prefetch as long as playout buffer space is available.	Difficult to accurately predict user view and behavior for prefetching.
8. Caching	Already-viewed content typically discarded at the client.	Can cache parts of virtual environment that may be visited or revisited; cache replacement and sharing policies important.
(D) Transport
9. Prioritization	Data is mostly fetched sequentially.	Prioritize data to fetch based on visibility, level of detail, and sharing.
10. Delivery	In-order, reliable, delivery to the application typical.	Do not always need in-order delivery or complete reliability.