The Augmented Reality Edge Networking Architecture (ARENA) is the center’s main integration platform. ARENA’s main goal is to make mixed reality “digital twin” systems as easy to program as the web. The ARENA streamlines cross-platform interaction with 3D content generated by any number of network connected agents (human or machine) in real-time. It performs the role of a distributed operating system for environments that exhibit a tight coupling between physical and virtual assets. One critical component is the ability for programs to execute and programs and data to migrate seamlessly across any number of connected compute resources including edge servers, in-network processing units and/or end-user devices like headsets and mobile phones. The system is built on emerging web standards like WebXR with safe and secure runtime support provided by WebAssembly.

During years 4 and 5, Theme 1 will continue to focus on prototyping compelling applications that disrupt the prevalent “cloud, middle, edge” system design paradigm. Our original demonstrator applications (autonomous systems, mixed reality, and smart cities) are now being explored under the unifying “Connected Environments” task. Moving forward, our emphasis will be on the system infrastructure needed for real-time network observability, visualization, and in-network compute migration and adaptation. Task “Composable, Secure Common Runtime” will develop and integrate a programmable distributed runtime system tailored to the demands of a highly heterogeneous ecosystem, upon which distributed ARENA applications can be built. To demonstrate tight cooperation between users and other computing systems in this environment, Task “Connected Environments” will explore our multi-agent drone application in the ARENA. This will showcase our architecture support for sensors and actuators at the edge and seamless transition between autonomous and human-in-the-loop networked control. CONIX is developing many visual analytics capabilities (e.g. 3D pose estimation for indoor, augmented reality; detecting complex activities across multiple outdoor surveillance cameras; processing and extracting relevant information from LiDAR and stereo cameras) which can be integrated into the ARENA platform. As these applications are developed, Task “Distributed Video Analytics” will produce new benchmarks and data-flow graph descriptions for distributed visual analytics applications.

DSSP Tasks These DARPA-Sponsored Special Project (DSSP) tasks will both leverage and enhance ARENA capabilities by developing a new kind of distributed wireless spectrum/communications sensing and visualization tool (XRshark), and by integrating with ATAK, an interactive geospatial tool for Android mobile phones that aids in coordinating mobile teams in real‐time. “Distributed RF Spectrum Sensing.” The main research outcomes for this task are to: (i) characterize network joining, churning, and updating state and its associated overhead; (ii) demonstrate that a distributed architecture can implement the functionality of a centralized one with a constrained performance and power penalty, and integrate with CONIX platforms and ARENA architecture. “Android Tactical Assault Kit (ATAK) XR.” This task will develop two main technical components: (1) design and integration of a rapidly deployable GPS‐denied localization capability for mobile phone platforms (i.e. ATAK) and (2) integration of a Web XR compatible runtime to view and interact with 3D content. The system should remain compatible with COTs Android and support upcoming spatial web frameworks.