12 November 2024

Sumitomo Electric Industries, Ltd. has developed low latency media converter (LLMC) technology that enables large-capacity cross reality (XR) video content to be transmitted over networks with ultra-low latency. This technology brings realistic, immersive experiences to remote locations, enhancing advanced XR services such as three-dimensional (3D) imaging, virtual reality (VR), and augmented reality (AR). The Company presented a proof of concept (PoC) report on the technology at the IOWN Global Forum.

1.    Background of Technology Development
With the advancement of high-capacity, high-speed, low-latency communication networks, along with the widespread use of AI, and the enhanced performance of graphics processing unit (GPU) that supports the advancement, expectations are rising for the emergence of new visual services such as XR, which connects distant spaces in the real world with a remote virtual space on the cloud, in real time.

2.    Details of the Technology
The LLMC technology jointly developed by Sumitomo Electric and ATEN International Co., Ltd. (Taiwan) converts DisplayPort*¹ video signals used in head mounted displays (HMDs) and XR glasses for 3D imaging and VR applications, into general-purpose Ethernet signals. The signals are transmitted over the network at an average speed of 0.07 milliseconds (msec), and are reverse-converted on the receiving side.

Combined with high-speed, low-latency networks such as Open APN*², this technology enables real-time long-distance transmission of high-definition and high-refresh-rate video signals*³. It also allows for low-latency reverse transmission of user control signals*⁴, including position information and gaze direction within 3D space.

3. Summary of Verification Results by the IOWN Global Forum Working Group
As part of the IOWN Global Forum Use Case Working Group, Sumitomo Electric, in collaboration with Sony Group Corporation, conducted PoC for the Interactive Live Music (ILM) service*⁵—an immersive, bidirectional remote XR video service. The PoC system incorporated a renderer node utilizing Sony’s technology, Sumitomo Electric's LLMC device, the Open APN Extra-Network Gateway FTU9100*⁶, and Infinera’s coherent optical transceiver ICE-X*⁷. The PoC successfully achieved the target latency of 10 msec for motion-to-photon*⁸, essential for preventing XR motion sickness. The LLMC device marked an average transmission latency of 0.07 msec per unit*⁹.

The companies presented these results as part of the Use Case Working Group at the IOWN Global Forum’s Summer Workshop on June 19 in New York and at the 7th General Meeting on October 8 in Taipei. Additionally, a demonstration was conducted by combining the PoC system with a commercially available VR device.

Moving forward, Sumitomo Electric plans to apply the insights gained from this PoC and related technologies to drive the development and proposal of products for next-generation services.


LLMC specifications

Network Interface	100 Gbps Ethernet（100GBASE-SR4）
Video Interface	DisplayPort1.4（DSC transparent transmission）
Control Interface	USB2.0

*1 DisplayPort is one of the interface standards designed for digital display devices, standardized by the standardization organization Video Electronics Standards Association (VESA).
*2 Open APN is an all-photonics network that enables direct connections between various locations via optical wavelength paths, and is composed of Open APN Extra-Network Gateway, Open APN Transceiver (APN-T), Open APN Gateway (APN-G), Open APN Interchange (APN-I), etc., and the standard is proposed at the IOWN Global Forum.
*3 High-definition and high-refresh-rate video signals: In this PoC, motion-to-photon measurements were performed for Full HD 120 fps (frames per second), 240 fps, 360 fps, and 4K 120 fps.
*4 User control signals for positions and gaze directions within 3D space are generally called 6 degrees of freedom (DoF).
*5 The Interactive Live Music (ILM) service, proposed by the IOWN Global Forum's Use Case Working Group, is an immersive, interactive next-generation 3D/XR service that utilizes AI-integrated communication. For the details of the Use Case documentation and PoC reference, please refer to the following links from the IOWN Global Forum:
・AI-Integrated Communications Use Case Document
・AI-Integrated Communications PoC Reference
*6 The Open APN Extra-Network Gateway FTU9100 is Sumitomo Electric’s Ethernet switch, which connects Open APN and existing networks while providing control functions for APN-T.
*7 Infinera’s ICE-X coherent optical transceiver enables one-to-many communication using the company’s advanced optical coherent technology, providing high-speed, long-distance communication to multiple subscribers with a single transceiver. In September 2023, Sumitomo Electric successfully conducted an overlay test, combining 10G-EPON for home use and ICE-X coherent communication for corporate services on a single-core bidirectional optical fiber network. (https://sumitomoelectric.com/press/2023/09/prs052)
*8 Motion-to-photon latency is the total time from when a user moves their head or gaze to when the sensor detects the motion, the information is sent to the rendering device, where a 2D image is created from the 3D spatial information, and the image is displayed on the image panel of the HMD or XR glasses. To prevent motion sickness, this latency needs to be under 10 msec, a target set by the IOWN Global Forum.
*9 Conventional low-latency media converters typically have a conversion time of about 1.3 msec per device. Additionally, limitations in network speed (10 Gbps) and lack of support for VESA’s Display Stream Compression (DSC) technology which enables low latency light compressed transmission make it difficult for conventional devices to achieve high refresh rates that improve latency on renderer node and display devices. This restricts them to more than 20 msec motion-to-photon latency, even without fiber optic delays. In contrast, this technology which enables the transmission of high refresh rate low-latency, lightly compressed DisplayPort signals using DSC over Ethernet and Open APN at speeds exceeding 10 Gbps (25Gbps to 100Gbps), achieved latencies of 6.7 msec and 8.6 msec in the measurement system shown in the figure, at refresh rates of 360 fps and 240 fps, respectively, which meets the target latency of 10 msec even over optical fiber transmission distances of approximately 300 km and 100 km from the edge data center to the user.


ATEN International Co.
ATEN International Co., Ltd. (TWSE: 6277), established in 1979, is a global leader in KVM and AV/IT connectivity solutions.
Headquartered in Taiwan, subsidiaries in multiple countries, including the U.S., U.K., China, and Japan, with R&D centers in Taiwan, China, and Canada.
For more information, please visit http://www.aten.com/.

Infinera Corporation
Infinera is a global supplier of innovative open optical networking solutions and advanced optical semiconductors that enable telecommunications service providers, cloud operators, government agencies, and enterprises to expand network bandwidth, accelerate service innovation, and automate network operations. Infinera's solutions deliver industry-leading cost-effectiveness and performance in applications such as long-distance, submarine, data center interconnect, and metro transport. For more information about Infinera, please visit www.infinera.com, follow us on X and LinkedIn, and subscribe to receive the latest updates.

IOWN Global Forum
The IOWN Global Forum was established in 2020 as a private sector organization to develop IOWN technologies and use cases. As of the end of 2023, this forum is comprised of over 150 organizations. The objective of the IOWN Global Forum is to accelerate the innovation and adoption of a new communication infrastructure to meet our future data and computing requirements through the development of new technologies, frameworks, specifications, and reference designs in areas such as photonics R&D, distributed computing, use cases and best practices. For more information, visit https://iowngf.org/.