Advancing Video IP Delivery from Basic Compression to 4K/8K, and XR Technologies
Evolution of Video Communication and Sumitomo Electric’s Mission
Hiroaki Nishimoto
Fellow,
Specially Appointed Management,
Chief Technology Supervisor of Infocommunications Business Unit & Innovative Network Solution Department
My career has been at the intersection of technological development and business operations in communications, driven by the allure of IP communications. I was convinced early on that IP communications would evolve into a robust and convenient infrastructure for transmitting video information. Anticipating that video applications would eventually dominate internet traffic, I identified significant technical and business opportunities.
Pioneering IP video distribution technology with excellent internet compatibility
Our commitment to video distribution technology dates back to 1978, five years before I joined the Company, with the launch of Hi-OVIS, the world's first two-way CATV system. This groundbreaking project provided bidirectional video communications by connecting about 150 ordinary homes and six public facilities through optical fiber, garnering significant attention both domestically and internationally. We pioneered the development of core products and technologies, such as optical fiber, optical data links, and video switching equipment.
In 1993, the introduction of Internet services in Japan laid the groundwork for significant advancements in communication technologies. This potential was abruptly tested in 1995 when the Great Hanshin-Awaji Earthquake struck, severely damaging the nation's telecommunications infrastructure. Despite the devastation, the resilience of the emerging Internet was showcased as it became a crucial tool for emergency communication. Local businesses, academic institutions, and individuals used the network to broadcast critical safety information and images from the disaster zones to a global audience. This real-time usage under extreme conditions highlighted the Internet's capacity to support robust, reliable communication in crisis situations.
Prompted by this, our team was driven to enhance and secure the reliability of video communications over IP networks. We began developing a sophisticated IP video distribution system that utilized advanced video compression technology from IBM Endicott. This initiative not only addressed the immediate needs for efficient, large-scale video transmission but also set the stage for future innovations in video distribution, particularly for high-stakes environments.
Overcoming the challenges of high-quality video streaming in the era of broadband
In 1999, Japan led the world into the broadband era with the adoption of the ADSL standard, bringing about an era of high-speed, high-capacity data communication. This shift was met with a growing demand for digital high-definition content, setting new challenges due to the limitations of the TCP/IP protocol, which transmits data by breaking it into small packets and reassembling them at the recipient's end. The frequent loss and corruption of data packets during peak times necessitated a novel approach beyond traditional retransmission methods to maintain video quality.
These challenges were particularly acute for paid video content, where users expected uninterrupted and clear viewing experiences. As broadband usage increased and the demand for flawless digital video grew, the need for a more robust solution to address network instabilities became apparent. To tackle this critical issue, in 2002, we proposed the development of a network system, including an IP set box (IP-STB), to the National Institute of Information and Communications Technology, a division under the Telecommunications and Broadcasting Organization of the Ministry of Internal Affairs and Communications. Our proposal was adopted as a national project, and addressed through a collaboration with Digital Fountain, a venture originating from UC Berkeley, and academic institutions such as Kyoto University and Osaka University. Our executives emphasized the gravity of this mission, reminding me that this national project was funded by taxpayers' money and failure was not an option. This admonition instilled a profound sense of responsibility and motivation in me, reinforcing the notion that such an opportunity to succeed was rare and must be seized.
Forward error correction (FEC) codes, specifically LT Code, were key to reconstructing lost packets. Leveraging expertise from Digital Fountain, we developed an improved FEC, which was integrated into an IP-STB. We successfully conducted the world's first trial service, demonstrating the effectiveness of this new FEC, and in 2004, it was adopted for commercial use by the NTT Group. I am proud to say that this achievement represented the world’s leading technology for maintaining video quality at the time.
Collaborating with external partners to embrace open innovation
2007 marked a pivotal year for our video distribution business as we collaborated with the NTT Group to develop the world's first IP-STB for the IP retransmission of full high-definition terrestrial digital broadcasting. This project was aimed at several key objectives, including reducing the delay between broadcast and IP retransmission to no more than 2.5 seconds, while ensuring the preservation of high-quality video, audio, and data to keep the “Identity of Contents.” We also focused on implementing robust digital rights management (DRM) to protect content by controlling its use and reproduction, dedicating significant resources to rigorous testing. Our office worked around the clock for this intense development phase.
Our IP-STBs dominated the market, achieving a 100% market share for two years and shipping nearly one million units of a single model. This success was largely due to our collaboration with outstanding external partners, embodying our commitment to open innovation. Throughout not only this project but in all our endeavors, collaborating and cooperating with external entities has been crucial part of our innovation. I believe that this collaborative approach not only fueled our past achievements but continues to be a key strategy in driving future innovations and maintaining our leadership in the market.
Pioneering 4K+10G and next-generation technologies
The introduction of 4K/8K resolutions has significantly impacted the video distribution market. To support the expansion of 4K IP broadcasting, we have accelerated the development and sales of 10G-EPON devices, enhancing FTTH (Fiber to the Home) infrastructure. Simultaneously, our STBs and distribution servers have been upgraded to support 4K content. In 2014, we achieved a milestone as the first company to have an IP-STB certified by a North American film distributor for meeting 4K premium content security requirements. By December 2020, our shipments of 4K-ready STBs surpassed one million units, and by May 2021, models supporting BS4K broadcasting and Android TV, including YouTube, also reached the one million mark. In total, STB shipments have topped five million units, making us a market leader in 10G-EPON and 4K-STBs in Japan, which has greatly enhanced the appeal of IPTV and CATV services.
Paving the way to the next generation with groundbreaking 3D/XR video streaming service
Following the success of 4K/8K, we are now focusing on 3D and cross reality (XR) video distribution services. These services aim to deliver immersive experiences using XR glasses, head mounted displays (HMDs), or 3D flat panel displays, synthesizing XR images based on the viewer’s line of sight. Effective deployment involves using cloud computing servers that utilize generative AI and machine learning technologies, integrated with eye-tracking sensors and display devices in the home, workplace, and mobile.
For realistic video experiences in XR services, it is crucial to achieve a motion-to-photon (MTP) latency of less than 10 milliseconds. This latency refers to the amount of time between eye movement to scene display in the cloud (data center), significantly stricter than the current 100 ms delay allowed for 2D video services over 5G networks. Extending the capabilities of all-photonic network (APN) to mobile and home networks under the Innovative Optical & Wireless Network (IOWN) Global Forum is key to enabling high-speed, high-capacity, and low-latency connectivity, necessitating further advancements in power efficiency, resilience, and security. The realization of advanced 3D and XR video distribution technologies will not only open up business opportunities for the entertainment industry, but will also revolutionize medical and healthcare support, including remote surgery, smart city security management, traffic management, and remote industrial maintenance. This expansion is expected to significantly enhance societal functions and improve quality of life, having a profound impact on both society and individual lives.
Committing to being No. 1, even in a very specific area
My mission is to develop technologies for new applications that enhance safety, security, and comfort, aiming for a sustainable society. This drive shapes my work in video communications, where we aim to create products that enrich society and improve life quality, prioritizing customer and end-user needs over profit.
In video distribution, we have achieved world-first innovations, emphasizing the importance of aiming for excellence, regardless of the niche. Achieving number one status, even in a small field, broadens our perspectives and sets the stage for future advancements. Success requires time and persistence, often taking a decade to become an industry standard. Our progress relies heavily on effective collaboration with both internal and external partners, recognizing that these relationships are vital for overcoming challenges and achieving our goals.
