Driven by a new round of revolution in information technology, a world for cyber-physical-social integration is rapidly forming. What kinds of cutting-edge technologies do we need to support the arrival of the intelligent era? Following the release of the Zhejiang Lab Dubhe Artificial Intelligence Open Source Platform and the Darwin Mouse brain-like computer, which host hundreds of millions of artificial neurons, Zhejiang Lab held a press conference on September 2 at the International Convention Center of the Future Sci-Tech City to announce its recent developments in areas such intelligent perception, intelligent networks, intelligent computing, and intelligent systems. A number of these research outcomes have been taken into the stage of commercial transformation.

Intelligent Healthcare, Intelligent Security Systems, and Intelligent Transportation

Machine intelligence is fully facilitating the development of the digital economy.

In order to enable pathology research on in vivo sub-cells and realize integrated diagnosis of multiple organs, Zhejiang Lab worked with leading research teams in China to develop the Ultra-fine, High-resolution, and Multi-directional Stereomicroscopic Endoscopic Imaging System. “We have enabled micron-scale imaging and achieved breakthroughs in critical technologies such as large-field confocal image stitching, multi-modal integrated packaging, high-precision 3D image reconstruction, and intelligent analysis. We have also created a clinical evaluation procedure based on microscopic endoscopies for pathology diagnosis of in vivo tissues, and developed key components with independent intellectual property rights owned by us,” said Professor Yang Qing at the press conference, who is head of the project team and Deputy Director at the Research Center for Smart Sensing. Currently, the prototype of the ultra-fine electronic endoscope has entered the stage of achievement transformation. In the future, this system will enable in vivo optical biopsies for multiple organs, filling a worldwide technological gap and providing a new technical strategy for the early diagnosis and surgical treatment of cancers across the digestive system.

The Multi-Center Intelligent Medical Information Platform is the first major project at the Zhejiang Lab Transformation Base for Digital Healthcare Achievements. "Our platform can identify useful information from the massive amount of medical data 'sleeping' in different medical institutions. It can thereby enable early screening of diseases such as kidney diseases, lung cancers, intestinal cancers, and pediatric ADHD, as well as support highly sophisticated medical research and clinical practices for these diseases,” said Professor Li Jinsong, Head of the project team and Director at the Research Center for Healthcare Big Data. So far, the platform has incorporated the medical data from multiple large A-level hospitals in Zhejiang, covering real clinical records of more than 14 million patients. It has also created a completed large-scale knowledge graph of electronic medical records, which ensures the sharing and in-depth utilization of multi-center clinical data without taking the original data away from medical institutions. Compared with similar foreign technologies, the homomorphic encryption method developed by the project team can enhance data security by 1024 times, and increase processing rate by more than ten times. Through demonstrative clinical application, the platform is ahead of similar technologies in the world in terms of overall technical levels, showing China’s ability to lead the development of medical information technologies.

How can we quickly analyze massive amounts of video data? Zhejiang Lab’s Distributed Intelligent Vision Platform, which is built upon the lab’s human feature description model and novel video analysis algorithm, can implement functions such as video enhancement, object tracking and detection, cross-camera object recognition, behavior analysis, and scenario understanding. “We make video analysis no longer reliant on facial features, suitable scenarios, and cooperative objects. Our platform attains nice recognition results in complex real-life scenarios such as bright illumination at night, dark illumination at night, crowds, rainy and foggy days, and severe noise pollution,” said Xu Xiaogang, Head of the project team. According to Xu, in terms of object detection, the platform has an accuracy of greater than 95% in scenarios with dark light and sharp contrast ratios. In terms of video-based behavior detection, it has an accuracy of greater than 90%. The two metrics both represent a leading level in China.

Apart from breakthroughs in intelligent vision, the platform also uses distributed terminal-edge-cloud computing to increase its computational speed. With highly real-time scheduling of video analysis tasks and efficient sharing of data resources on terminals, edges, and clouds, the platform can reduce its computational cost by more than 10 times. In addition, it is able to protect terminal data via a lightweight privacy protection algorithm for video information. According to Xu, the platform can also provide customized algorithm services. Relevant technologies have already been applied in areas such as safety monitoring in production and detection of product defects. In the future, these technologies will be further applied in scenarios such as safe driving and health condition monitoring.

In the area of driverless cars, the Zhejiang Lab project team, which works on an intelligent virtual testing and training platform for driverless cars, has developed an open and high-quality testing platform. Through digital twins and generalized algorithms, the platform provides extreme cases for high-value testing. Currently in China, it is one of the platforms with the most variety of extreme cases. With extremely dangerous cases rarely seen in real life, such as radar failures on rainy days, vehicle rollovers on the road, and collapsed trees on sidewalks, it can effectively improve self-driving algorithms’ road safety and intelligent driving capabilities.  

“We are working with Zhejiang Lab to process road data in an in-depth way and generate testing scenarios to support the development of a holographic scene library for intelligent driving. In addition, we use the lab’s virtual simulation and testing platform for intelligent driving software to register vehicles’ trajectories in 3D space and recreate corresponding 3D scenes and cases, so as to predict the behaviors of traffic participants and better facilitate the decision-making process of vehicles,” said Li Lin, Deputy Chief Engineer at the Shanghai International AutoCity Group Co., Ltd. According to Li, the company will deepen its cooperation with the lab to continuously improve the holographic scene library. The two sides will build an ecosystem for driverless cars, and promote the sharing of test data and platforms in the Yangtze River Delta. They will also work together to develop test case generation and 3D visualization tools for intelligent driving, establish industry standards, and explore the application of digital twins in the field of transportation.

6G Communications and Brain-Machine Integration

Cutting-edge technologies for the future are gradually becoming real.

As recognized by global research communities, Terahertz communication is one of the key technologies for 6G. To support 6G wireless connectivity of massive devices and ubiquitous intelligent high-rate wireless communication, Zhejiang Lab has launched a project to develop Tbit-scale optoelectronic terahertz communication devices and systems. “We have built an experimental verification system for ultra-high-rate optoelectronic terahertz communication. So far, we have achieved wireless terahertz transmission with a linear speed up to 600 gigabytes per second, and system metrics reaching world-leading levels,” said Senior Researcher Yu Xianbin, Head of the project team, “We will continue our research, and strive for breakthroughs in Tbit-scale terahertz communication. We will develop world-leading chips, devices, and platforms to provide vital technical support for ubiquitous, broadband, and intelligent 6G communication.”

With the arrival of the AI era, human-machine symbiosis has become possible. Brain-machine integration and brain-like computing have also been regarded as strategic areas in technological competition. Researchers therefore acknowledge the importance of building a research facility capable of providing real-time feedback of in vivo readout-input processes. The facility should integrate multimodal observation, and control approaches based on optical, electronic, magnetic, and acoustic data of the brain to present micron-scale and centimeter-scale brain activities taking place within microseconds or throughout the entire life cycle.

Focusing on this foundational research area, and aiming at problems such as homogenized brain observation modes, separated neural control approaches, and insufficient brain-machine integration devices, Zhejiang Lab has worked with Zhejiang University to develop a world-leading research facility for brain observation and brain-machine integration. “Running on 7T NMR equipment, the facility integrates optical, electronic, magnetic, and acoustic data of the brain to readout and control brain activities. As a multi-modal brain observation and control facility with 7T electromagnetic compatibility, it provides important support for us to further understand, analyze, simulate, and access human brains. It can facilitate research in multiple fields such as brain science, brain diseases, and artificial intelligence,” said Wang Yueming, a distinguished expert at Zhejiang Lab.

Mutual Development of Hard and Soft Science

Integrated research mechanism promotes all-link innovation.

In addition to concrete technological achievements, Zhejiang Lab has also released a series of research outcomes via white papers, including Thinking Critically and Acting Cautiously: the Development of AI Social Governance Experiments, White Paper on Endogenous Security Technologies for the Industrial Internet, and White Paper on the Vision and Technological Development of Intelligent Expressways.

In particular, Thinking Critically and Acting Cautiously: the Development of AI Social Governance Experiments is China’s first science book that explains AI social governance experiments to the public. It points out that AI social governance experiments are a series of investigations and studies on AI’s role in social governance. They reflect a positive attitude towards AI technologies that emphasizes critical thinking and cautious action. They also represent a comprehensive research topic and a systematic epistemological theory about the relationship between technology and society.

“As an unconventional technology that will change the future, AI brings new opportunities for us to modernize our social governance models and improve our governance capabilities. It also presents legislative, ethic, and moral challenges for social governance,” said Zhao Zhifeng, Director of the Department of Research Development at Zhejiang Lab, “We have established the Research Center for AI Social Experiment, which evaluates the potential social impact of AI technologies through long-term and wide AI social governance experiments in fields such as elderly care, environmental treatment, and education.” According to Zhao’s interpretation, this white paper systematically explains the significance and intension of AI social governance experiments. It describes the opportunities and challenges of employing AI technologies in social governance, shares a number of enlightening social governance cases, discuss key concerns about the role of AI social governance experiments in fields such as employment, education, healthcare, transportation, elderly care, and culture, explores approaches to implementing AI social governance experiments in these areas, and sets expectations for worldwide collaborative research of AI social governance.