基于普通器件实现快1000倍的相机与机器视觉

黄铁军 , 郑雅菁 , 余肇飞 , 陈瑞 , 李源 , 熊瑞勤 , 马雷 , 赵君伟 , 董思维 , 朱林 , 李家宁 , 贾杉杉 , 付溢华 , 施柏鑫 , 吴思 , 田永鸿

工程(英文) ›› 2023, Vol. 25 ›› Issue (6) : 110 -119.

PDF (3049KB)
工程(英文) ›› 2023, Vol. 25 ›› Issue (6) : 110 -119. DOI: 10.1016/j.eng.2022.01.012
研究论文

基于普通器件实现快1000倍的相机与机器视觉

作者信息 +

1000× Faster Camera and Machine Vision with Ordinary Devices

Author information +
文章历史 +
PDF (3121K)

摘要

在数码相机中,我们发现了一个重大缺陷,即从胶片相机继承的图像和视频模型阻碍了相机捕捉快速变化的光子世界。我们提出了一种新的视觉形式,称为视象(vform),这是一个比特序列阵列,其中每个比特表示光子的累积是否达到了一个阈值,从而可以记录和重建任何时刻场景的光强。仅使用消费级CMOS(互补金属氧化物半导体器件)传感器和集成电路,开发了一种比传统相机快1000 倍的脉冲相机。将视象看作生物视觉中的脉冲序列,进一步开发了基于脉冲神经网络的机器视觉系统,它可以将机器的速度和生物视觉的机理结合起来,从而实现了比人类视觉快1000 倍的高速目标检测和跟踪,并通过辅助裁判和目标瞄准系统证明了脉冲相机和超级视觉系统的效用。视象模型和芯片有望从根本上改变图像和视频的概念以及摄影、电影和视觉媒体等相关行业,并开启一个全新的基于脉冲神经网络的速度自由的机器视觉时代。

Abstract

In digital cameras, we find a major limitation: the image and video form inherited from a film camera obstructs it from capturing the rapidly changing photonic world. Here, we present vidar, a bit sequence array where each bit represents whether the accumulation of photons has reached a threshold, to record and reconstruct the scene radiance at any moment. By employing only consumer-level complementary metal-oxide-semiconductor (CMOS) sensors and integrated circuits, we have developed a vidar camera that is 1000× faster than conventional cameras. By treating vidar as spike trains in biological vision, we have further developed a spiking neural network (SNN)-based machine vision system that combines the speed of the machine and the mechanism of biological vision, achieving high-speed object detection and tracking 1000× faster than human vision. We demonstrate the utility of the vidar camera and the super vision system in an assistant referee and target pointing system. Our study is expected to fundamentally revolutionize the image and video concepts and related industries, including photography, movies, and visual media, and to unseal a new SNN-enabled speed-free machine vision era.

关键词

Vidar相机 / 脉冲神经网络 / 超级视觉系统 / 全时成像

Key words

Vidar camera / Spiking neural networks / Super vision system / Full-time imaging

引用本文

引用格式 ▾
黄铁军,郑雅菁,余肇飞,陈瑞,李源,熊瑞勤,马雷,赵君伟,董思维,朱林,李家宁,贾杉杉,付溢华,施柏鑫,吴思,田永鸿. 基于普通器件实现快1000倍的相机与机器视觉[J]. 工程(英文), 2023, 25(6): 110-119 DOI:10.1016/j.eng.2022.01.012

登录浏览全文

4963

注册一个新账户 忘记密码

参考文献

基金资助

()

AI Summary AI Mindmap
PDF (3049KB)

2133

访问

0

被引

详细

导航
相关文章

AI思维导图

/