本文为影眸科技创始人，上海科技大学MARS实验室学生负责人吴迪于「驱动未来的AI技术」系列主题，「元宇宙」专场的分享内容节选。点击进入「机器之心Pro」，查看更多优质内容。
This is U-Di, founder of Shadow Science and Technology, student director of the MARS Laboratory at Shanghai University of Technology, on the theme of "A technology for the future", an excerpt of the shared content from the WonCosmos field. Click into , "The Heart of Machines Pro" , for more quality content.
元宇宙虚拟数字人的构建与未来身份系统
Virtual Digital Man Build and Future Identity System
吴迪的分享主要包括两部分：
U-D's share consists mainly of two parts:
构建 - 如何生成自己的数字角色；
交互 - 如何去和别人在元宇宙中通过表情、动作进行交互。
- How to interact with other people in the meta-cosmos through expression, movement.
吴迪表示，在未来的元宇宙体验里，希望用户简单地上传自己的几张照片，然后上传到待生成虚拟形象的神经网络，就可以生成自己的超高精度的三维虚拟形象，并且可以接入到不同的元宇宙平台和他人去进行交互。
Udy said that in the future metacosystem experience, it was hoped that users would simply upload their photos and then upload them to the neural network that was waiting to create virtual images, creating their own super-precision three-dimensional virtual images and having access to different metacosystem platforms to interact with others.
为达成上述目标，首先要有超高精度的人脸重建技术，影眸科技所使用的是光度立体法。该方法通过不同的光照分布，不同偏正态的光打在物体的表面去获取，通过光打在物体表面反射方向，来达到物体上细节的展现。该技术最早被用在南加大好莱坞去拍摄一些电影，如《双子杀手》。后来，南加大公司被谷歌收购了。所以Light Stage技术只有谷歌和影眸科技可以搭建。
In order to achieve these objectives, the first thing to be done is a super-precision human face reconstruction technology, which uses the light-based stereoscopic method. The method is obtained on the surface of the object through different light distributions, with different light-forms, and the light in the direction of reflection on the surface of the object, in order to achieve the details of the object. The technology was first used to make films such as " Double Killers ", which was later acquired by Google. So LightStage technology can only be built by Google and Shadow Technology.
此外，影眸科技自主研发了世界最高精度的人脸扫描设备Plennoptic stage ，其由很多灯光和高速相机组成，它的灯光在拍摄时会产生不同的变化，通过不同变化的光照打在人脸上的反射信息，可以捕捉到高精度的人脸材质。我们模特通过此设备里扫描时，它的光照也会变化。这是一个主动变化光照的扫描系统，每一帧拍在人身上光照都不一样。同时它可以达到 1000 赫兹的扫描，所以可以做4D高精度的动态连续扫描。
In addition, image technology has developed the world's highest-precision human face scan device, Plennoptic stage, which consists of a lot of lights and high-speed cameras, which can change the way they are taken, and can capture high-precision human face through a different light that can reflect on a person's face. When we scan this device, the light changes. It's a proactive photo-scan system, each of which is different from the human body. At the same time, it can reach 1,000 Hertz, so it can do a four-D high-precision dynamic continuous scan.
分享过程中，吴迪还展示了Plennoptic stage设备的宣传片，可以看到该设备可以做连续动态扫描，被拉扯的细节，包括毛孔都被一帧一帧清晰地表现出来。
In the process of sharing, U-Di also displayed a promotional film on the Plennoptic stage device, which can be viewed as a continuous dynamic scan, and the details of being pulled, including the maggots, are displayed in a single frame.
吴迪表示，通过进一步对Plennoptic stage采集到的数据进行自动化的重拓扑，然后再把它的图像和它所捕捉到的模型给输入到神经网络里面，就可以做一个基于 AI 的人脸控制和驱动。因为Plennoptic stage所有的帧都是以 20 FPS 去连续录制的毛孔细节，所以把贴图模型交给神经网络，神经网络就可以学到如何通过视频去表达出动态的贴图，生成动态细节，达到非常真实的实时人脸表现。这样，就可以通过单个摄像头让用户来驱动他的三维角色。
U-Dy said that by automating further the data collected by Plennoptic stage and then entering its images and the models it captures into the neural network, it could be an AI-based human face control and driver. Because all the frames in Plennoptic stage are detailed in 20 FPS to be recorded on a continuous basis, leaving the pixmap model to the neural network, where the neural network learns how to express dynamic stickers via video, generate dynamic details, and achieve very real real-time human face performance. In this way, the user can drive his three-dimensional role through a single camera.
吴迪说，可以想象一下，未来，来自两个不同地方在同一个元宇宙的世界里的人，通过 VR 眼镜去得到他们的面部表情和动作，两个人就可以通过自己三维角色，在同一个空间里面做非常精细的表情或者口型交互，这是真正元宇宙未来应该做到的事情。
U-Dy said that in the future, if you can imagine, people from two different places in the same meta-cosm, who get their facial expressions and movements through VR glasses, two people can do very fine expressions or verbal interactions in the same space through their three-dimensional roles, which is what the real meta-cosm should do in the future.
对于如何构建元宇宙里的人身构建，吴迪分享了影眸科技所搭建的一套全世界最大的光场采集系统ChallenCapSystem。ChallenCap System里共有24,000 多个灯光节点，由六种原色组成，均为彩色可控；它以高速度在人身上打出不同的光照，然后去构建出人身上的材质和三维模型。这是全世界唯一的一套达到 8 米的三维采集设备。它所有的灯光节点都可控制，未来上面还会装上非常多的相机。就可以达到一个高精度的三维采集。通过这样一套设备，用户采集到人物的模型包括他的一些动作数据之后，后续通过几台手机甚至单个手机就可以去实现对人物进行动作捕捉，包括手指骨头都可以在一个手机的工单上实现。这也是MARS实验室所孵化的另外一个创业项目。
With regard to how to construct the body in the metaspace, Udy shares the world's largest optical collection system, ChallengeCapSystem. There are more than 24,000 light nodes in ChallengeCap System, made up of six original colours, all colorable; it takes a different light into a person at a high speed, and then builds a three-dimensional model of the body. This is the only three-dimensional collection facility in the world that reaches 8 metres. All of its light nodes can be controlled, and there will be a very large number of cameras on it in the future. A high-precision three-dimensional collection can be achieved.
此外，吴迪对元宇宙中人物的身份进行了探讨。吴迪将其分为三类，分别是IP 型数字人、服务型的数字人、ID 型数字人。吴迪说道，元宇宙中非常重要的是人物风格化，这需要采集到高精度模型，让它适配元宇宙里所有的角色风格比如说它想去迪士尼风格平台里玩，就可以以自己的形象和结构为基础，把自己变成迪士尼的王子和公主，和他人进行交互。
In addition, U-Dy discussed the identity of the characters in the Woncosm. U-Dy divided them into three categories, IP-type digitals, service-type digitals, and ID-type digitals. U-Dy said that what is very important in the Woncosm was the style of the characters, which requires a high-precision model that matches all the characters in the Woncosm, for example, to play in a Disney-style platform, so that he can interact with others on the basis of his image and structure, turning himself into a Disney prince and princess.
最后，吴迪表示，元宇宙除了构建、交互，还有渲染环节，这也是MARS实验室包括未来上海科技大学重点研究的方向。
In conclusion, Udi stated that the Woncosmos, in addition to building and interacting with each other, has a resonance link, which is the direction of the MARS laboratory, including the future focus of the Shanghai University of Science and Technology.
当前影眸科技达到的效果是依靠所捕捉到高精度三维重打光驱动数据库，然后把它添加到网络 SOFGAN 里，通过形态、风格去生成一个人脸。而对于未来如何让终端去运行渲染效果，吴迪分享了影眸科技目前在研发一个称作GGU的处理单元。GGU（GraphicsGeneration Unit）是一个AI图像生成处理单元，把模型的参数和视角输入到网络之后，网络会自动地输出视角下的物品、人、场景。这样的渲染方法我们称作为 NeRF 或者叫 neural rendering 。通过GGU，终端、 AR 眼镜、手机就可以基于 NLP 网络算法实时做 NeRF 和 neural rendering 高清晰度实时渲染。
The effect of the current image technology is to capture a high-precision three-dimensional light-drive database and then add it to the network SOFGAN to generate a face through form and style. For the future, U-Dy shares a treatment unit called GGU. The GGU (GraphicsGeneration Unit) is an AI image-generation processing unit that automatically produces objects, people, scenes under a view, as a NeRF, or called Neural Rendering. Through GGU, terminals, AR, mobile phones can be made real-time NeRF and & nbsp; high-resolution redding & nbsp;
注册有任何问题请添加 微信：MVIP619 拉你进入群
打开微信扫一扫
添加客服
进入交流群
发表评论