一种单视图江豚三维模型重建方法
A SINGLE-VIEW 3D MODEL RECONSTRUCTION METHOD FOR YANGTZE FINLESS PORPOISE
-
摘要: 在江豚三维重建领域, 存在水下图像色偏失真、江豚数据集不足、获取江豚多视角图像困难等问题, 而新兴方法尚未出现针对江豚的应用研究。为了解决这些难题, 本文提出了一种结合扩散模型和神经辐射场的单视图江豚三维模型重建方法。首先, 改进水下图像增强方法, 有效地解决水下图像色偏失真的问题。其次, 自制江豚多视角图像数据集, 微调训练视角条件扩散模型, 实现由单视图合成多视角图像, 为单张图像重建江豚提供了新思路。最后, 由神经辐射场进行重建, 得到江豚三维模型。对江豚三维重建的结果使用平均倒角距离和法向量一致性进行了对比评估, 平均倒角距离低于现有方法, 法向量一致性高于现有方法, 表明本文方法能够有效重建出符合江豚体色及形态的三维模型, 合成新视角图像PSNR、SSIM、LPIPS值分别为38.968、0.972、0.294, 效果优于现有方法, 经过水下图像增强的重建结果的平均倒角距离值最低为0.428, 法向量一致性最高达到0.882。Abstract: In the context of 3D reconstruction of finless porpoises, challenges such as color distortion in underwater images, the limited availability of finless porpoise datasets, and the difficulty in obtaining multi view images of finless porpoises. However, emerging methods have not yet been applied to finless porpoises. To address these challenges, we proposes a single view 3D model reconstruction method for finless porpoises that combines diffusion models and neural radiation fields. Firstly, an improved underwater image enhancement method has been developed, effectively addressing the issue of color cast and distortion in underwater images. Secondly, we created a custom multi view image dataset of finless porpoises and fine-tuned a perspective conditional diffusion model to achieve the synthesis of multi view images from a single view, providing a new approach for reconstructing finless porpoises from a single image. Finally, a 3D model of the finless porpoise was reconstructed using neural radiation field. The results of 3D reconstruction of finless porpoises were compared and evaluated using the average Chamfer distance and normal vector consistency. Our method achieved a lower average Chamfer distance and higher normal vector consistency compared to existing methods, indicating that it more accurately reconstructed the body color and morphology of finless porpoises. Moreover, the underwater image enhancement led to further reductions in average Chamfer distance and improvements in normal vector consistency.