Provenance constrained temporal reconstruction of lens obscured images for intelligent vehicles
Keywords:
intelligent vehicles, automotive cameras, lens obstruction, temporal reconstruction, pixel provenance, motion compensation, selective reconstruction, driver assistanceAbstract
Earlier camera frames can help recover a view obscured by lens contamination. However, a genuine historical pixel may show an outdated object position. This paper presents a temporal reconstruction rule that accepts only recent, unobstructed source pixels, requires agreement between two historical samples, and excludes potentially dynamic regions. Rejected pixels remain unknown, and reconstructed images never enter the source history. A controlled experiment with computer-generated road sequences tests three motion levels, three obstruction sizes, three alignment biases, and four dynamic-mask retention settings. With exact background alignment and complete dynamic exclusion, the rule reconstructs 9.72% of obscured pixels with zero error on the accepted subset. Clean-source warping reconstructs 63.31%, with a mean absolute RGB error of 0.0596. With a two-pixel alignment bias, constrained coverage rises to 13.79%, but 2.91% of accepted pixels have an error above 0.10. Removing dynamic exclusion also allows outdated pedestrian evidence to pass. The results show the trade-off between recovery coverage and error. Tracking pixel origin alone cannot establish that reused content matches the current scene. Evaluation on recorded driving sequences and downstream perception tasks is still needed
References
Yogamani, S., Hughes, C., Horgan, J., Sistu, G., Varley, P., O’Dea, D., Uricar, M., Milz, S., Simon, M., Amende, K., Witt, C., Rashed, H., Chennupati, S., Nayak, S., Mansoor, S., Perrotton, X., & Perez, P. (2019). WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 9308–9318). IEEE. DOI: 10.1109/ICCV.2019.00940
Uricar, M., Krizek, P., Sistu, G., & Yogamani, S. (2019). SoilingNet: Soiling detection on automotive surround-view cameras. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC) (pp. 67–72). IEEE. DOI: 10.1109/ITSC.2019.8917178
Teed, Z., & Deng, J. (2020). RAFT: Recurrent all-pairs field transforms for optical flow. In Computer Vision – ECCV 2020 (pp. 402–419). Springer. DOI: 10.1007/978-3-030-58536-5_24.
Meister, S., Hur, J., & Roth, S. (2018). UnFlow: Unsupervised learning of optical flow with a bidirectional census loss. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). DOI: 10.1609/aaai.v32i1.12276.
Li, Z., Lu, C.-Z., Qin, J., Guo, C.-L., & Cheng, M.-M. (2022). Towards an end-to-end framework for flow-guided video inpainting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 17541–17550. DOI: 10.1109/CVPR52688.2022.01704.
Zhou, S., Li, C., Chan, K. C. K., & Loy, C. C. (2023). ProPainter: Improving propagation and transformer for video inpainting. Proceedings of the IEEE/CVF International Conference on Computer Vision, 10443–10452. DOI: 10.1109/ICCV51070.2023.00961.
Geifman, Y., & El-Yaniv, R. (2019). SelectiveNet: A deep neural network with an integrated reject option. In K. Chaudhuri & R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97, pp. 2151–2159). PMLR. https://proceedings.mlr.press/v97/geifman19a.html
International Organization for Standardization. (2022). Road vehicles—Safety of the intended functionality (ISO Standard No. 21448:2022). https://www.iso.org/standard/77490.html
Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3354–3361. DOI: 10.1109/CVPR.2012.6248074.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
