13ª Semana Integrada do Instituto de Física de São Carlos

A computational approach for image-based monitoring and morphological characterization of skin lesions

21 de ago. de 2023 14:00

1h 30m

Salão de Eventos USP

Normal 14h00 - 15h30

Descrição

Measuring and quantifying the progression or treatment of wounds, scars, and tumors constitutes a fundamental aspect of our research group projects. The accuracy of such assessments can be compromised due to the irregularity and dimensions of the lesions, as well as the variability introduced by operators during measurements. To address these challenges, we have developed two image processing methods using Python that utilize images captured by webcams or cellphones. The first method involves the creation of depth maps, enabling 3D object reconstructions using point clouds. Additionally, five image processing methods based on RGB data were developed to facilitate automatic, practical, and observer-independent visualization of lesions. Camera distortions, such as pincushion and barrel distortions resulting from the lens, are prevalent. To rectify these distortions, we have implemented a script using OpenCV that can correct the captured images. Upon undistorting the images, we utilize Epipolar Geometry to calculate the depth of each pixel in the 2D image by recognizing corresponding real points in both image sources. (1) Two depth calculation methods have been developed based on the results obtained from Epipolar Geometry. The first method is a stereo setup that directly implements a stereo system, while the second method (the primary one) employs a single image (Mono Imaging) and utilizes a neural network to displace the image and treat it as the second image for depth calculation. The advantages of utilizing artificial intelligence (AI) for depth calculation include faster processing, improved resource efficiency, and enhanced precision by reducing the likelihood of human error. However, the stereo method is still under development for scenarios where the AI may be deceived, such as cases involving reflections. obtained a depth map, we can proceed with 3D reconstruction using point clouds, enabling visualization in three dimensions. We are currently working on converting point clouds to meshes, which will facilitate volume calculations and use in Monte Carlo simulations—an area of growing interest within our research group. Furthermore, we have developed five methods that leverage RGB channels to enhance lesion visualization. (2) These methods aim to highlight the respective areas, mitigating user-dependent subjectivity and enabling a more standardized approach. The primary focus of this software is to provide a user-friendly and accessible tool that establishes standards within our research group. Moreover, the software is open-source, allowing users to contribute improvements tailored to their specific needs. Importantly, our software is independent of third-party limitations, ensuring unhindered usage and adaptability to evolving research requirements. In the next steps, we intend to continue the improvement of the capabilities and functionality of the software, further advancing its performance in image analysis and visualization, as well as adding the ability to generate 3D meshes via Point Clouds and a Graphic User Interface (GUI) for easier usage. Lesions from patients and in vivo experimental models will be used to validate the developed methods. (3)

Referências

1 JAIN, R.; KASTURI, R.; SCHUNCK, B. G. Machine vision. New York: McGraw-Hill, 1995. 549 p.

2 SAKNITE, I. et al. Comparison of single-spot technique and RGB imaging for erythema index estimation. Physiological Measurement, v. 37, n. 3, p. 333-346, 2016. DOI: http://dx.doi.org/10.1088/0967-3334/37/3/333.

3 HANOCKA, R. et al. Point2mesh: a self-prior for deformable meshes. ACM Transactions on Graphics, v. 39, n. 4, p. 126-1-126-12, July 2020. DOI: http://dx.doi.org/10.1145/3386569.3392415.