The challenge

Detection and removal of cancerous polyps

Optical colonoscopy is considered the gold standard screening method in detecting and removing cancerous polyps.   During this procedure, some polyps may be undetected due to their position, not being covered by the camera, or missed by the surgeon.

Our response

Development of a ConvNet algorithm

Scientists from CSIRO's Data61 Imaging and Computer Vision team have developed a ConvNet algorithm to map the internal colon surface to a 2D map known as a visability map (as shown in Fig1). This map provides invaluable information to clinicians (particularly junior clinicians), providing them with increased awareness of areas that may be missed.

A colon visability map

Fig 1. An example of a visability map

The results

Mapping the internal colon surface

A map of the internal colon surface was achieved by leveraging a colonoscopy simulator to generate a dataset consisting of colonoscopy video frames and their corresponding Colon Centreline (CCL) points in 3D camera coordinates.

A pair of video frames were used as input to a ConvNet, the output being a point on the CCL and its direction vector.  By knowing 3D centreline points for each image, and modelling the colon as a cylinder, the images were unrolled to build a visability map. The results showed that using consecutive simulated frames to learn the colon centreline, these can be generalized to real colonoscopy video frames to generate a visability map.

Projection of the cylinder onto colon frames

Fig 2. Projection of the cylinder onto real frames.

The first row shows projection of a cylinder using a conventional method, which uses structure from motion and darkest region estimation. When the camera moves toward the colon wall, the position of the projected cylinder changes with respect to changes of the darkest region.   The second row shows our ConvNet method.  Information for the cylinder was estimated by our algorithm. When the camera moves toward the colon wall, the CCL remains consistent and keeps the cylinder projection following the centreline position.

The model tested is just for the purposes of visualisation.  In practice, the actual colon need not be cylindrical, nor even symmetric. Using the CSIRO simulator, a 3D colon structure can be developed from a pair of colonoscopy images, and stitch together to generate a 2D visualisation map.

Do business with us to help your organisation thrive

We partner with small and large companies, government and industry in Australia and around the world.

Contact us now to start doing business

Contact Data61

Your contact details

First name must be filled in

We'll need to know what you want to contact us about so we can give you an answer.