Author: Dr Ramez Barsoum, Resident Medical Officer, Princess Alexandra Hospital, Brisbane, QLD, Australia. DermNet NZ Editor in Chief: Adjunct Prof Amanda Oakley, Dermatologist, Hamilton, New Zealand. Copy edited by Gus Mitchell/Maria McGivern. September 2018.
A convolutional neural network (CNN) is a type of deep artificial neural network used in image processing . The network takes an input image and using a catalogue of existing images produces an output that matches the input image. A neural network draws on similarities from interconnected biological neurons and is structured in such a way to learn and improve on its performance, based on how many images the CNN ‘sees’ and the amount of convolutions (combined inputs of two images, a new one and an existing catalogued image, to create a third output) the CNN generates.
CNNs are seen as a beneficial new tool for dermatologists to help better diagnose lesions. The work that a CNN does to produce a diagnostic result from an image is similar to how a dermatologist uses their training and knowledge: diagnosing lesions by a dermatologist generally involves an input image (of a cutaneous lesion) being fed through a processing network (the skills and knowledge of the dermatologist who analyses it and synthesises available information) to output a ‘class’ (or diagnosis) or a ‘probability of classes’ (differential diagnosis) .
The visual nature of dermatology lends itself well to digital lesion imaging, and CNNs have huge potential to change practice. It is a multifaceted means of analysing data that involves complex mathematics and requires massive computational power to combine biology, mathematics and computer science.
CNNs have been used in military and civil applications, including unmanned aerial vehicles, the technology sector, and commerce . They are found in everyday applications such as social media platforms that automatically recognise faces, in self-tagging photo galleries, and on shopping websites that come up with suggestions based on your internet browsing habits.
In the medical field, researchers have been using CNNs to diagnose diabetic eye disease, arrhythmias, and skin cancers [3–5].
The basis of a CNN is a computer method that is able to differentiate between different image classes based on unique features that can reliably be used to identify the image class, such as edges and curves. This base is then expanded upon with more abstract features being added through a series of convolutional pooling and output layers.
An image is fed into a computer and processed as different arrangements of pixels (dots), based on their colour. This process converts sections of the image according to a particular filter . The filters often start off analysing simple features like straight lines, diagonal lines, curved lines, or dots. Every time a filter is passed over the original image, it creates a new, smaller version of the original photograph. The positive filter matches are assigned a positive value, and the areas that do not match are assigned a value of less than 1. This results in a convoluted image; for example, a straight-line filter passed over an image of an acral naevus with a parallel furrow pattern on dermoscopy will show a strong positive convolution image. This step can be repeated with further features to achieve a more accurate output (ie, a diagnosis or differential diagnosis).
If the resulting image is large, the layers of the neural networks may need to be “pooled” between subsequent convolutions, fixing on an area of interest in the image and removing the parameters around that area . There are different types of pooling, but the most common type used is the maximum pooling method (see figure below).
There is often an additional normalisation step, which is a common technique for improving the performance and stability of a neural network. This works by standardising the inputs analysed by the neural network, in order to ensure each input has roughly the same scale. Thus, the neural network will not assign undue significance to one input filter over another simple due to a difference in scale. This process vastly increases the learning rate of a neural network.
To generate an output differential diagnosis for a suspected lesion, the neutral network needs to apply a fully connected layer based on all the layers it has previously processed. This is akin to a dermatologist synthesising the different clues into a provisional diagnosis with a set of differentials.
The CNN can now be trained using additional functions to improve accuracy and to "teach" itself to identify new lesions (such as back-propagation, which teaches the network when it selects a wrong outcome to change the weight assigned to traits when selecting an output class) .
The benefits of CNNs in the diagnosis of skin lesions include accuracy, speed, and low cost.
Caveats about the use of CNNs include the unrealistic expectations of patients and health practitioners, security and privacy issues, and medicolegal accountability.
See the DermNet NZ bookstore.
© 2019 DermNet New Zealand Trust.
DermNet NZ does not provide an online consultation service. If you have any concerns with your skin or its treatment, see a dermatologist for advice.