Convolutional neural networks in dermatology

Author(s): Ramez Barsoum, Resident Medical Officer, Princess Alexandra Hospital, Brisbane, Australia. DermNet New Zealand Editor in Chief: Adjunct Prof Amanda Oakley, Dermatologist, Hamilton, New Zealand. Copy edited by Gus Mitchell. September 2018.


Convolutional neural networks in dermatology - codes and concepts
open

What is a convolutional neural network?

A convolutional neural network (CNN) is a type of deep artificial neural network used in image processing [1]. It is a multifaceted means of analysing data that involves complex mathematics and requires massive computational power to combine biology, mathematics and computer science. It draws on similarities from nature, where neurons are interconnected, capable of learning and improving on performance, based on how many lesions it ‘sees’.

The visual nature of dermatology lends itself well to digital lesion imaging and CNNs have huge potential to change practice. Diagnosing lesions by a dermatologist generally involves an input image (of a cutaneous lesion) being fed through a processing network (the skills and knowledge of the dermatologist who analyses it and synthesises available information) to output a ‘class’ (or diagnosis) or a ‘probability of classes’ (differential diagnosis) [2].

Who uses convolutional neural networks?

CNNs have been used in military and civil applications, for example, in unmanned aerial vehicles [3], in the technology sector, and in commerce. They are found in everyday applications such as social media platforms that automatically recognise faces, in self-tagging photo galleries, and on shopping websites that come up with suggestions based on your browsing habits.

In the medical field, researchers have been using CNNs to diagnose diabetic eye disease, arrhythmias, and skin cancers [3–5].

Tell me more about convolutional neural networks

The basis of a CNN is a computer method that is able to differentiate between different image classes based on unique features that can reliably be used to identify the image class, such as edges and curves. This is expanded upon with more abstract features through a series of convolutional layers.

Step 1: Convolutional layer(s)

An image is fed into a computer and processed as different arrangements of dots (pixels), based on their colour. This converts sections of the image according to a particular filter [2]. The filters often start off as simple features like straight lines, diagonal lines, curved lines, or dots. Every time a filter is passed over the original image, it creates a new, smaller version of the original photograph. The positive filter matches are assigned a positive value, and the areas that do not match are assigned a value less than 1. This results in a convoluted image: for example, a straight-line filter passed over an image of an acral naevus with parallel furrow pattern on dermoscopy will show a strong positive convolution image. This step can be repeated for more features to achieve a more accurate output.

Step 2: Pooling layer

If the resulting image is large, it may be required to “pool” layers between subsequent convolutions [2]. There are different types of pooling, but the most common type is the maximum pooling method (figure 1).

A simple “max pool” method of a 4x4 image into a pool of 2x2

There is often an additional normalisation step.

Step 3: Output layer

To generate an output differential diagnosis for a suspected lesion, the neural network needs to apply a fully connected layer based on all the layers it has previously processed. This is akin to a dermatologist synthesising the different clues into a provisional diagnosis with a set of differentials.

The CNN can now be trained using additional functions to improve accuracy and to “teach” itself to identify new lesions (such as back-propagation, which teaches the network when it selects a wrong outcome to change the weight assigned to traits when selecting an output class) [1].

What are the benefits of convolutional neural networks?

Benefits of CNNs in the diagnosis of skin lesions include accuracy, speed and low cost.

  • The clinical diagnostic accuracy for melanoma is dependent on the experience and training of the examining doctor; CNNs have been able to perform as well as board-certified dermatologists in limited circumstances and their accuracy will continue to improve in the future [6,7].
  • CNNs currently take seconds to minutes to arrive at a diagnosis when confronted with an image of a skin lesion. The inputs, algorithms and outputs can be undertaken outside office hours, and can be accessible to anyone with access to internet. Compare this short time with the wait and travel times associated with a dermatologist appointment, which is often several months or longer.
  • The algorithms can be adaptive and they can learn from adding new images over time.
  • CNNs are predicted to be able to diagnose lesions for a fraction of the cost of visit to a dermatologist.

What are the disadvantages of CNNs?

Caveats include unrealistic expectations of patients and health practitioners, security and privacy issues, and medicolegal accountability.

  • There is a substantial amount of excitement surrounding CNN technology, but the advantages will take time to materialise. Massive amounts of data and input are required to “train” CNNs. Humans are needed to choose which lesions should be imaged and examined by the CNN, and these health professionals will also need to be trained.
  • Tools offering diagnostic support will need to be officially approved as medical devices, and re-approved, as algorithms expand [8].
  • CNN networks will likely be entirely online using cloud-based storage; these need to have excellent cybersecurity systems to ensure backup in case of database or server failure, and to use authentication devices to prevent unauthorised access. Encryption and secure transfer protocols are required to store personal health data, and research must only use anonymised data.
  • Health professionals using CNNs will need to understand that performance using one dataset is not necessarily applicable to another one; there will be incorrect diagnoses including false positives (overdiagnosis of benign lesions as malignant) and false negatives (missed diagnoses of cancer).
  • Medicolegal accountability for the health professional relying on CNNs requires clarification, as there is no notable precedence. Can a computer algorithm be held accountable for a wrong diagnosis or a missed diagnosis?

Ask an online dermatologist now
Securely upload your symptoms and pictures using First Derm
(Sponsored content)

 

Related information

 

References

  1. Rawat W, Wang Z. Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review. Neural Comput 2017; 29(9): 2352–449. PubMed
  2. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542(7639): 115­–8. PubMed
  3. Carrio A, Sampedro S, Rodriguez-Ramos A, Campoy P. A Review of Deep Learning Methods and Applications for Unmanned Aerial Vehicles. Journal of Sensors, 2017. Volume 2017, Article ID 3296874, 13 pages, 2017. https://doi.org/10.1155/2017/3296874.
  4. Rajalakshmi R, Subashini R, Anjana RM, et al. Automated diabetic retinopathy detection in smartphone-based fundus photography using artificial intelligence. Eye (Lond) 2018; 32(6): 1138–44. Journal
  5. Roffman D, Hart G, Girardi M, et al. Predicting non-melanoma skin cancer via a multi-parameterized artificial neural network. Sci Rep 2018; 8(1): 1701. PubMed
  6. Haenssle HA, Fink C, Schneiderbauer R, et al. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol 2018; 29: 1836–42. PubMed
  7. Mar VJ, Soyer HP. Artificial intelligence for melanoma diagnosis: How can we deliver on the promise? Ann Oncol 2018. PubMed
  8. Digital Health Criteria. US Food & Drug Administration. Available at: https://www.fda.gov/medicaldevices/digitalhealth/ucm575766.htm (accessed 17 September 2018).

On DermNet NZ

Other websites

Books about skin diseases

See the DermNet NZ bookstore.