Convolutional Deep Nets

• One of the other nets is a convlutional network.
• These take advantage of receptive fields (like natural vision systems).
• So, a hidden layer has a 2D topology, with inputs from one layer to the next being restricted by area.
• They also take advantage of pooling.
• This merely combines some inputs to reduce the data being procesed.
• The overall layers are feedforward, but they can break apart, and recombine.



• Another mechanism is attention which is used for text in systems like BERT.
• Finally, really big deep nets, with millions or even billions of parameters, are really expensive to train. Extra layers can be added to publicly available systems to specialise these systems.