keras Classifying Spatiotemporal Inputs with CNNs, RNNs, and MLPs
Introduction
Spatiotemporal data, or data with spatial and temporal qualities, are a common occurrence. Examples include videos, as well as sequences of image-like data, such as spectrograms.
Convolutional Neural Networks (CNNs) are particularly suited for finding spatial patterns. Recurrent Neural Networks (RNNs), on the other hand, are particularly suited for finding temporal patterns. These two, in combination with Multilayer Perceptrons, can be effective for classifying spatiotemporal inputs.
Remarks
In this example, a VGG-16 model pre-trained on the ImageNet database was used. If a trainable VGG-16 model is desired, set the VGG-16 weights parameter to None for random initialization and set the cnn.trainable attribute to True.
The number and kind of layers, units, and other parameters should be tweaked as necessary for specific application needs.
