A flexible framework of neural networks