Choosing Between ReLu and Tanh for Neural Networks

Key Requirements:

1. Image Classification Task:
   • The task involves categorizing images of chairs into various types such as "Office Chair" and "Dining Chair."
2. Neural Network Design:
   • The network will likely involve multiple hidden layers, potentially using a convolutional neural network (CNN) architecture given the image classification context.
3. Activation Function Selection:
   • The choice is between ReLu and Tanh for the hidden layers of the neural network.
4. Performance Metrics:
   • The model should have high accuracy, fast convergence during training, and robustness to common neural network issues like vanishing gradients.
5. Consideration of Model Efficiency:
   • The model should be computationally efficient, both in terms of training and inference time.

Clarifying Questions:

• What is the expected size of the dataset?
• Are there any specific computational constraints or hardware limitations?
• Is there a preference for real-time inference or batch processing?
• Will the network be fine-tuned or trained from scratch?
• Are there any pre-existing models or frameworks in use that could influence activation function choice?