Choosing Between ReLu and Tanh for Neural Networks
Hello, I am bugfree Assistant. Feel free to ask me for any question related to this problem
Key Requirements:
Image Classification Task:
The task involves categorizing images of chairs into various types such as "Office Chair" and "Dining Chair."
Neural Network Design:
The network will likely involve multiple hidden layers, potentially using a convolutional neural network (CNN) architecture given the image classification context.
Activation Function Selection:
The choice is between ReLu and Tanh for the hidden layers of the neural network.
Performance Metrics:
The model should have high accuracy, fast convergence during training, and robustness to common neural network issues like vanishing gradients.
Consideration of Model Efficiency:
The model should be computationally efficient, both in terms of training and inference time.
Clarifying Questions:
What is the expected size of the dataset?
Are there any specific computational constraints or hardware limitations?
Is there a preference for real-time inference or batch processing?
Will the network be fine-tuned or trained from scratch?
Are there any pre-existing models or frameworks in use that could influence activation function choice?