What are some recommended ways to tune hyperparameters and/or develop domain-specific architectures for a large neural network model? That is, how to further tune a large neural network that already takes a long time to train?
My question is two-fold (on hyperparameters and architecture):
The standard seems to be bayesian hyperparameter optimization but even so, this optimization can take a long time if one wishes to take advantage of the sequential optimization in this method (optimizing based on prior knowledge).
There is also evolutionary methods for developing novel architectures but how can the average data scientist execute this method while avoiding expensive computational resources? (I am not too familiar with evolutionary algorithms)
Are there any clever approaches/techniques to further tune hyperparameters/develop domain-specific architectures for neural networks that already take a long time to train?