Most Popular
1500 questions
11
votes
1 answer
What is the relationship between gradient accumulation and batch size?
I am currently training some models using gradient accumulation since the model batches do not fit in GPU memory. Since I am using gradient accumulation, I had to tweak the training configuration a bit. There are two parameters that I tweaked: the…
JVGD
- 1,108
- 1
- 6
- 14
11
votes
1 answer
What is the intuition behind the attention mechanism?
Attention idea is one of the most influential ideas in deep learning. The main idea behind attention technique is that it allows the decoder to "look back” at the complete input and extracts significant information that is useful in decoding.
I am…
Pluviophile
- 1,263
- 6
- 19
- 39
11
votes
1 answer
Are Q-learning and SARSA the same when action selection is greedy?
I'm currently studying reinforcement learning and I'm having difficulties with question 6.12 in Sutton and Barto's book.
Suppose action selection is greedy. Is Q-learning then exactly the same algorithm as SARSA? Will they make exactly the same…
hyuj
- 131
- 4
11
votes
2 answers
Do deep learning algorithms represent ensemble-based methods?
According to the Wikipedia article on deep learning:
Deep learning is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of…
Erba Aitbayev
- 357
- 1
- 10
11
votes
3 answers
Is there a strong argument that survival instinct is a prerequisite for creating an AGI?
This question stems from quite a few "informal" sources. Movies like 2001, A Space Odyssey and Ex Machina; books like Destination Void (Frank Herbert), and others suggest that general intelligence wants to survive, and even learn the importance for…
Eric Platon
- 1,510
- 10
- 22
11
votes
2 answers
If IQ is used as a measure of intelligence in humans, could it also be used as a measure of intelligence in machines?
If IQ were used as a measure of the intelligence of machines, as in humans, at this point in time, what would be the IQ of our most intelligent AI systems?
If not IQ, then how best to compare our intelligence to a machine, or one machine to…
D. Wade
- 541
- 2
- 7
11
votes
8 answers
Why is search important in AI?
Why is search important in AI? What kinds of search algorithms are used in AI? How do they improve the result of an AI?
Zoltán Schmidt
- 633
- 7
- 14
11
votes
4 answers
Why would an AI need to 'wipe out the human race'?
I'm reading such nonsense about how an AI would turn the world into a supercomputer to solve a problem that it thought it needed to solve. That wouldn't be AI. That's procedural programming stuck in some loop nonsense. An AI would need to evolve and…
user3573987
- 191
- 7
11
votes
3 answers
Are there any rules of thumb for having some idea of what capacity a neural network needs to have for a given problem?
To give an example. Let's just consider the MNIST dataset of handwritten digits. Here are some things which might have an impact on the optimum model capacity:
There are 10 output classes
The inputs are 28x28 grayscale pixels (I think this…
Alexander Soare
- 1,339
- 2
- 11
- 27
11
votes
1 answer
Monte Carlo Tree Search: What kind of moves can easily be found and what kinds make trouble?
I want to start with a scenario that got me thinking about how well MCTS can perform:
Let's assume there is a move that is not yet added to the search tree. It is some layers/moves too deep. But if we play this move the game is basically won.…
Nocta
- 111
- 2
11
votes
3 answers
Who was the first person to recognize the distinction between human-like general intelligence and domain-specific intelligence?
In the 1950s, there were widely-held beliefs that "Artificial Intelligence" will quickly become both self-conscious and smart-enough to win chess with humans. Various people suggested time frames of e.g. 10 years (see Olazaran's "Official History of…
liori
- 513
- 2
- 9
11
votes
1 answer
What regulations are already in place regarding artificial general intelligences?
What regulations are already in place regarding artificial general intelligences? What reports or recommendations prepared by official government authorities were already published?
So far, I know of Sir David King's report done for UK government.
liori
- 513
- 2
- 9
11
votes
1 answer
How does the forget layer of an LSTM work?
Can someone explain the mathematical intuition behind the forget layer of an LSTM?
So as far as I understand it, the cell state is essentially long term memory embedding (correct me if I'm wrong), but I'm also assuming it's a matrix. Then the…
user8714896
- 797
- 1
- 6
- 24
11
votes
2 answers
Is there any existing attempt to create a deep learning model which extracts vector paths from bitmaps?
I need an algorithm to trace simple bitmaps, which only contain paths with a given stroke width.
Is there any existing attempt to create a deep learning model which extracts vector paths from bitmaps?
It is obviously very easy to generate bitmaps…
arthur.sw
- 161
- 1
- 8
11
votes
1 answer
What is the reason AMD Radeon is not widely used for machine learning and deep learning?
What is the reason AMD Radeon is not widely used for machine learning and deep learning? Is it mainly an issue of lack of software? Or is Radeon's GPU not as good as NVIDIA's?
noviceFedora
- 147
- 1
- 5