Most Popular
1500 questions
7
votes
2 answers
Term for algorithms that are not trained
Before the advent of neural architectures, many AI domains (e.g. speech recognition and computer vision) used algorithms that consisted of a series of hand-crafted transformations for feature extraction.
In speech recognition everything to do with…
Mew
- 181
- 2
7
votes
1 answer
Is elitism preferred over non-elitism in the cross-over operator?
There are two potential approaches when performing cross-over operation in genetic algorithms.
Use only the elites in the pool, probably the ones that are also going to be directly transferred to the next generation.
Use all the population present…
Alireza
- 405
- 3
- 15
7
votes
2 answers
"Attention is all you need" paper : How are the Q, K, V values calculated?
The seminal Attention is all you need paper (Google Brain team, 2017) introduces Transformers and implements the attention mecanism with "queries, keys, values", in an analogy to a retrieval system.
I understand the whole process of multi-head…
Soltius
- 261
- 2
- 9
7
votes
2 answers
CNNs: What happens from one neuron volume to the next?
I've gone through several descriptions of CNNs online and they all leave out a crucial part as if it were trivial.
A "volume" of neurons consists of several parallel layers ("feature maps"), each the result of convolving with a different…
MackTuesday
- 181
- 6
7
votes
3 answers
Which predictive algorithm can be used to predict a number given other numbers?
I am currently searching for a supervised learning algorithm that can be used to predict the output given a large enough training set.
Here's a simple example. Suppose the training dataset is {[A=1, B=330, C=1358.238902], result=234.244378} and the…
Cryptonaut
- 81
- 3
7
votes
1 answer
A comprehensive list of machine learning algorithms
I could not find a comprehensive list of machine learning algorithms online.
Can we try to make one here and improve and update it as needed?
There are challenges in the algorithm definitions and how to group and organize them, but my preliminary…
Bernardo Costa
- 241
- 3
- 8
7
votes
3 answers
What are some academic AI podcast out there?
I am looking for AI podcasts that are purely academic-oriented that I can use for learning purposes. Thanks for any resource pointers.
The AI podcasts I am aware of are (not sure how many of these can be considered academic):
The AI Podcast
Linear…
Rajib Bahar
- 189
- 8
7
votes
2 answers
Why can't language models, like GPT-3, continuously learn once trained?
GPT-3 has a prompt limit of about ~2048 "tokens", which corresponds to about 4 characters in text. If my understanding is correct, a deep neural network is not learning after it is trained and is used to produce an output, and, as such, this…
MaiaVictor
- 365
- 3
- 10
7
votes
2 answers
What is a Markov chain and how can it be used in creating artificial intelligence?
I believe a Markov chain is a sequence of events where each subsequent event depends probabilistically on the current event. What are examples of the application of a Markov chain and can it be used to create artificial intelligence? Would a…
WilliamKF
- 2,513
- 1
- 25
- 31
7
votes
1 answer
Whys and Why-nots using Rust for AI
The title says it all. I would like to know more about what attributes and design choices of Rust that make it a good (or bad) language for the entire ecosystem of AI (both research and production)
What I know so far:
Rust is memory safe (unlike…
hongshan.li
- 128
- 1
- 6
7
votes
2 answers
Deep Learning with Best-so-far instead of Where-you-are
It is my understanding that when training a Deep NN in Tensorflow/PyTorch/... we only keep the current state of the network in memory, except perhaps when we manually decide to save the current weights to the HDD/SSD.
Now, naively speaking it may…
Stefan Perko
- 171
- 7
7
votes
2 answers
In Value Iteration, why can we initialize the value function arbitrarily?
I have not been able to find a good explanation of this, other than statements that the algorithm is guaranteed to converge with arbitrary choices for initial values in each state. Is this something to do with the Bellman optimality constraint…
Arham
- 73
- 3
7
votes
2 answers
Is there an argument against using the (reviewed) predictions of a model as ground truth to further train exactly this model?
I plan to use my predictions as ground truth to continue training my model. These predictions are of course reviewed during this process. Is there an argument against that (reinforcement of slight mistakes/overfitting etc.)?
Here my specific use…
thzu
- 73
- 4
7
votes
2 answers
What is the difference between a loss function and reward/penalty in Deep Reinforcement Learning?
In Deep Reinforcement Learning (DRL) I am having difficulties in understanding the difference between a Loss function, a reward/penalty and the integration of both in DRL.
Loss function: Given an output of the model and the ground truth,…
Theo Deep
- 185
- 1
- 5
7
votes
1 answer
What algorithms are used for image segmentation of images where objects are not composed of pixels that are similar in value?
In the process of segmentation, pixels are assigned to regions based on features that distinguish them from the rest of the image. Value Similarity and Spatial Proximity, for example, are two important principles that assume that points in the same…
user7369