AI-Crash-Course
Intro
Meta Talk
- Evolutionary AI guy
- Backprop ES/HyperNEAT lover
- "Ghetto AI" creator
- lychee.js Engine
- artificial.engineering
Disclaimer
- I'm an engineering guy
- I say fuck a lot, because shit's on fire, yo.
- I hate Maths. No, srsly. It's wrong.
Genetic Programming Basics
Genetic Programming
- Idea is to represent states as DNA
- DNA can be combined with others
- Imagine DNA as "Arrays of values"
Neural Networks
DNA Serialization
Genetic Programming
- 2 babies are produced, a son and a daughter
- 2 babies have each more genes of mother or father
Breeding
Breeding
- Technically it's ZW + ZZ genome crossover
- DNA split is randomized
- DNA split divides characteristics
Genetic Programming
- DNA is Array of neuron weights
- Breeding allows inheriting characteristics
- Breeding allows inheriting behaviours
Evolution Basics
Evolution Basics
- Evolution has three phases
- Phase 1 is Training
- Phase 2 is Evaluation
- Phase 3 is Breeding
Evolution Cycle / Epoche
Evolution Basics
- After each Epoche / Cycle:
- Population gets killed and replaced
- Some of them are Mutants (20%)
- Some of them are Survivors (20%)
- The best ones get to breed (60%)
Evolution Basics
- Multi-Agent System
- Competitive Environment
- Agent gets rewarded or punished
- Agent has fitness as status
- Agents with better fitness get to multiply
Live Demo
Game Demo (Manual)
Simulation Architecture
- Evolution has an interval / timeout
- Population is an Array of Agents
- Each Generation represents one Evolution Cycle
Agent Architecture
- Each Agent has a brain (FFNN)
- Each Agent controls an Entity
- Each Agent has a fitness property
Brain Architecture
- Sensor 1: Center (position.y) of Plane
- Sensor 2: Center (position.y) of next Goal
- Control 1: calls method Plane.flap()
Brain Architecture
- 2 Input Neurons
- 1 Hidden Layer (with 2 Neurons)
- 1 Output Neuron
Brain Architecture
Live Demo
Game Demo (Autopilot)
Problems with Evolution
- Mutation Rate good at start
- Mutation Rate bad at end
Problems with Breeding
- No Behavioural Analysis
- No Tracking of DNA History
- No Tracking of Performance
Neural Network Basics
Neural Network Basics
- Networks have layers
- Layers have neurons
- Neuron connections are weighted
- Weight represents "relatedness"
Neural Network Basics
Programmer's Perspective
- Inputs are variables
- Neuron connections are conditions
- Outputs are returned values
Decision Trees
- Neural Networks are decision trees
if/elseis a one-way connectionif/elseifis a two-way connection
Decision Trees
if/else example:
if (a === true) {
if (b === false && a === true) {
return true;
}
}
return false;Decision Trees
if (a === true) {
if (b === false) {
return true;
}
}
return false;Decision Trees
Decision Trees
Decision Trees
Wait a sec
Wait a second... what about vectors and stuff?
Vectorization
if (whatever < 0.5) {
// whatever is false-ish
} else if (whatever > 0.5) {
// whatever is true-ish
}Vectorization
- You can convert pretty much anything
- enums parameters, arguments, strings
Vectorization
if (a > 0.1 && a < 0.2) {
// dict entry #1
} else if (a > 0.2 && a < 0.3) {
// dict entry #2
} else {
// etc ...
}Vectorization
- Use the power of dictionaries!
- Get better results with offsets (thresholds)
- Dictionaries can be external (A)NNs, too.
Backpropagation Basics
Backpropagation Basics
- Neuron weights are randomized
- Error function tweaks weights
- Weights are matched to expected outputs
Backpropagation Basics
Backpropagation Basics
- Pong Game Example
- X/Y/Z of ball position
- X/Y/Z of paddle position
- output > 0.5 to move upwards
- output < 0.5 to move downwards
Pong Demo
Game Demo (autobattle)
Pong Network
Pong Training
- Backpropagation will lead to reinforced weights
- Error function will match weights closer to expected values
Pong Network
Pong Network
Backpropagation Basics
- All Neurons are already given and static
- Network only adapts in weights, NOT in connections
- Reinforcement Learning is an ANALYTICAL model
Backpropagation Basics
- Gradient calculation for each step is inefficient
- Gradient boosting is extrapolated and sub-optimal
Problems with Reinforcement
- No enemy means no improvement
- No failure means no improvement
- Always tries to "hack" rewards
- (Pong Game: Play coop for more rewards)
Reinforcement Learning
- Analytical model
- NOT a generational model
Adaptive Neural Networks
Adaptive Neural Networks
Adaptive Neural Networks
- Randomized Neurons, weights and connections
- Randomization always wins
- Most ANN algorithms use evolution
ANN Algorithms
- unsupervised evolution cycles
- relative progress measurement
- track performance ("fitness")
- track behaviour ("species")
NEAT
NEAT
- Neuro Evolution of Augmenting Topologies
- Genetic Algorithm that generates ANNs
- tracks best fitness of evolved solutions
- uses CPPN (Composition Pattern Producing Network)
NEAT
NEAT History
- can predict dominant genes
- can predict dominant species
- can predict where to spawn neurons
- respects diversity with gene history markers
- tracks crossover history and -speciation
NEAT Agents
- evolutionary simulation
- tracks agent's fitness
- tracks agent's dominance
- tracks agent's history
NEAT Agents
Advantages of NEAT
- Adaptive and unsupervised
- Mutates connections between neurons
- Mutates nodes (neurons) in locations
- History markers for better prediction
MarI/O Demo
MarI/O Demo (by Sethbling)
Backprop NEAT
Backprop NEAT
- ANNs can be backpropagated, too
- Backpropagation will help determine necessary neurons
- Solves evolution's mutation-rate problem
Backprop NEAT
Backprop NEAT Demo (by hardmaru)
HyperNEAT
HyperNEAT
- Uses HyperCube encoding for inputs
- Uses HyperCube encoding for neuron connections
- allows auto-detecting geometric relations
HyperNEAT
HyperNEAT Demo
HyperNEAT Racers Demo
HyperNEAT
- CPPN now also tracks speciation
- CPPN is reinforced
- CPPN sees and predicts relations of inputs to outputs
Papers
References
The End
Questions and Answers