Course: Modern Robotics

COSC 4560 / COSC 5560
M/W/F 10:00-10:50am
Room: Engineering 2108

Professor Nick Cheney
Office hours: by appointment (Engineering 4081B)

TA: Joost Huizinga
TA office hours: Tu/Th 2-4pm (Engineering 4086)

Syllabus (pdf)

Assigned Readings

Assigned readings should be completed before the class to which they are listed. "BIA 1-1.5" translates to:
In the Bio-Inspired Artificial Intelligence textbook, please read from the start of chapter 1 to the end of section 1.5)

Day Date Topic Assigned Reading Assignments Slides
M 1/22 Course Logistics -- pdf
W 1/24 Why Evolutionary Robotics? Evolutionary Robotics (Bongard 2013) (10 pages),
Why Evolutionary Robotics Will Matter (Stanley 2011) (3 pages)
F 1/26 Introduction to Artificial Evolution BIA 1-1.9 (32 pages) pdf
M 1/29 Types of Evolutionary Algorithms and Applications BIA 1.10-1.19.1 (34 pages) HW 1 Due 1/28 pdf
W 1/31 -- NO CLASS -- BIA 3-3.3 (25 pages) --
F 2/2 Artificial Neural Networks BIA 3.4-3.5, 3.10-3.10.1 (19 pages) pdf
M 2/5 Multi-Objective Evolutionary Optimization BIA 1.22-1.25 (14 pages) HW 2 Due 2/4 pdf
W 2/7 A Brief History of Artificial Intelligence and Embodied Cognition How the Body Shapes the Way We Think, Chap. 1 (Pfeifer, Bongard 2006) (19 pages) and Chap. 2 (31 pages) pdf
F 2/9 Minimally Cognitive Systems Further Experiments in the Evolution of Minimally Cognitive Behavior (Clocum, Downey, Beer 2000) (10 pages) pdf
M 2/12 Evolved Virtual Creatures Evolving 3D Morphology and Behavior by Competition (Sims 1994) (11 pages) HW 3 Due 2/11 pdf
W 2/14 Artificial Embryogeny A Taxonomy for Artificial Embryogeny (Stanley and Miikkulainen 2003) (33 pages) pdf
F 2/16 Pattern Formation and Abstractions of Development Compositional Pattern Producing Networks: A Novel Abstraction of Development (Stanley 2007) (28 pages) pdf
M 2/19 Evolving Morphologies with CPPNs Evolving Complete Robots with CPPN-NEAT (Auerbach, Bongard 2011) (7 pages),
Unshackling Evolution (Cheney, MacCurdy, Clune, Lipson 2013) (7 pages)
HW 4 Due 2/18 pdf
W 2/21 Evolving Neural Networks with HyperNEAT A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks (Stanley, D'Ambrosio, Gauci 2009) (35 pages) pdf
F 2/23 Evolving Morphologies with L-Systems The Advantages of Generative Grammatical Encodings for Physical Design (Hornby, Pollack 2001) (7 pages),
Creating High-Level Components with a Generative Representation for Body-Brain Evolution (Hornby, Pollack 2002) (22 pages)
M 2/26 -- NO CLASS -- -- HW 5 Due 2/25 --
W 2/28 Why Evolve Morphologies? Morphological change in machines accelerates the evolution of robust behavior (Bongard 2010) (7 pages) pdf
F 3/2 Novelty Search Abandoning Objectives: Evolution through the Search for Novelty Alone (Lehman, Stanley 2011) (31 pages) pdf
M 3/5 Stepping Stones, Local Competition, and MAP-Elites On the Deleterious Effects of A Priori Objectives on Evolution and Representation (Woolley, Stanley 2011) (8 pages),
Evolving a Diversity of Creatures through Novelty Search and Local Competition (Lehman, Stanley 2011) (7 pages)
HW 6 Due 3/4 pdf
W 3/7 Behavioral Repertoires and Rapid Adaptation Robots that can adapt like animals (Cully, Clune, Tarapore, Mouret 2015) (4 pages) pdf
F 3/9 Swarms and Multi-agent Systems Predator confusion is sufficient to evolve swarming behaviour (Olson et al. 2013) (7 pages) pdf
M 3/12 SPRING BREAK! -- NO CLASS -- -- --
W 3/14 SPRING BREAK! -- NO CLASS -- -- --
F 3/16 SPRING BREAK! -- NO CLASS -- -- --
M 3/19 The Evolution of Neuroplasticity Evolution of Plastic Control Networks (Floreano, Urzelai 2001) (6 pages),
Phenotypic Plasticity in Evolving Neural Networks (Nolfi, Miglino, Parisi 1994) (10 pages)
HW 7 Due 3/18 pdf
W 3/21 The Evolution of Development Co-evolution of Morphology and Control of Soft-bodied Multicellular Animats (Joachimczak, Wrobel 2012) (7 pages),
Evo-devo in silico: a Model of a Gene Network Regulating Multicellular Development in 3D Space with Artificial Physics (Joachimczak, Wrobel 2008) (9 pages)
F 3/23 How Learning Can Guide Evolution How Learning Can Guide Evolution (Hinton, Nowlan 1987) (7 pages),
How morphological development can guide evolution (Kriegman, Cheney, Bongard 2017) (9 pages)
M 3/26 -- NO CLASS -- HW 8 Due 3/25
W 3/28 -- NO CLASS -- --
F 3/30 -- NO CLASS -- --
M 4/2 Midterm Project Proposal Pitches -- Project Proposal (Midterm) Due 4/1 --
W 4/4 Midterm Project Proposal Pitches Cont. -- --
F 4/6 Midterm Project Proposal Pitches Cont.
M 4/9 Surrogate Evaluation Models and Fitness Predictors Surrogate-assisted evolutionary computation: Recent advances and future challenges (Jin 2011) (13 pages),
Coevolution of Fitness Predictors (Schmidt, Lipson 2008) (13 pages)
Check-in Due 4/8 pdf
W 4/11 Active Learning in Symbollic Regression Automated reverse engineering of nonlinear dynamical systems (Bongard, Lipson 2006) (6 pages),
Distilling Free-Form Natural Lawsfrom Experimental Data (Schmidt, Lipson 2009) (4 pages)
F 4/13 Nested Optimization and Active Learning Resilient Machines Through Continuous Self-Modeling (Bongard, Zykov, Lipson 2006) (4 pages) pdf
M 4/16 Other Methods for Machine Learning in Neural Networks BIA 3.6-3.9 (42 pages) Check-in Due 4/15 pdf
W 4/18 Intro to Reinforcement Learning Reinforcement Learning: A Survey (Kaelbling, Littman, Moore 1996) (39 pages) pdf
F 4/20 Deep Reinforcement Learning for Robotics End-to-End Training of Deep Visuomotor Policies (Levine, Finn, Darrell, Abbeel 2015) (27 pages) pdf
M 4/23 Locomotion Controllers for Virtual Creatures via Deep Reinforcement Learning The Emergence of Locomotion Behaviours in Rich Environments (Heess et al. 2017) (9 pages),
Terrain-Adaptive Locomotion Skills Using Deep Reinforcement Learning (Peng, Berseth, van der Panne 2016) (10 pages)
Check-in Due 4/22 pdf
W 4/25 Evolutionary Computation vs. Reinforcement Learning Evolution Strategies as a Scalable Alternative to Reinforcement Learning (Salimans et al. 2018) (8 pages),
Deep Neuroevolution: Genetic Algorithms are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning (Such et al. 2018) (12 pages)
F 4/27 The Reality Gap Automatic design and manufacture of robotic lifeforms (Lipson, Pollack 2000) (4 pages),
Crossing the Reality Gap in Evolutionary Robotics by Promoting Transferable Controllers (Koos, Mouret, Doncieux 2013) (8 pages)
M 4/30 Evolutionary Optimization and a Testbed for Biological Experiements The evolutionary origins of modularity (Clune, Mouret, Lipson 2013) (7 pages) Check-in Due 4/29 pdf
W 5/2 The Pitfalls of Evolutionary Optimization Natural Selection Fails to Optimize Mutation Rates for Long-Term Adaptation on Rugged Fitness Landscapes (Clune et al. 2008) (7 pages) pdf
F 5/4 Final Project Presentations -- Videos Due 5/3 --
F 5/11 Final Project Presentations 10:15am-12:15pm -- --

Reading Reactions

All graduate students will also be asked to email me a one-half-to-one-page long reaction to the readings each week. This should not simply be a summarization, but your thoughts and opinions about the papers -- showing critical thinking and making connections to previous works from the course and your own research, as well as potential future implications of the paper. Undergraduate students with a future aspirations in science, research, and engineering are strongly encouraged to also complete reading reactions and email them to me for extra credit. Paper reactions are due at the end of each week (Sunday at 11:59pm).

Homework Assignments

The Homeworks assigments are hands-on opportunities to demonstrate your knowledge of the concepts covered in class as well as your ability to think creatively and code-up effective and cool programs! These assignments are cumulative, and will build on each other to results in a fully-functioning evolutionary robotics optimization sandbox by the end of the assigments!
(Note: all assignments are due by 11:59pm on the day listed below)

Day Due Date Assignment Topics Covered Code
Su 1/28 HW #1 Build a Simple Hillclimber zip
Su 2/4 HW #2 Build a Neural Network zip
Su 2/11 HW #3 Optimize your Neural Network with your Hillclimber zip
Su 2/18 HW #4 Getting Started with Robotic Physics Simulation in Bullet zip
Su 2/25 HW #5 Use your Hillclibmer to Optimize your Neural Network to Control your Simulated Robot zip
Su 3/4 HW #6 Use an Evolutionary Algorithm to Optimize your Robot's Controller zip
Su 3/18 HW #7 Evolve the Body Plan of your Robot with an Evolutionary Algorithm zip
Su 3/25 HW #8 Use a Multi-Objective Evolutionary Algorithm to Optimize Multiple Features of your Robot zip

Course Projects

Course projects are a way to get your hands dirty and work on real problems that you care about, while showing off what you've learned this semester and becoming a practicing evolutionary robotics researcher! You will be asked to propose and impelment a new experiment to test within the simulation and optimization framework that you've built in through your homework assigments. At the end of the semester, you'll create a video to share with your class (and the world!) and talk about what you've made. A couple check-in along the way are meant to keep you on track. I've listed the dates for these check in's below, and they should be emailed to me by 11:59pm on the day listed.

Day Due Date Deliverable Description
F 4/1 Project Proposal pdf
Su 4/8, 4/15, 4/22, 4/29 Mid-project Check-ins pdf
Th 5/3 Final Project Video pdf

Class Discussion Forum

Being part of an engaged and active community discussion is a huge part of being an research, scientist, or engineer. All course announcements and discussion related to lectures, homeworks, announcements, and anything else that's fun and relevent will be held on the Piazza board here:
Please sign up and be an active part of this discussion (doing so is part of your class participation grade!).