Evolving AI Lab

Evolving Modular, Regular Neural Networks

Our work shows that networks evolve to be modular when there is a cost for network connections: such modularity improves evolvability. Modularity is ubiquitous in natural networks, such as genetic regulatory networks, protein networks, and the neural networks that make up animal brains. It is thought that such modularity is a key reason natural networks are so complex and functional. Evolving modular artificial neural networks should thus allow the evolution of more complex phenotypes, and may be a prerequisite to evolving truly artificially intelligent robots.

Videos

Evolving Artificial Neural Networks That Are Both Modular and Regular

This video accompanies the following paper(s):

Huizinga J, Mouret JB, Clune J (2014) Evolving neural networks that are both modular and regular: HyperNEAT plus the Connection Cost Technique. Proceedings of the Genetic and Evolutionary Computation Conference. 697-704. (pdf)

Why does modularity evolve? The evolutionary origins of modularity

Engineered and evolved things are organized in modules (e.g. organs or car parts), yet why modularity evolves remains one of biology's most important open questions. This paper shows for the first time that modularity evolves not because it speeds up adaptation, as the leading theory holds, but because it saves on "wiring costs". Connections in biological networks have costs (e.g. building and maintaining them), and modular networks use fewer connections. These results help explain the ubiquitous modularity in biological networks, such as genetic modules and the neural modules in our brains, and will help scientists evolve smarter artificial intelligence. Interestingly, the modular networks that evolve do adapt faster, meaning that adaptation is a consequence of modularity, not its main cause.

This video accompanies the following paper(s):

Clune J, Mouret JB, Lipson H (2013) The evolutionary origins of modularity. Proceedings of the Royal Society B. 280: 20122863. (pdf)

Talk at the Santa Fe Institute: Two Projects in BioInspired AI. Evolving regular, modular, hierarchical neural networks, and robot damage recovery.

This video accompanies the following paper(s):

Huizinga J, Mouret JB, Clune J (2014) Evolving neural networks that are both modular and regular: HyperNEAT plus the Connection Cost Technique. Proceedings of the Genetic and Evolutionary Computation Conference. 697-704. (pdf)

Cully A, Clune J, Tarapore D, Mouret JB (2015) Robots that can adapt like animals. Nature. Vol 521 Pages 503-507. (pdf)

Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills

This video accompanies the following paper(s):

Ellefsen KO, Mouret JB, Clune J (2015) Neural modularity helps organisms evolve to learn new skills without forgetting old skills. PLoS Computational Biology 11(4): e1004128. (pdf)

Evolving Regular, Modular Neural Networks

I (Jeff Clune) summarize my research into evolving modular, regular neural networks, which are digital models of brains. The property of regularity is produced by using HyperNEAT, a generative encoding based on concepts from developmental biology. The property of modularity arises because we add a cost for connections between neurons in the network. Evolving structurally organized neural networks, including those that are regular and modular, is a necessary step in our long-term quest of evolving computational intelligence that rivals or surpasses human intelligence.

This video accompanies the following paper(s):

Clune J, Stanley KO, Pennock RT, Ofria C (2011) On the performance of indirect encoding across the continuum of regularity. IEEE Transactions on Evolutionary Computation. 15(3): 346-367. (pdf)

Clune J, Mouret JB, Lipson H (2013) The evolutionary origins of modularity. Proceedings of the Royal Society B. 280: 20122863. (pdf)

Evolving Modular Networks: Video for "The Evolutionary Origins of Modularity"

This video accompanies the following paper(s):

Clune J, Mouret JB, Lipson H (2013) The evolutionary origins of modularity. Proceedings of the Royal Society B. 280: 20122863. (pdf)

Talk summarizing "Evolving Neural Networks That Are Both Modular and Regular: HyperNEAT Plus the Connection Cost Technique"

Talk given by Joost Huizinga at the 2014 GECCO Conference in Vancouver, British Columbia.

This video accompanies the following paper(s):

Huizinga J, Mouret JB, Clune J (2014) Evolving neural networks that are both modular and regular: HyperNEAT plus the Connection Cost Technique. Proceedings of the Genetic and Evolutionary Computation Conference. 697-704. (pdf)

Publications

Mengistu H, Huizinga J, Mouret JB, Clune J (2016) The evolutionary origins of hierarchy. PLoS Computational Biology. (pdf)
Roby Velez, Jeff Clune (2016) Identifying core functional networks and functional modules within artificial neural networks via subsets regression. Proceedings of the Genetic and Evolutionary Computation Conference. Nominated for a best paper award. (pdf)
Huizinga J, Mouret JB, Clune J (2016) Does aligning phenotypic and genotypic modularity improve the evolution of neural networks?. Proceedings of the Genetic and Evolutionary Computation Conference. (pdf)
Ellefsen KO, Mouret JB, Clune J (2015) Neural modularity helps organisms evolve to learn new skills without forgetting old skills. PLoS Computational Biology 11(4): e1004128. (pdf)
Huizinga J, Mouret JB, Clune J (2014) Evolving neural networks that are both modular and regular: HyperNEAT plus the Connection Cost Technique. Proceedings of the Genetic and Evolutionary Computation Conference. 697-704. (pdf) (Source code)
Clune J, Mouret JB, Lipson H (2013) The evolutionary origins of modularity. Proceedings of the Royal Society B. 280: 20122863. (pdf) (Experimental Data, Source Code, Scripts)
Suchorzewski M, Clune J (2011) A novel generative encoding for evolving modular, regular and scalable networks. Proceedings of the Genetic and Evolutionary Computation Conference. 1523-1530. (pdf)
Clune J, Beckmann BE, McKinley PK, Ofria C (2010) Investigating whether HyperNEAT produces modular neural networks. Proceedings of the Genetic and Evolutionary Computation Conference. 635-642. (pdf)