Title page for ETD etd-08172007-151826

Type of Document Master's Thesis
Author Renner, Michael Robert
URN etd-08172007-151826
Title Machine Learning Simulation: Torso Dynamics of Robotic Biped
Degree Master of Engineering
Department Mechanical Engineering
Advisory Committee
Advisor Name Title
Granata, Kevin P. Committee Chair
Hong, Dennis W. Committee Member
Kasarda, Mary E. F. Committee Member
Reinholtz, Charles F. Committee Member
Sandu, Corina Committee Member
  • Dynamic Bipedal Walking
  • Reinforcement Learning
  • Q-Learning
  • Torso
  • NEAT+Q
Date of Defense 2007-08-10
Availability unrestricted
Military, Medical, Exploratory, and Commercial robots have much to gain from exchanging wheels for legs. However, the equations of motion of dynamic bipedal walker models are highly coupled and non-linear, making the selection of an appropriate control scheme difficult. A temporal difference reinforcement learning method known as Q-learning develops complex control policies through environmental exploration and exploitation. As a proof of concept, Q-learning was applied through simulation to a benchmark single pendulum swing-up/balance task; the value function was first approximated with a look-up table, and then an artificial neural network. We then applied Evolutionary Function Approximation for Reinforcement Learning to effectively control the swing-leg and torso of a 3 degree of freedom active dynamic bipedal walker in simulation. The model began each episode in a stationary vertical configuration. At each time-step the learning agent was rewarded for horizontal hip displacement scaled by torso altitude--which promoted faster walking while maintaining an upright posture--and one of six coupled torque activations were applied through two first-order filters. Over the course of 23 generations, an approximation of the value function was evolved which enabled walking at an average speed of 0.36 m/s. The agent oscillated the torso forward then backward at each step, driving the walker forward for forty-two steps in thirty seconds without falling over. This work represents the foundation for improvements in anthropomorphic bipedal robots, exoskeleton mechanisms to assist in walking, and smart prosthetics.
  Filename       Size       Approximate Download Time (Hours:Minutes:Seconds) 
 28.8 Modem   56K Modem   ISDN (64 Kb)   ISDN (128 Kb)   Higher-speed Access 
  etd_part1.pdf 209.51 Kb 00:00:58 00:00:29 00:00:26 00:00:13 00:00:01
  etd_part2.pdf 320.46 Kb 00:01:29 00:00:45 00:00:40 00:00:20 00:00:01
  etd_part3.pdf 1.64 Mb 00:07:35 00:03:54 00:03:25 00:01:42 00:00:08

Browse All Available ETDs by ( Author | Department )

dla home
etds imagebase journals news ereserve special collections
virgnia tech home contact dla university libraries

If you have questions or technical problems, please Contact DLA.