diff --git a/_data/pubs.yml b/_data/pubs.yml index 7855ad7..0bb1420 100644 --- a/_data/pubs.yml +++ b/_data/pubs.yml @@ -1,3 +1,14 @@ +- title: "From Pixels to Torques with Linear Feedback" + authors: [Jeong Hun Lee, Sam Schoedel, Aditya Bhardwaj, Zachary Manchester] + projects: [linear_pixels_to_torques] + publisher: International Workshop on the Algorithmic Foundations of Robotics + abbrv: WAFR + pub-type: conference + venue: Chicago, IL + date: 2024-10-07 + pdf: linear_pixels_to_torques.pdf + status: In Review + - title: "Contingency-Aware Station-Keeping Control of Halo Orbits" authors: [fausto, zac, Martin Lo, Ricardo Restrepo] publisher: Conference on Decision and Control diff --git a/_projects/linear_pixels_to_torques.md b/_projects/linear_pixels_to_torques.md new file mode 100644 index 0000000..ce01263 --- /dev/null +++ b/_projects/linear_pixels_to_torques.md @@ -0,0 +1,20 @@ +--- +title: Pixels to Torques with Linear Feedback + +description: | + Data-driven, linear output-feedback policies can effectively control a robotic system using vision. + +people: + - jj + - sam + - zac + +layout: project +image: "/img/linear_pixels_to_torques/linear_pixels_to_torques.gif" +last-updated: 2024-06-26 +--- + +We demonstrate the effectiveness of simple observer-based linear feedback policies for "pixels-to-torques" control of robotic systems using only a robot-facing camera. Specifically, we show that the matrices of an image-based Luenberger observer (linear state estimator) for a "student" output-feedback policy can be learned from demonstration data provided by a "teacher" state-feedback policy via simple linear-least-squares regression. The resulting linear output-feedback controller maps directly from high-dimensional raw images to torques while being amenable to the rich set of analytical tools from linear systems theory, alowing us to enforce closed-loop stability constraints in the learning problem. We also investigate a nonlinear extension of the method via the Koopman embedding. Finally, we demonstrate the surprising effectiveness of linear pixels-to-torques policies on a cartpole system, both in simulation and on real-world hardware. The policy successfully executes both stabilizing and swing-up trajectory tracking tasks using only camera feedback while subject to model mismatch, process and sensor noise, perturbations, and occlusions. + +# Resources +* Code coming soon! \ No newline at end of file diff --git a/img/linear_pixels_to_torques/linear_pixels_to_torques.gif b/img/linear_pixels_to_torques/linear_pixels_to_torques.gif new file mode 100644 index 0000000..f488df9 Binary files /dev/null and b/img/linear_pixels_to_torques/linear_pixels_to_torques.gif differ diff --git a/papers/linear_pixels_to_torques.pdf b/papers/linear_pixels_to_torques.pdf new file mode 100644 index 0000000..519a39f Binary files /dev/null and b/papers/linear_pixels_to_torques.pdf differ