$ echo "🌸 # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 🌸# # # # # # 🌸 # # # # # # 🤖 #"
$ echo "# # # # # # 🌸🌸🌸🌸 # # # # #🌸 🌸 # # # #🌸🌸🌸🌸 # # # #🌸 🌸 # # # # # # # # # # # # # # # # 🌸 # #"
$ echo "# # # 🤖 # 🌸 # # #🌸 # # 🌸 # # 🌸 # # #🌸 # # #🌸 # 🌸 # # 🌸 # # # # # # # #🌸 # # # # # # # # # # #"
$ echo "# # # # # # 🌸 # #🌸 # # # 🌸 # # # #🌸 # #🌸 # # #🌸 # 🌸 # # # 🌸 # # # # # 🌸 # 🌸 # # # # # # # # # #"
$ echo "# # # # # # 🌸🌸🌸 # # # # 🌸 # # # #🌸 # #🌸 🌸 🌸 # # 🌸 # # # 🌸 # # # # # # # #🌸 # # # # # # # # # #"
$ echo "# # # # # # 🌸 # 🌸 # # # #🌸 # # # #🌸 # #🌸 # # #🌸 # 🌸 # # # 🌸 # # # # # 🌸# #🌸 # # # # # 🌸 # # # #"
$ echo "# # 🌸# # # 🌸 # # 🌸 # # # 🌸 # # 🌸 # # #🌸 # # #🌸 # 🌸 # # 🌸 🤖 # # # # # # 🌸 # # # # # # # # # # #"
$ echo "# # # # # # 🌸 # # # 🌸 # # # 🌸 🌸# # # # 🌸🌸🌸🌸 # # # #🌸 🌸 # # # # # # # # # # # # # # # # 🌸 # # # "
$ echo "# # # # # # # # # # # # # 🌸# # # # # # # # # # # # # # # # # # # # # # 🌸 # # # # # # # # # # # # # # 🌸 #"
Robotics and Mechatronics in short is the science and art of making physically intelligent yet artificial beings. AI is about mimicking the human senses ~ (Computer Vision == eyesight; NLP == language) and Robotics ~ physical touch.
I prepared some notes on Robotics & Mechatronics and took a course on Robot Kinematics. The usual flow:
+ Robot Kinematics: Kinematics is the study of the relationship between a robot's joint coordinates and its spatial layout, and is a fundamental and classical topic in robotics. Kinematics can yield very accurate calculations in many problems, such as positioning a gripper at a place in space, designing a mechanism that can move a tool from point A to point B, or predicting whether a robot's motion would collide with obstacles. Kinematics is concerned with only the instantaneous values of the robot's coordinates, and ignores their movement under forces and torques.
+ Robot Dynamics: The dynamics in robotics, in English defined as robot dynamics, studies the forces acting on a robotic mechanism and the accelerations that these forces produce. The robotic mechanism is generally considered as a rigid system, in order to apply the dynamic laws of rigid bodies. The two main problems in robot dynamics are:
- Forward dynamics: given the forces, work out the accelerations.
- Inverse dynamics: given the accelerations, work out the forces.
- Hybrid dynamics: given the forces at some joints and the accelerations at others, work out the unknown forces and accelerations.
+ Robot Motion Planning and Control: Motion planning, also path planning (also known as the navigation problem or the piano mover's problem) is a computational problem to find a sequence of valid configurations that moves the object from the source to destination.
+ Robot Manipulation : The manipulation task usually requires moving an object to a specified location in space, without particular regard as to how the manipulator can accomplish the task. The model considered here greatly simplifies the problems of grasping, stability, friction, mechanics, and uncertainties and instead focuses on the geometric aspects.
+ Robot State Estimation and Learning : How can robots determine their state and properties of the surrounding environment from noisy sensor measurements in time? how to get robots to incorporate uncertainty into estimating and learning from a dynamic and changing world.
+ Robot Perception: Robots' ability to interact with their surroundings is an essential capability, especially in unstructured human-inhabited environments. The knowledge of such an environment is usually obtained through sensors. The study of acquiring knowledge from sensor data is called robotic perception.
I made a video on "state of the art robotics research" to explore and learn about new robotics projects. [ video ].
projects : [ Adaptive Walking of Bio-inspired Musculoskeletal Soft Quadruped Robot in Slow Motion, 으뜸병원로봇동영상, Arduino Quadruped Robot Dancing to 'Small Steps', ウマの走行能力のヒミツに迫る 解剖学的な歩行ロボット, CRL Legged Robotics Team in Swiss Robotics Day 2023, 方舟无限ARX5 轻型力控机械臂, ARX-5-robot arm dancing -real2sim2real2sim2real…, Hexapod Robot Mat6 Climbs Stairs, Hexapod Mark IV August Update, KITECH Dual Arm Robot Performs Manipulation Tasks, peg-in-hole with dual robot arm, ARX方舟无限 Ultron / Dual-arm Robot - for Embodied AI, PDH-GS120, Spirit, RAF 'Jackal', Sonnie Robot [ Shanghai AI event], BirdBot - [Ostrich-inspired 'BirdBot' ], Robotic drones with suction pads can fly, swim or hitch a ride to get around, BAS-200, Thrust vectoring drone, #Oblivion Drone 166, Drone swarm avoids crashing in dense forest using new algorithm, PteroDynamics X-P4 Transwing, VR control of NASA Valkyrie humanoid astronaut robot using PSYONIC Ability Hand, 2023 Summer Project, Pneumatic Augmentation for Explosive Hopping Robots, Exo-H3 Promo 2024, Hierarchical Optimization-Based Control for Agile and Adaptive Legged Robots, Diffusion Policy Assistive Action Prediction, 10 years of swarm study Dec 2023, SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding, Gradient-Based Trajectory Optimization With Learned Dynamics - Supplementary, Project Darkbot Final Demo, Gyroscopic Bipeds: A Different Way of Walking with Dr Pauline Pounds, Introducing HyperLeg: Human-like Robot Leg and Foot for Highly Dynamic Motions, Menteebot by Mentee Robotics is agile, SpaceHopper, Rogue 1 Lethal Unmanned Aerial System, ETH RSS, CMU RISS, China's bug-like microbot races at blistering speeds, Festo BionicBee ]
Robotics Simulation tools : MuJoCo, AirSim, CARLA, Webots, Gazebo, Raisim and a lot more here.
Probabilistic Robotics (Intelligent Robotics and Autonomous Agents series) |
Introduction to Autonomous Mobile Robots |
Modern Robotics: Mechanics, Planning, and Control |
Principles of Robot Motion: Theory, Algorithms, and Implementations |
Planning Algorithms |
|---|
I am interested in Perception and Navigation of Robotic Systems. Feel free to check my pages on robotics hardware : @robotic-components-i, robotic-components-ii. I also found these helpful resources : @github/awesome-robotics, awesome-robotics-libraries, awesome-robotics, awesome-ros2, python-robotics, webots + josm; dih-hero.eu, DRAKE.mit, IEEE RAS CRV , rerun.io.
Aerial robotics, particularly unmanned aerial vehicles (UAVs) or drones, have seen a surge in research and development owing to their diverse applications across various domains. Aerial robotics has witnessed exponential growth, driven by advancements in AI, robotics, and sensor technologies. UAVs are now deployed for tasks ranging from surveillance and reconnaissance to delivery and disaster response.
I was working on hand gesture recognition on aerial systems at my lab. We had dji tello in lab and plans to research with crazyflie 2.1. I developed a control station for drone autonomy in lab and presented my hand gesture recognition work to Senator, Dr. Anjes Tjarks ( Behörde für Verkehr und Mobilitätswende ), Hamburg and LSBG CEO, Dr Klotz at DigiLab, LSBG. [ post ]
The following Institutes have been doing amazing work on aerial robotics : [ MIT Spark Lab ] [ EPFL - Aerial Robotics ] [ UZH - Robotics and Perception Group ] [ Imperial - Aerial Robotics ] [ ETH - Z : ASL - Flying Robots ] [ GRASP Lab : Aerial Robotics ] [ Oxford - Dynamic Robot Systems Group ] [ Stanford - Aerospace Robotics Lab ] [ CMU - AirLab ] [ Smart Robotics Lab - TUM ] [ University of Tokyo - DRAGON Lab ] [ Microsoft - Aerial Informatics and Robotics Group ] [ Caltech - Autonomous Robotics and Control Lab ] [ MIT - Robust Robotics Group ] [ Autonomous Robots Lab, NTNU ] [ Biomorphic Intelligence Lab - TU Delft ] [ Bio-inspired Soft Aerial Robotics ] [ Spark Lab ]
- MIT Robotics - Kevin Chen - Agile, robust, and multifunctional micro-aerial-robots.
- RI Seminar: Kostas Alexis : Autonomous Exploration and Inspection using Aerial Robots.
- MIT Robotics - Roland Siegwart - Flying Robots.
projects : Design, Modeling and Control of Aerial Robot DRAGON, Versatile Articulated Aerial Robot DRAGON: Aerial Manipulation and Grasping by Vectorable Thrust, Learning High-Speed Flight in the Wild (Science Robotics, 2021), Swarm of Micro Flying Robots in the Wild [All], Controlling Robotic Swarms, Crazyflie in research, 2020-2022, EPFL - Aerial Robotics, This drone has legs: Watch a flying robot perch on branches, catch a tennis ball in mid-air, GAAS, This Transformative Drone Can Change its Shape Mid-Flight, PteroDynamics X-P4 Transwing® sizzle video, 'Energy-efficient flapping-wing robots' workshop puts bio-inspired aerial robots to the test, MonoNav: MAV Navigation via Monocular Depth Estimation and Reconstruction, Semester project in Autonomous Aerial Robotics at UMD, The bio-inspired 'transformer' that crawls, rolls and flies, Reliable Robotics remotely operates a large cargo aircraft with no one on board, A Robot That Walks, Flies, Skateboards, Slacklines, Optimal elastic wing for flapping-wing robots through passive morphing, OmniNxt:A Fully Open-source and Compact Aerial Robot with Omni Visual Perception, GPS free drone navigation with VIO, ROS Aerial Robotics WG meeting April 2024, Impact-Aware Planning and Control for Aerial Robots with Suspended Payload, PufferBot: Actuated Expandable Structures for Aerial Robots, FALCON: Fast Autonomous Aerial Exploration using Coverage Path Guidance, OmniNxt:A Fully Open-source and Compact Aerial Robot with Omni Visual Perception, Aerial Robotics community group meeting.
resources : Bitcraze Aerial Robotics Meeting, UZH Robotics and Perception Group, BitCraze Videos, International Conference on Unmanned Aircraft Systems (ICUAS), Springer - Aerial Robotics, Springer - Aerial Robotics, MDPI - Aerial Robotics for Inspection and Maintenance, list of aerial robotics classes. Here are some books - A First Course in Aerial Robots and Drones : pdf, Control of Ground and Aerial Robots, online textbook, Udacity - Flying Car and Autonomous Flight Engineer, Optimal elastic wing for flapping-wing robots through passive morphing, aerial-robotics-course, @MITRoboticsSeminar.
The following Institutes have been doing amazing work on robot manipulation : MIT ROBOTIC MANIPULATION, University of Leeds - Robotic Manipulation, CMU Robot Perception Lab, Robotics: Perception and Manipulation (RPM) Lab, UMich - Robot Perception & Manipulation, UWash - Personal Robotics Lab, Georgia Tech - Adaptive Robotic Manipulation (ARM) Laboratory, NVIDIA Seattle Robotics Lab.
Some recent work on Robot Manipulation:
- Perception and Manipulation in Robotics: Neural Network Approaches
- Robot Perception and Learning for Navigation, Manipulation, and Locomotion
- Robot learning and perception for contact-rich manipulation
- MIT Robotics - Dieter Fox - Toward Foundational Robot Manipulation Skills
- MIT Robotics - Jeffrey Ichnowski - Dynamic Robot Manipulation
- StanfordWebinar - Autonomous Robotic Manipulation: What’s Within Reach? Jeannette Bohg
- PhD Thesis Defense - Maria Bauza - Visuo-Tactile Perception for Dexterous Manipulation
- PhD Thesis Defense - Siyuan Dong - High-resolution Tactile Sensing for Reactive Robotic Manipulation
- Stanford Seminar - Robotic Autonomy and Perception in Challenging Environments
- RI Seminar: Michael Kaess: Factor Graphs for Robot Perception
Institutes : Institut de Robotica, UMich - Robot Perception & Manipulation Robot Perception & Manipulation, Robotics: Perception and Manipulation (RPM) Lab, MPI - Autonomous Robotic Manipulation, DLR - Perception and Cognition, Stanford ARM Lab, Flight Robotics and Perception Group (FRPG).
resources : Active Manipulation for Perception, DeepRob: Deep Learning for Robot Perception; class - MIT 6.4210/6.4212 Robotic Manipulation, Workshop on "Benchmarking in Robotic Manipulation", Workshop on "Learning, Perception, and Abstraction for Long-Horizon Planning", Robot Manipulation Final Projects, @github/robosuite, robot perception projects, class : Modern Computer Vision, Computer Vision — Andreas Geiger, MIT - Robotic Manipulation Perception, Planning, and Control, MIT Robotic Manipulation 2023 (class), DeXtreme: Transferring Dexterous Manipulation from Simulations to Reality.
Simultaneous localization and mapping is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it. I have good notes on SLAM here @visual-slam.
Some recent work on SLAM:
- CSIRO's Wildcat SLAM: A Robust SLAM System for Robot Teams in the Wild
- Swarm-SLAM : Sparse Decentralized Collaborative Simultaneous Localization and Mapping Framework for Multi-Robot Systems
- An Introduction to Robot SLAM (Simultaneous Localization And Mapping)
- FastSLAM: A Factored Solution to the Simultaneous Localization and Mapping Problem
- Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
- Introduction to Visual SLAM - repo
- Online LiDAR-SLAM for Legged Robots with Robust Registration and Deep-Learned Loop Closure
- Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM
- ICCV'23 Robot Learning & SLAM Workshop
resources: Easy SLAM with ROS using slam_toolbox, Understanding SLAM Using Pose Graph Optimization, SLAM Robot Mapping, SLAM Course - Cyrill Stachniss, Simultaneous Localization and Mapping (SLAM) for Robotics - NTNU, VSLAM for Robotic Applications, How to get your robot to see in 3D! (Depth Cameras in ROS, Dr. Franz Andert, SLAM - Matlab, SLAM for Dummies, Indoor Robot Localization with SLAM, Multi Agent SLAM - DLR, SLAM variants, Springer - SLAM, ICRA SLAM paper list, paperswithcode - SLAM / kimera, @github/awesome-visual-slam, awesome-slam, semantic-slam, lidar-slam, slam-algorithms.
Some recent work on Motion Planning:
- Robot Motion Planning using A* (Cyrill Stachniss)
- Introduction to Motion Planning for Flying Robots: Dr. Sebastian Scherer
- Advanced Motion Planning: FMT*, Informed RRT*, BIT*, and RABIT* | An OMPL Tutorial with Examples
- RI Seminar: Russ Tedrake: Robust motion planning for walking robots and robotic birds
- Russ Tedrake: Motion Planning Around Obstacles with Convex Optimization
- Siddharth Srivastava on "Task and Motion Planning for Robots..."
- Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning
- Robot Motion Planning: Challenges and Opportunities for Increasing Robot Autonomy
resources: Motion Planning for Robots, UMich - Motion Planning, A review of motion planningalgorithmsforintelligent robotics, Springer - Robot Motion Planning, MIT - CSAIL : Task and Motion Planning for Autonomous Robots, ACM - Robot Motion Planning, Neighborhood Attention Transformer (CVPR 2023), Transformer Architecture, Self-Attention Using Scaled Dot-Product Approach, A Dive Into Multihead Attention, Self-Attention and Cross-Attention.
Some recent work on Navigation:
- A comprehensive study for robot navigation techniques
- Visual language maps for robot navigation
- Improving Autonomous Robotic Navigation Using Imitation Learning
- Mobile Robot Navigation and Obstacles Avoidance based on Planning and Re-Planning Algorithm
- Autonomous Robot Navigation in Highly Populated Pedestrian Zones
- Robot Mapping and Navigation in Real-World Environments
- Mobile Robot Navigation Using Deep Reinforcement Learning
- Navigation System of the Autonomous Agricultural Robot “BoniRob”*
resources: Autonomous Navigation - Matlab, Making robot navigation easy with Nav2 and ROS!, Where Can Machine Learning Help Robotic State Estimation?, Springer - Mobile Robot Navigation, paperswithcode - robotnavigation, Safe Robot Navigation in Dense Crowds.
A multi-agent system is a computerized system composed of multiple interacting intelligent agents. Multi-agent systems can solve problems that are difficult or impossible for an individual agent or a monolithic system to solve. Reinforcement Learning is widely used in Multi Agent System problems and my deep reinforcement learning notes are here @/drl.
Some recent works on Multi Agent Systems:
- "Learning to Communicate in Multi-Agent Systems" - Amanda Prorok
- Understanding Equilibria in Multi-Agent Systems - Michael Wooldridge, University of Oxford
- Scalable and Robust Multi-Agent Reinforcement Learning - Microsoft Research
- The Role of Multi-Agent Learning in Artificial Intelligence Research at DeepMind
- Can AI Learn to Cooperate? Multi Agent Deep Deterministic Policy Gradients (MADDPG) in PyTorch
- MIT Robotics - Dimitra Panagou - Safety and Resilience in Multi-Agent Systems
- DLRLSS 2019 - Multi-Agent Systems - James Wright
- CMU - Multi Agent Systems, An Introduction to Multi-Agent Systems, IEEE - Multi-Agent Systems: A Survey
Institutes : Deepmind, TUD - Intelligent Multi-Agent Systems, Automatic Coordination of Teams (ACT) Lab at Brown University, Autonomous Multi-Robots Lab. Delft University of Technology, huntlab, Harvard University : Designing Emergence Laboratory, Inria - Lagadic, Chroma, LABUST, Stanford - Multi-robot Systems Lab (MSL), MPI - Robotic Perception Group.
resources : ScienceDirect - Swarm Intelligence, Multi Agent Systems, Springer - Swarm Intelligence, Swarm Intelligence Principles, Advances, and Applications, YT, AAMAS (International Conference on Autonomous Agents and Multiagent Systems).
github : multi_agent_path_planning, camel, MARL-papers, Petting Zoo, awesome-deep-rl, mava, MARL-papers-with-code
The decade of talking to robots in our native natural language is here with LLMs:
Human Robot Interaction with Large Language Models (LLMs) :
- LLM-GROP : Task and Motion Planning with Large Language Models for Object Rearrangement
- LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action
- MOO : Open-World Object Manipulation using Pre-trained Vision-Language Models
- Large Language Models as Zero-Shot Human Models for Human-Robot Interaction
- Language Models for Human-Robot Interaction
[ Science Direct : HRI ]. To know more about LLMs, check my repo on AI @aicore_n.
-
[ ROS GPT ], [ @github/Awesome-LLM-Robotics, @github/Everything-LLMs-And-Robotics ], Special Issue: Large Language Models in Robotics, Large Language Models-powered Human-Robotic Interactions, OpenAI
-
demo video : ChatGPT for Robotics; article : Google Research, 2022 & beyond: Robotics.
2024: The Year of Humanoid Robots? (+ More Tech News)
Important resources on space robotics:
[ Mars Rover ] [ Robotic Servicing Arm ] [ European Robotic Arm ] : ERA
[ IEEE Space Robotics ] [ NASA Space Robotics Challenge ] [ Space Robotics: A Comprehensive Study of Major Challenges and Proposed Solutions ] [Science Direct - Space Robotics] [ Robotic Manipulation and Capture in Space: A Survey ] [ Space Robotics ] : [ pdf ] [ Robots in space | Meet the experts ] [ PLANETARY ROVERS - Robotic Exploration of the Solar System ] [ space.ros.org ]
Institutes: DLR - Institute of Robotics and Mechatronics, DFKI - Robotics Innovation Center (Space Robotics), Tohoku University - The Space Robotics Lab (Space Exploration Lab), MIT Astrodynamics, Space Robotics, and Controls Lab (ARCLab), Autonomous Space Robotics Lab (ASRL) at the University of Toronto, LASR Lab, Autonomous Robotics and Control Lab at Caltech, Planetary Robotics Laboratory (PRL) - JPL, NASA
Continuum Robotics : Jessica Burgner-Kahrs - Continuum Robotics Laboratory in UofT; Nanorobotics - NanoRobotics Lab - CMU, Nanorobotics- GRASP Lab, Multi Scale Robotics Lab; CLOVER.
Companies : Boston Dynamics, Festo, AnyBotics, Clear Path Robotics, Rethink Robotics, Google Robotics.
Youtube: @BostonDynamics, @ANYbotics, @Festo @BPSspace, @RSS, @integza, @Skyentific, Understanding Sensor Fusion and Tracking, Kalman Filter, Robust Control, Autonomous Navigation, PID Control, AMECA, @Robotics Today, @UofTRoboticsInstitute @MITRobotics, How Space Robots Shape The Future, IEEE RAS.
Class : ETH-Z Autonomous Mobile Robots, University of Pennsylvania : GRASP Lab, Modern Robotics: Mechanics, Planning, and Control, Technische Universität München: Autonomous Navigation for Flying Robots )(AUTONAVx) ; list of all robotics classes @ [MIT], @Stanford : [Robotics and Autonomous Systems Graduate Certificate], CMU - Robotics Institute, UMich - Mobile Robotics.
MIT 16.485 : Visual Navigation For Autonomous Vehicles (VNAV) | 6.141/16.405j - Robotics: Science and Systems | TUM : Lecture: Visual Navigation for Flying Robots | Introduction to Robotics @ Princeton | UMich - Robotics 101 (Fall 2020) | Introducing PyPose: A Library for Differentiable Robotics, Realtime 3D Drone Localization with Lidar (Laser) SLAM | Bendy robot arms made cheaper, lighter and more efficient thanks to electrostatic tech, Chebyshev Linkage Robot