What is reinforcement learning in humanoid robot training?

What is reinforcement learning in humanoid robot training?

The word robot first appeared in the play Rossum’s Universal Robots written by the Czechoslovakian playwright Karel Čapek. In the story, he described human-like mechanical beings and referred to them as robots. The term comes from the Czech word robota, meaning hard work or forced labor. It is fascinating that the origin of robots can be traced back to a literary work.

A Humanoid Robot refers to a robot designed by imitating the physical structure of a human being. Most industrial robots are built in forms optimized for specific environments or production processes. The same logic applies to why a Humanoid Robot is shaped similarly to a person. To perform tasks that humans can perform, the robot must be able to reproduce human-like movements.

Humans naturally go through the process of learning to walk and speak from early childhood. Even without explicit instruction, we instinctively acquire such abilities. Robots, however, do not. Unlike humans, they cannot learn on their own unless specific training data is provided. This is why Reinforcement Learning and other AI-based training methods have recently received significant attention in the field of intelligent robotics.

1. Methods of Data Learning

You have likely heard the term machine learning at least once. In Korean, it is often referred to as mechanical learning, and it describes a series of processes in which algorithms are developed and trained to learn in a human-like manner. Learning like a human means building the ability to identify patterns on its own and discover methods for solving problems. To achieve this, a substantial amount of data is required. Methods in machine learning can generally be divided into three categories. supervised learning, unsupervised learning, and the main topic of today’s discussion, Reinforcement Learning.

Supervised Learning

Supervised learning is a method in which both the problem and the correct answer are provided at the same time. For example, imagine developing an AI model that classifies gender from photographs. To train this model, we supply the computer with images of men and women and clearly indicate which image corresponds to which gender. Based on the given problems (the images) and the answers (the labels), the AI analyzes the differences between the two categories and gradually discovers patterns through repeated learning.

With enough accumulated training data, the AI becomes capable of inferring the correct answer even when it encounters new images. Because this approach guides the AI toward the correct answer by presenting both the problem and the solution, it is called supervised learning. You can think of it as a method for enhancing the AI’s reasoning ability and improving its performance in predicting accurate results.

Unsupervised Learning

Unsupervised learning, unlike supervised learning, is a method in which only the problem is presented without providing the correct answer. The key concept here is data clustering. Simply put, the AI identifies patterns within the input data and groups similar items together on its own.

This approach is particularly effective when dealing with unlabeled data or extremely large datasets. For example, consider a company preparing to launch a new product and trying to understand how its product should be positioned in the market.

To do this, the company must gather a wide range of information, such as what features consumers react to, how price expectations are shifting, and how the pricing trends of competing products are changing. While there may be plenty of data such as comments, reviews, sales volume, and pricing, there are no predefined “correct answers” indicating which group each piece of data belongs to. The data exists, but there are no labels that state, “this data fits into this category.”

In such cases, when features like price, comments, and sales volume are fed into the AI, it automatically organizes the data into clusters based on similarity, even though it has not been given the correct labels. When humans later review these clusters, they can derive meaningful categories such as.

Customers paying a high price and reporting high satisfaction
Customers paying a low price but reporting high satisfaction
Customers paying a high price but reporting low satisfaction
Customers paying a low price and reporting low satisfaction

In other words, the AI did not classify the data based on predefined criteria like “high price” or “low price.” Instead, it grouped similar patterns together, and humans then interpreted those groups and assigned meaning. This is the essence of unsupervised learning.

The primary goal of unsupervised learning is to enable AI to discover hidden structures, patterns, and relationships within the data. Because it does not require labeling like supervised learning, it significantly reduces the time and cost associated with annotation and allows large datasets to be utilized far more efficiently.

Differences Between Supervised and Unsupervised Learning

Supervised learning is a method in which both the problem data and the correct answers, or labels, are provided to the AI. For example, if you have thousands of facial images, a human must manually indicate whether each one is male or female. This labeling process requires significant time, effort, and cost. The AI then learns by associating the features of each labeled image with its correct answer, enabling it to accurately predict gender when it encounters new images.

In contrast, unsupervised learning provides the AI only with the problem data, without any labels. The AI analyzes the features of the images on its own and performs a clustering task, grouping together photos that share similar characteristics. For instance, images with similar hairstyles or facial shapes may be grouped together. Humans then review these clusters and interpret them, saying things like “this cluster contains mostly male images” or “that cluster consists mainly of female images.” Because this method does not require manual labeling, it allows large volumes of data to be processed quickly and cost-effectively.

What is Reinforcement Learning?

In everyday life, people tend to expect some form of reward when they take action or put in effort. At work, we hope for compensation that reflects the intensity of our labor, and similar reward-based structures often appear in relationships with friends, family, or partners. In psychology, this tendency is described as reward-based behavior. Reinforcement Learning is a machine learning method developed from this idea of reward.

In Reinforcement Learning, an AI system interacts continuously with its environment while attempting to achieve a given task. Through this interaction, it learns which actions or paths lead to the highest possible reward. Put simply, the AI observes the environment, selects an action, and receives a reward or a penalty depending on the outcome. Its goal is to refine its behavior so that it maximizes the total reward. By repeating cycles of success and trial and error, the AI discovers better strategies and gradually improves its decision making.

Reinforcement Learning has recently gained significant attention in many areas of artificial intelligence. One reason for this growth is that it does not require pre-collected training data. The AI learns from its own experiences within the environment and improves over time, which makes this method effective when dealing with complex conditions or situations where data preprocessing is difficult. Because this learning process resembles the way humans improve through repeated attempts, Reinforcement Learning is increasingly being applied in robotics, including fields that involve the development of the Humanoid Robot.

2. Learning Methods for Humanoid Robots

How, then, are these learning methods applied to the training process of the Humanoid Robot that we frequently see in recent media? If someone were to ask what has driven the rapid growth of the intelligent robotics industry, three key factors would come to mind. The first two are large language models such as OpenAI’s ChatGPT and Google’s Gemini, followed by vision language models that combine visual and linguistic understanding. The third is Reinforcement Learning, which was explained earlier.

Physical AI

Have you heard the term Physical AI? It refers to forms of artificial intelligence that possess a physical body and can interact directly with the real world. Humanoid Robot systems, autonomous mobile robots, and quadruped robots that frequently appear in the news can all be seen as part of the broader evolution toward Physical AI.

For a robot equipped with intelligence to operate properly in the physical world, it must complete three essential stages. It needs to perceive surrounding objects, make decisions about how to act, and execute those decisions through physical movement. Vision systems such as cameras and LiDAR, sensors that read the robot’s internal state and distance information, and actuators that translate decisions into precise motion all play vital roles. Robotics is an industry that requires multiple advanced technologies to work together.

For robots to interact naturally with humans, communication through language is essential. Large language models allow a robot to understand human speech more effectively and follow commands with greater accuracy. Without the emergence of these models, the current level of interest and expectation surrounding the Humanoid Robot would likely be very different from what we see today. A well-known example is Figure AI’s Figure 01, which demonstrated human interaction capabilities by integrating an advanced language model.

AI technologies that can recognize and process various types of data, including images, text, and audio, are called multimodal models. Among them, the Vision-Language Model (VLM) is designed to handle visual information and language simultaneously.

A Vision-Language Model does not interpret images and text separately. Instead, it connects the two by understanding them within the same representational space. This allows a robot not only to recognize that a box exists but also to understand where that box is located and what actions need to be taken. You can think of it as an enhanced version of perception and processing capability.

How Do Robots Learn?

How does learning actually take place in robots? Ogata, a senior researcher at Japan’s National Institute of Advanced Industrial Science and Technology (AIST), is a robotics and artificial intelligence expert with more than thirty years of experience. According to his explanation, the core technologies behind the development of the Humanoid Robot are Reinforcement Learning and imitation learning.

Imitation learning, as the name suggests, is a method that trains a robot using datasets that mimic human behavior. For instance, actions and situations are repeatedly recorded using console devices or motion capture systems. The robot then learns to reproduce these past movements on its own by training on the accumulated demonstration data. Among the machine learning techniques introduced earlier, this method is most closely related to supervised learning.

Reinforcement Learning, on the other hand, allows the robot to discover the optimal set of actions for achieving a goal by going through its own cycles of trial and error. Directly applying this process to a physical robot can lead to high risks such as mechanical failure or safety issues. This is why many recent approaches rely heavily on virtual physics environments such as NVIDIA’s Isaac Sim. Robots perform large numbers of training cycles and experiments inside simulation, and the data obtained is later validated and refined in the real world. This process, known as simulation to reality (Sim2Real), is a crucial part of Reinforcement Learning.

The advantage of Reinforcement Learning is that the robot can independently identify problems, improve its behavior, and eventually reach a correct solution. However, the amount of experimentation required is enormous. Even with the help of virtual physics engines, an extensive number of training samples is necessary. The very fact that this simulated data must be repeatedly tested and refined in real physical environments shows how challenging this technology development process can be.

3. The Key Is Sim2Real

In the earlier discussion on Reinforcement Learning for the Humanoid Robot, we explained that verifying simulation-trained data in the real world is a crucial part of the learning process. Ultimately, the purpose of Physical AI is to deliver real value to consumers, whether companies or individuals, in actual physical environments. The central question becomes whether the robot can perform real tasks and fulfill mission requirements in real-world conditions.

No matter how much data a robot learns in simulation, the real world contains countless variables. If essential components such as sensors, vision systems, and actuators do not operate reliably under these conditions, the industrialization or widespread adoption of intelligent robots becomes impossible.

At Bonsystems, we recognized these core challenges in the Humanoid Robot industry and focused on developing actuators optimized for domestically produced humanoid platforms. As a result, we introduced the BCSA V4 series that has been highlighted throughout our recent communications.

The BCSA series is an actuator line based on the motion principles of cycloid gears. It has gained significant attention not only in Korea but also from companies in the United States, Canada, India, and other global markets. Among the series, the BCSA V4 RI model is one we strongly recommend. It is designed with emphasis on durability, stability, and lightweight construction, which are central trends in the rapidly growing Humanoid Robot sector.

This series is suitable for compact joints such as the robot arm and shoulder and is available in outer diameters ranging from 70 millimeters to 96 millimeters and reduction ratios from 19 to 49. The two Humanoid Robot models showcased by Bonsystems at the 2025 RoboWorld exhibition also feature the BCSA V4 RI version. Details such as maximum torque and backlash can be found on the product page.

4. Frequently Asked Questions (FAQ)

※ 참고 자료

1. 「Vision-Language Models: The AI That’s Learning to See and Speak」, Medium (2024.09.17)
2. 「Supervised vs Unsupervised Learning Explained」, seldon (2025.03.09)
3. 「What is Physical AI?」, NVIDIA (2025.10.28)