a1tD0000003mEOQIA2

Course: Graphical and Python programming using NAO
4: Let's Dance

  • 9-12 grade
  • Intermediate

Lesson Description:

This lesson will introduce a new programming concept: time-driven programming. It will emphasize the differences in sequential programming. In order to illustrate this concept, you will program the NAO robot to dance. 

Objective:
Make the NAO dance as a series of movements programmed with:
Graphical interface (beginners) and Python (advanced).

 


 

Standards Covered

CCSS.ELA-LITERACY.RST.11-12.10

By the end of grade 12, read and comprehend science/technical texts in the grades 11-CCR text complexity band independently and proficiently.

CCSS.ELA-LITERACY.RST.11-12.3

Follow precisely a complex multistep procedure when carrying out experiments, taking measurements, or performing technical tasks; analyze the specific results based on explanations in the text.

CCSS.ELA-LITERACY.RST.11-12.8

Evaluate the hypotheses, data, analysis, and conclusions in a science or technical text, verifying the data when possible and corroborating or challenging conclusions with other sources of information.

CCSS.ELA-LITERACY.RST.11-12.9

Synthesize information from a range of sources (e.g., texts, experiments, simulations) into a coherent understanding of a process, phenomenon, or concept, resolving conflicting information when possible.

image description

Lesson Modules


Teaching Tips:

The lesson will introduce time driven programming using NAO.  

Time-driven programming is a computer programming paradigm where the control flow of the computer program is driven by a clock. In other words, the code you run will run for a certain period of time. 

The counterpart is event-based programming, in which the program reacts to events and sequences including conditional sequences such as if-else.

In our context, Choregraphe event-based programming is simply a diagram of boxes linked together. (Examples from the lessons from before such as (Hello World, or Speech Reco). Time-based programming in Choregraphe uses a timeline. This will be discussed at length in today's lesson. 

Step 1: students are invited to think about what is a time-based programming.

Step 2: In order to spark the discussion, you can launch a pre-programmed movement ( preferably a dance).

Step 3: You can show using CLASS VIEW to the class what a keyframe is which explains why a dance is seen as time-based programming 

Then ask the students to describe what they see. You can guide them to the following concept that movement of the dance is a series of positions that the robot takes in a timely manner or order.

Time-based programming

We will introduce in this lesson the concept of time-based (or time-driven) programming and understand it by programming the NAO to dance.

Step 1: Review the previous lesson - event-based programming

In the previous lessons, we programmed our robot to do things in sequence. We had him do the following: 

  1. Say things (" Hello World ")
  2. Have a simple conversation ( " Speech Recognition ")

This is called event-based programming because the code makes operation in sequences in a cause and effect type of way. For example, when you press "start" the program starts.

How keyframes and time-based programming are related?

Step 2: Demo of a dance.

Look at the robot dancing. 

What else can you describe as event-based?
  

Step 3: Keyframes

Robot movement is a series of keyframes.


 


Teaching Tips:

 This module serves as a reminder to carefully manipulate the robot and take into account balance. 

Balancing the NAO
 
As you may have already learned from physics class, objects have a center of gravity. An object’s center of gravity affects whether its pose is stable. For rigid bodies, the object is stable if the center of gravity lies within the base of the object (see figure below).
 
  
 
For the NAO, the same concept applies. When the NAO is standing on both feet, the base lies between both feet. When the NAO is standing on one foot, the base lies only across that foot. The NAO’s center of gravity is located in its torso. Because the NAO can change its “shape” when different parts of its body (arms, legs, head) move, the center of gravity will also shift. You can create complex postures with the NAO while keeping it balanced!
 
     
 
To adjust the pose of the NAO,  make sure that the weight of the NAO is resting on both feet. Sometimes, even when both feet appear to be on the ground, all the weight of the NAO is actually resting on a single foot. Such a position is less stable and can cause the NAO’s motors to heat up faster. Also, it becomes more difficult to keep the NAO in balance, since the center of gravity must remain within the bounds of that single foot.
 
One way to test the weight distribution is to try to move the foot while stiffness is on. If there is weight resting on that foot, then it will be considerably more difficult to move it. The foot will feel “heavier” than if no weight was resting on it.
 
 


Teaching Tips:

You can find the file to download here.

CLASS VIEW has a step by step guide to creating a movement with a real robot.  

You can tell the students to be creative and create a different dance move if they want.

Basic Task: Macarena Hand Motions

Step 1: Open your Choregraphe 

Step 2: Follow the steps to create the robot dance hands-motion 


Choregraphe allows us to record body positions (keyframes) and combine them on a timeline to make continuous motions. We will learn how to create keyframes by making the NAO do the Macarena.
 
1. First, create a new timeline box and name it “Macarena”. A timeline box allows us to edit keyframes on a timeline.



2. Double click on the selected box. The timeline editor will appear, as shown below.


 
The timeline is used to view and modify individual keyframes, which appear as small rectangular black boxes on the timeline. The numbers on the timeline refer to the sequence of frames – by default, each number represents a duration of 10 milliseconds, so the NAO will take 100 ms to reach frame 10, 150 ms to reach frame 15, and so on.  This frame rate can be adjusted.
 
You can click on a keyframe to modify it. The NAO interpolates its joint positions between keyframes. Interpolation means that the NAO will move its body smoothly from one keyframe to another automatically.
 
If you connect to the NAO, you can view and save keyframes with the actual robot. If you connect to a “local Naoqi”, you can view and save keyframes using the virtual 3D NAO on the right side of Choregraphe.


3. We want to start the dance in a standing position. 

Find the best way to do so.
 
 
4. We want to save this pose as a keyframe. Right-click on frame 10 (100 ms) in the timeline, and select Store joints in keyframe -> Whole Body. Now at this keyframe, the NAO will move to the Init pose. The reason we stored this position at frame 10 is so the NAO moves to the keyframe slowly, from whatever position it was initially in. If we had stored the keyframe at frame 1 (10 ms), the NAO would jerk violently to this position and fall over. 




5. Click on a frame about 10 frames after your first keyframe.  Here we will make the robot lift its right arm with its palm down, the first move of the Macarena.

6. Click the upper part of the NAO’s right arm on the 3D view. The dialog below will appear. Click “Stiffen Chain On/Off” to disable stiffness in the arm. The icon should turn green.



7. At this stage create motions by moving the virtual robot on Choregraphe. 

If you have a real robot you can move the real robot arms and create positions that way. 

8. Next, let’s check that the keyframes are working correctly. While holding onto the robot, click on the first keyframe. The robot should lower its arm. Then click on the second keyframe. The robot should raise its arm again. Now try clicking the small arrow to the left of the timeline. Both keyframes should play in succession. This is how the keyframes will play with the proper timing when you are finished. 



 
9. Now that you’ve learned how to make keyframes, complete the rest of the keyframes for the Macarena hand motions (do not do the turning or the hip shake steps- yet). The steps are:

1) Raise right arm, palm down
2) Raise left arm, palm down
3) Put right palm up
4) Put left palm up
5) Cross right arm
6) Cross left arm
7) Put right arm on head
8) Put left arm on head
9) Put right arm on hip
10) Put left arm on hip
11) Put right arm on rear
12) Put left arm on rear

 

 
A few tips:

  1. Save your work frequently.
  2. (if you have a real robot) Be sure to hold onto the NAO when turning stiffness on or off.
  3. (if you have a real robot) If the NAO says that its motors are hot, turn stiffness off for a few minutes to let it rest.
  4. Be sure that the NAO does not hit itself with its hands. You may need to add intermediate keyframes to prevent this, especially for moving the hands from the hips to the rear in keyframes 11 and 12.
     

12. (Optional) Once the NAO can do the Macarena, try to make it dance faster.


Teaching Tips:

You can find the files for this program here

This section is more advanced and can be done if you have the time or if some students finished early.

Intermediate: Do the Macarena
 
We’ve completed the hand motions for the Macarena. Now we will add the hip shake and turn motions. The turns will involve learning how to use loops in Choregraphe.
 
1. Add the hip shake in the same way you added the arm motions, with additional keyframes. This time you will need to move the NAO’s legs. The figure below shows the NAO performing the Macarena hip shake 


 
Be sure to hold the NAO tightly around its waist when disabling the leg stiffness. Do not hold the NAO by its limbs or head. Make the NAO shift its hips left, right, and then back to the center.
 
When adjusting the position of the legs, be careful that the NAO remains balanced. As a humanoid robot with two legs, it is easy for the NAO to fall over to its side if the legs are not positioned correctly. Try out the position on yourself before adjusting the angles on the NAO. It is recommended to remove stiffness on one leg at a time so that you can rest some weight on one leg while adjusting the other. Check out the tutorial on balancing the NAO above.
 

2. Play the entire sequence to make sure it is stable and correct. If certain actions cause the robot to fall, e.g., the hip motion, slowing it down may help; otherwise, the keyframe itself may have to be modified. 
You have completed the keyframe motion segment of the Macarena. Next, we will add the quarter rotations.
 
 
3. Return to the workspace. In addition to your Macarena box, add a Stand-Up, Sit Down, Move To, and a Counter box (found under Flow Control). Connect the boxes as shown below.




4. The Counter box repeats a sequence of boxes a set number of times. Click the wrench in the Counter box to set the number of times to repeat. Change the “Final Value” to 3. The loop maintains a counter, which begins at the initial value (0) and is incremented by the Step Value (1) after each time the boxes in the loop are executed. The loop stops when the counter goes above the final value (3). How many times will the loop execute? 


 

5. Configure the Move To box to do a quarter turn.

6. Run the program by clicking the Play button. The NAO should do the Macarena.
 


Teaching Tips:

You can find the files of this module here


Advanced Task: Nodding Off
 
Next, we’ll learn how to control joint angles using Python. We’ll make two boxes, one to make the NAO nod yes and one to make it nod no. Each joint of the robot can be controlled. For this exercise, we will control the neck joints - the joints are called HeadYaw (turns the head left and right) and HeadPitch (turns the head up and down).
 

1. Create a new box to nod yes, and open the Python source file by double-clicking the box.

2. Copy the following source code into the onInput_onStart method.





The angleInterpolation method interpolates the NAO’s joint angles. This function takes four parameters:

  1. The names of the joints we are controlling (“HeadYaw” and “HeadPitch”).
  2. The final angles (in radians) to set these two joints to.
  3. The time the joints should arrive in the final position (0.5 seconds later for both).
  4. And whether the joint positions are absolute or relative (absolute is True).

    The parameter that varies is the list of angles we set the joints to. First, the head faces straight ahead, then upwards, then downwards, and finally straight ahead again. The HeadPitch joint controls the up and down motion of the robot.

    The 
    for loop in Python has the same function as the Choregraphe counter box. It repeats the code in the loop, incrementing the counter variable, i, from 0 to 1 to 2 for a total of three iterations through the loop. This makes the NAO nod three times.


    3. Make a new box, and modify the code to have the NAO shake its head left and right. You will want to change the HeadYaw joint instead of the HeadPitch. So, when setting the angles in the for loop, they should be [1.0, 0.0] and [-1.0, 0.0]. 


     

    4. (Optional) Change the speed of the nodding by adjusting times.  
     

Teaching Tips:


Additional Exercises
 

  1. Make the NAO wave its hand and say "hello" when someone says hello.
    Hint: Use a Speech Recognition box to detect “hello”, and attach the output to a keyframe motion and a Say box.
  2. Implement your own dance, and compete in a class dance-off.
  3. Use Python to control both head joints at once, and make the NAO’s head move diagonally, e.g., from the bottom-left to top-right.
  4. Implement the NAO Crystal Ball - Ask NAO a question, and have it randomly respond by saying yes or no and shaking the head in the appropriate way.
    Hint: Use a Speech Recognition box, attach both outputs to a Random Int box (found in the Math section) with range 0 to 1, and attach that to a Switch box.

 
 


Teaching Tips:

Solutions
 
Basic:

  1. What is a keyframe?
    A set of fixed positions the robot moves its joints to.

     
  2. What does the NAO do in between keyframes?
    It interpolates between the joint angles of adjacent keyframes.

     
  3. Where is the NAO’s center of gravity?
    In its torso.

     
  4. Where must the center of gravity be for the NAO to remain balanced?
    Directly above the robot’s feet (or foot) which are touching the floor.

     
  5. When creating a box in Choregraphe, how do you make it a box where keyframes can be edited?
    Change the “Box offspring” to “Timeline”.

     
  6. Why is it not suggested to set a keyframe in the first frame of the timeline?
    The NAO will jump immediately to that position and will likely fall due to the rapid motion.

 
Intermediate:

  1. What types of motions will cause the NAO to fall?
    Rapid motions which give it momentum, or motions which shift the center of gravity away from above the feet.

     
  2. Explain the behavior of a For box in Choregraphe.
    It executes the loop connected to the box multiple times.

     
  3. A For loop box has an initial value of 2, a final value of 10, and a step value of 2. How many times does the loop execute?
    5 times. The counter value will be 2, 4, 6, 8 and 10.

 
Advanced:

  1. Which joint is the HeadYaw joint and which is the HeadPitch?
    The HeadYaw is for side to side motion, the HeadPitch for up and down motion.

     
  2. What units are angles passed to the angleInterpolation function in?
    Radians.

     
  3. How many radians are 45 degrees?
    (pi)/4
     
  4. How many degrees is  (pi)/3 radians?
    60 degrees

 
 

 
 

Questions
 
Basic:

What is a keyframe?

What does the NAO do in between keyframes?

Where is the NAO’s center of gravity?

Where must the center of gravity be for the NAO to remain balanced?

When creating a box in Choregraphe, how do you make it a box where keyframes can be edited?

Why is it not suggested to set a keyframe in the first frame of the timeline?

 
Intermediate:

What types of motions will cause the NAO to fall?

Explain the behavior of a For loop box in Choregraphe.

A For loop box has an initial value of 2, a final value of 10, and a step value of 2. How many times does the loop execute?

 
Advanced:


Which joint is the HeadYaw joint and which is the HeadPitch?

What units are angles passed to the angleInterpolation function in?

How many radians are 45 degrees?

How many degrees is (pi)/3  radians?