[Touchdesigner]Even new graduates can do hand tracking! With leapmotion

It’s called detecting the shape of a hand in real space.
It is a technology that is quite familiar in the AR and VR circles.
If you’re not familiar with it, it’s easier to understand if I actually show you.
So…
The tracking is pretty good, right?

version

touchdesigner 2020.22080
windows10 Pro

leap motion

What is leap motion?

(Citation: https://www.yama.info.waseda.ac.jp/ja/research/leap-browsing-api)

It’s a device that enables hand gesture control.
If you don’t have one, please borrow one, buy one, or whatever. Now.
You won’t regret it!
(This is not a project from Ultraleap)
Please connect leapmotion. Maybe there is a possibility of downloading a driver or something.
Since I said so much, when you open the project, first
Let’s take out the leap motion CHOP.

When you hold your hand over the leapmotion, various information about your hand will be returned as numerical values (if there is no response, the leapmotion is not connected properly or the driver is not installed).

We will control these values.
Then connect the math CHOP and change the value to Mult-Add → Multiply → 0.01 (the value is too large for the coordinates, so make it smaller).

Next, connect the select CHOP. To select the finger data, enter Channel Names as

hand0/finger[0–5]/next_*:t[xyz]

to select the finger data. The names may be different depending on your mac or version.
In that case, click the black ▶ button next to the channel names to find a similar name and modify it accordingly.

If you actually look at it, you’ll see the names of the finger joints and other things, but for this time
hand0(of one hand)
finger[0–5](from the thumb to the little finger in order)
next_*(all the fingertips, starting from the third joint at the back)
t[xyz](xyz coordinates of each)
That’s how it works. For your reference

(Citation: https://ameblo.jp/yamatokawashima/entry-12436212230.html)
My handwriting is messy (laughs).
Now, connect shuffleCHOP, and set Method to Sequence Every Nth Channel, and N Value to 3.
Now each coordinate is summarized in x,y,z.

Let’s rename CHOP to set To to t[xyz] and connect the null CHOP.
Next, let’s create a line SOP to create a model. Set Number of Points to 4. This will create three joints.

Next, connect the copySOP and set Number of Copies to 5. There are five fingers, right?

Then, connect the chop to SOP and put the null-chan you just created into the CHOP (drag and drop).

Why are you doing this?

The thumb has only two joints, so it is uselessly erasing the detected points.
If you select the point number in the display options on the groupSOP, the points you want to delete will be displayed in yellow.

Now, let’s connect null and attach geometryCOMP and constantMAT.
Now, when you hold your hand over the leampmotion, five fingers will appear.

Then, add ChannelNames from above.
The wrist is the wrist, isn’t it?
Connect these three in order with mergeCHOP.
After that, follow the same procedure as for the fingers.
shuffleCHOP(Method:Sequence Every Nth Channel, N Value:3 )
→ rename
renameCHOP(To:t[xyz])
→ nullCHOP
nullCHOP
To create a palm model, we will use a circleSOP.
Place a circleSOP and set Divisions to 6.

We will connect the fingers at the base and the wrist later, so 5 fingers + 1 wrist = 6.

After that, connect the choptoSOP and D&D the CHOP for the palm you just created in the CHOP field.

D&D!

After that, as with the fingers, use geometryCOMP from nullSOP and set MAT to phong. The color is up to you.
You don’t have to do this, but I think it looks better if you do.
Connect the leapmotionCHOP to the selectCHOP and change the CHOP name to tracking.
And change the Channel Names to
and change the Channel Names to

Then, place the sphereSOP and change the primitive type to polygon.

Next, connect the transformSOP and change the Uniform Scale to
for Uniform Scale!
Then, connect nullSOP and geometryCOMP, and attach a palm phong as MAT.

Here, we will use instancing to place it.
First, bring up SOPtoCHOP and D&D the nullSOP of the finger in the SOP field.

Then, set the instancing of geometryCOMP to On, and D&D the SOPtoCHOP to the Default instance OP.
Then, set TranslateX to tx, TranslateY to ty, and TranslateZ to tz.

Instancing complete!

Let’s place CameraCOMP and LightCOMP.
Parameters are here.

This is how I was able to make my hand appear in the virtual space.
It’s a bit long, but what do you think?

Hand tracking is almost the same, but it also detects if the hand is being held.
It’s very simple, though.

The leapmotion is a precision instrument, and the liquid crystal is easily broken, so please be careful.
When I was a student, I cracked the LCD of a leapmotion that I borrowed from my seminar professor, and I got down on my knees to apologize (apologize, apologize, apologize!). (Apologies, apologies, apologies!).

I don’t want you all to feel this miserable!

Please be careful.

I know I sounded like I could do hand tracking as a fresh graduate, but the bottom line is that I am indebted to my predecessors for the best technology. Let’s be grateful for that and use it to the fullest.

Also, I’d like to reiterate that this is
This is not an advertisement for leapmotion.
Don’t worry.
Let’s all play touch! (Inazuma Eleven style)