Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to track eye pose? #9

Open
lucasjinreal opened this issue Mar 15, 2022 · 8 comments
Open

how to track eye pose? #9

lucasjinreal opened this issue Mar 15, 2022 · 8 comments

Comments

@lucasjinreal
Copy link

Hi, I found the eye's pose doesn't make the model animate. Anyway to support eye pose correctly?

@mmmmmm44
Copy link
Owner

Ensure that there is data sending from the python to the unity.

If you are using unity chan model, check the ear_min_ratio and ear_max_ratio. The values are the window of the eye blinking animation. The ear_min_ratio sets the lower-bound of the animation (i.e. the value with eyes completely closed), while the ear_max_ration sets the upper-bound of the animation (i.e. the value with eyes completely opened).

Also, for unity-chan model, you have to drag the EYE_DEF and EL_DEF from the auto-blink script.
image

If you are using live-2d model, the name of the variables are the same, with the same meaning.

If you plan to use your custom model, you have to edit the live-2d script or unity-chan script to suit your model.

@lucasjinreal
Copy link
Author

not eye blink, but eye ball's movement. current doesn't support right?

@mmmmmm44
Copy link
Owner

Yes, the unity-chan model does not support eye-ball movement.

However, other 3D models may support that. It depends on the model itself

@lucasjinreal
Copy link
Author

@mmmmmm44 I am deploying a model to make eye balls can move

@mmmmmm44
Copy link
Owner

I see.

So you are deploying a custom 3D model which supports eye-ball movements right?

You can check out the code in Live2D model (HiyoriController.cs the function IrisMovement), copy that to your modified UnityChanController.cs script, change / add the corresponding variables to map with your custom model.

Also, if you want to play with the data loading/ saving feature, then remember to edit the UnityChanPref.cs class. Basically it's like a JSON data class. Add the corresponding variables you want to save.

@lucasjinreal
Copy link
Author

@mmmmmm44 thanks, what's the pref meaning here?

@mmmmmm44
Copy link
Owner

mmmmmm44 commented Mar 19, 2022

The save load data function is based on the Unity tutorial about using JSON to save and load user preferences (i.e. the setup of the unity application, the detection parameters). The explanation of files applys to both live2D and unity-chan project.

The tutorial: https://youtu.be/uD7y4T4PVk0?t=346

There are three files involved in the save load preference system

  1. SaveDataManager.cs
  2. FileManager.cs
  3. xxxPrefs.cs (xxx = UnityChan or Hiyori, depending on your project)

The first one contains two functions, WriteToFile(string, string) and LoadFromFile(string, string). They are for writing/ reading the json file to/ from the computer

The second one contains two functions, SaveJsonData(...) and LoadJsonData(...). They are for saving/ loading the json data into the corresponding variables of the controller programs, such that the user does not have to set the parameters again when opens the application again.

For saving JSON file we need a class to wrap the datas to be saved/loaded, and that is what the third files contain, a class to wrap the parameters.

In both of the xxxController.cs programs, there are two functions namely PopulateSaveData(xxxPref) and LoadFromSaveData(xxxPref), which after pressing the Load/ Save button in the Setting panel, the program will load the JSON file to override the current parameters of the detection/ save the current parameters to a JSON file.

In the UISystem.cs scripts, there are two functions, SaveData() and LoadData(), which is called if you press the save/load button in the UI pannel.

[EDIT]
The save/load feature works both in PlayMode and standalone application.

I hope I answer your question.

@redearly123
Copy link

@mmmmmm44 I am deploying a model to make eye balls can move

Hi, have you found any 3D models that support eyeball movement?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants