getyourlook.streamlit.app

Here is a Minimal Viable Product - a simple WebApp where user can take/upload photo and receive haircut recommendations.

Architecture

diagram was made with excalidraw.com

All models and files are cached to save resources and for the app to have quick response. I used a Multi-task CNN¹ as a face detector and a face shape classificator to get the shape of the face.

Future improvements

Apply style to the photo

As it is a MVP, the recommeded haircuts are in the form of images manually generated with stability-ai. To make WebApp more user-oriented images should be generated from the input image and recommendation prompts, so the user can see not some random images, but all recommended haircuts applied to his/her image.

Men haircuts

Right now only female haircuts are shown and the model was trained only on female celebrities. The model should be retrained also with male faces, and haircut recommendations should include more variety of haircuts.

Hair type

It would be cool to take into account hair type, because if the user has curly hair, why recommend a haircut with straight hair, also some haircuts look differently depending on the thickness of hair.

Sun-glasses shape

Face shape can help not only with a haircut, but also with the shape of sun-glasses or how to apply a make-up.

Color palette

Ideally the face also is analysed for skin/eyes/hair color. The corresponding color palette is recommended based on 'spring', 'summer', 'autumn' or 'winter' types. For this task additional model should be trained or current model should be changed to the MTCNN.

Camera calibration and distortion

It seems that shape classification also depends on camera distortion. As all users have different cameras it is impossible to apply some random hardcoded undistortion. Maybe camera information can be retrived from the image and matrices for undistortion can be generated.

Footnotes

1. (Implementation) Zhang, K., Zhang, Z., Li, Z., and Qiao, Y. (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10):1499–1503. ↩