Manuscript received July 22, 2025; revised October 11, 2025; accepted October 29, 2025; published March 26, 2026.

Abstract—The core necessity for the video analysis-based application is the prediction of human face orientation. Here, we propose a unique and novel solution for predicting face orientation through a combination of facial features like the nose and lip region, along with non-facial features like the background region of the facial image. The proposed solution has two stages. The first stage, namely the Nose-Lip Segmenter (NLSeg), generates a nose-lip mask (m2) through a custom-trained U-Net model. The U-Net model segments the nose and lip region very accurately with an Intersection Over Union (IOU) score of 87.56%. The second stage generates a background region mask (m1) through two different methods. In the first method, a background region mask (m1) is constructed through the kmeans clustering and Hue-Saturation-Value (HSV) skin color model. In the second method, the background region mask (m1) is constructed through the media-pipe selfie segmentation model. The proposed algorithm blends the nose-lip mask (m2) and background region mask (m1) together to predict face orientation across left, right, and neutral directions. The efficiency of both approaches, namely (NLSeg+method 1) and (NLSeg+method 2), is studied and evaluated on six public datasets namely BIWIKinect, AFLW2000-3D, Pandora, Pointing04, Stirling/ESRC 3D face, and multi camera pedestrians video dataset obtained from École Polytechnique Fédérale de Lausanne (EPFL). The proposed work yields very accurate results in spite of challenges like diverse head pose angles, facial expressions, or improper segmentation. The accuracy is 98% when subjects are non-occluded and is above 90% in the case of partial occlusion.

Keywords—head pose classification, Hue-Saturation-Value (HSV) color space, K-means clustering, media pipe, semantic segmentation, U-Net

Cite: Shet R. Prakash and V. N. Manju, "Nose-Lip and Background Region Segmentation for Precisely Predicting Face Orientation," Journal of Image and Graphics, Vol. 14, No. 2, pp. 172-193, 2026.

Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC-BY-4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.