Coordinates extracted from mediapipe are always same

Question

I am trying to extract coordinates of all features from the mediapipe library using face mesh but for every image that I am testing it's giving the same coordinates. I don't understand what's wrong here. If anyone could help, it would be great!

    import mediapipe as mp
    import cv2
    import matplotlib.pyplot as plt
    # from mpl_toolkits.mplot3d import Axes3D
    import json
    import os
    from os import path

    file_name = "./json_output"
    img_base = cv2.imread("./johnny-depp-sunglasses-hat-smile-wallpaper.jpg")
    img = img_base.copy()

    mp_face_mesh = mp.solutions.face_mesh
    face_mesh = mp_face_mesh.FaceMesh(static_image_mode=True)
    results = face_mesh.process(img)
    landmarks = results.multi_face_landmarks[0]

    xs, ys, zs = [], [], []
    CONTOUR_LIST = ["FACEMESH_LIPS", "FACEMESH_FACE_OVAL", "FACEMESH_LEFT_IRIS", 
         "FACEMESH_LEFT_EYEBROW","FACEMESH_LEFT_EYE", "FACEMESH_RIGHT_IRIS", 
            "FACEMESH_RIGHT_EYEBROW", "FACEMESH_RIGHT_EYE"]

This is the main function:

def extract_landmarks(inp):
    
    img = img_base.copy()

    for landmark in landmarks.landmark:
        x = landmark.x
        y = landmark.y
        z = landmark.z

        xs.append(x)
        ys.append(y)
        zs.append(z)

        relative_x = int(x * img_base.shape[1])
        relative_y = int(y * img_base.shape[0])

        cv2.circle(img, (relative_x, relative_y),
                  radius=5, color=(0,0,255),
                  thickness=-1)
    
#     fig = plt.figure(figsize=(15,15))
#     plt.imshow(img[:,:,::-1])
#     plt.show()

    img = img_base.copy()
    
    for i in inp:
        for src_id, tar_id in i:
            source = landmarks.landmark[src_id]
            target = landmarks.landmark[tar_id]

            relative_source = int(source.x * img.shape[1]), int(source.y * img.shape[0])
            relative_target = int(target.x * img.shape[1]), int(target.y * img.shape[0])

            cv2.line(img, relative_source, relative_target, 
                    color=(255,255,255), thickness=2)

        fig = plt.figure(figsize=(15,15))
        plt.imshow(img[:,:,::-1])
        plt.show()

        result = inp
#         print(result)
        
        my_json = list(result)
        
        
#       my_ans = [{f"{CONTOUR_LIST[k]}":{'x':x, 'y':y}} for k in range(len(CONTOUR_LIST)) for i in my_json for x,y in i]
          
        my_ans = [{f"{CONTOUR_LIST[k]}":{'x':x, 'y':y}} for k in range(0, 8) for i in my_json for x,y in i] 
          

#         
#         print(my_ans, sep="\n", end="\n")
#         print("\n")
#         print("\n")
#         coordinates.append(my_ans)
#         print(my_ans, end="\n", sep="\n")
        
        if os.path.exists(file_name):
            print("Already exists!")
#             with open(file_name, 'w') as f:
#                 f.write(json.dumps(my_ans, indent=4, separators=(',',': ')))
        else:
            with open(file_name, 'w') as file:
                json.dump(my_ans, file, 
                        indent=4,  
                        separators=(',',': '))
                
        return len(my_json)

And this is the code for calling the function:

features = []

features.append(mp_face_mesh.FACEMESH_LIPS)
features.append(mp_face_mesh.FACEMESH_FACE_OVAL)
features.append(mp_face_mesh.FACEMESH_LEFT_IRIS)
features.append(mp_face_mesh.FACEMESH_LEFT_EYEBROW)
features.append(mp_face_mesh.FACEMESH_LEFT_EYE)
features.append(mp_face_mesh.FACEMESH_RIGHT_IRIS)
features.append(mp_face_mesh.FACEMESH_RIGHT_EYEBROW)
features.append(mp_face_mesh.FACEMESH_RIGHT_EYE)

extract_landmarks(features)

For every image, I am getting the same coordinates.

score 0 · Answer 1 · answered Sep 21 '22 at 09:21

0

The coordinates in json_output is fixed, look at the face mesh.

The content of json_output is the numbers in the image.

relative_x and relative_y are the coordinates relative to the width and height of the picture.

answered Sep 21 '22 at 09:21

liuyulvv

154
10

I'm not getting what you said. Exactly what do I have to do here? – Raghav Kavimandan Sep 21 '22 at 09:52
@RaghavKavimandan I tried your code, and the output of `json_output` I got was `{x:270, y:409}...{x:81, y:82}...`. If your output is different from mine, you should also release your output. – liuyulvv Sep 22 '22 at 00:19
That's what I'm saying, I'm getting the same coordinates for every image. It's similar to your output – Raghav Kavimandan Sep 23 '22 at 06:07
@RaghavKavimandan That's what I'm saying. `270`,`409`,`81`,`82` are the index of the `face_mesh`, which are marked in `face_mesh` and fixed. So your output should have been the same. You must point out what your coordinates mean. – liuyulvv Sep 24 '22 at 01:36

Coordinates extracted from mediapipe are always same

1 Answers1