Let’s do Face Recognition in a WorldCup Match

What is Facial Recognition?

A facial recognition system is a technology capable of identifying or verifying a person from a digital image or a video frame from a video source. There are multiple methods in which facial recognition systems work, but in general, they work by comparing selected facial features from given image with faces within a database (Wikipedia).

Facial Recognition Use Cases

There are many industries using face recognition:

• Unlock phone: This is an old one. We’ve already been using this technology to unlock our smartphones
• Prevent retail crime: How would you feel If you knew that some retail companies have a large database of people faces to match them whenever they are causing trouble? This system can identify problematic clients when they are entering in the stores
• Find missing people
• Identifying people in social media: Facebook tags is a great example here
• Diagnose diseases
• Protect schools from threats
• Track school attendance: Some schools in China are using these systems
• Validate identity in ATMs
• Find lost pets
• Recognize drivers
• Control access to sensitive areas

Artificial Intelligence Trends To Watch This Year - Data Driven Investor

Tutorial for Facial Recognition

Let’s do face recognition in a video using the Face Recognition library!

I shared a link at the end of the post so you can get this python library, too.

I will use a video from the WorldCup 2014 to detect the faces of the German team players!

Installing the Face_Recognition Python library

First, let’s install the dependencies:

!pip install face_recognition

This package is needed for the video codecs compatibility:

!pip install ffmpeg

Importing the libraries:

Import libraries
import face_recognition
import cv2

Let’s open the video file:

input_video = cv2.VideoCapture("worldcup2014.mp4")
length = int(input_video.get(cv2.CAP_PROP_FRAME_COUNT))

Make sure the resolution and frame rate matches the input video. My video has a resolution of 1280x720 and a frame rate of 25.07:

fourcc = cv2.VideoWriter_fourcc('M','P','E','G')
output_video = cv2.VideoWriter('selfie_oscars.mpeg', fourcc, 25.07, (1280, 720))

Collecting Football players pictures

For the experiment, I collected some sample pictures from 11 german football players:

Bastian Schweinsteiger is my fav!

Now, let’s load the selected pictures, so the machine can learn to recognize them in the video:

lahm_image = face_recognition.load_image_file("lahm.png")
lahm_face_encoding = face_recognition.face_encodings(lahm_image)[0]
ozil_image = face_recognition.load_image_file("ozil.jpg")
ozil_face_encoding = face_recognition.face_encodings(ozil_image)[0]
mueller_image = face_recognition.load_image_file("mueller.png")
mueller_face_encoding = face_recognition.face_encodings(mueller_image)[0]
neuer_image = face_recognition.load_image_file("neuer.jpg")
neuer_face_encoding = face_recognition.face_encodings(neuer_image)[0]
klose_image = face_recognition.load_image_file("klose.jpg")
klose_face_encoding = face_recognition.face_encodings(klose_image)[0]
kroos_image = face_recognition.load_image_file("kroos.jpg")
kroos_face_encoding = face_recognition.face_encodings(kroos_image)[0]
boateng_image = face_recognition.load_image_file("boateng.jpg")
boateng_face_encoding = face_recognition.face_encodings(boateng_image)[0]
howedes_image = face_recognition.load_image_file("howedes.jpg")
howedes_face_encoding = face_recognition.face_encodings(howedes_image)[0]
hummels_image = face_recognition.load_image_file("hummels.jpg")
hummels_face_encoding = face_recognition.face_encodings(hummels_image)[0]
krammer_image = face_recognition.load_image_file("kramer.jpg")
krammer_face_encoding = face_recognition.face_encodings(krammer_image)[0]
bastian_image = face_recognition.load_image_file("bastian.jpg")
bastian_face_encoding = face_recognition.face_encodings(bastian_image)[0]

These are all the faces that we will try to find in our video:

known_faces = [
    lahm_face_encoding,
    ozil_face_encoding,
    mueller_face_encoding,
    neuer_face_encoding,
    klose_face_encoding,
    kroos_face_encoding,
    boateng_face_encoding,
    howedes_face_encoding,
    hummels_face_encoding,
    krammer_face_encoding,
    bastian_face_encoding
]

Let’s initialize some variables:

face_locations = []
face_encodings = []
face_names = []
frame_number = 0
while True:
    # Grab a single frame of video
    ret, frame = input_video.read()
    frame_number += 1
# Quit when the input video file ends
    if not ret:
        break

Now, let’s change from BGR to RGB color:

# Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
    rgb_frame = frame[:, :, ::-1]

These lines will find all the faces and face encodings in the current frame of the video:

    face_locations = face_recognition.face_locations(rgb_frame)
    face_encodings = face_recognition.face_encodings(rgb_frame, face_locations)
face_names = []
    for face_encoding in face_encodings:
        # See if the face is a match for the known face(s)
        match = face_recognition.compare_faces(known_faces, face_encoding, tolerance=0.50)
       
        name = None
        if match[0]:
            name = "Philipp Lahm"
        elif match[1]:
            name = "Mesut Ozil"
        elif match[2]:
            name = "Thomas Muller"
        elif match[3]:
            name = "Manuel Neuer"
        elif match[4]:
            name = "Miroslav Klose"
        elif match[5]:
            name = "Toni Kroos"
        elif match[6]:
            name = "Jerome Boateng"
        elif match[7]:
            name = "Benedikt Howedes"
        elif match[8]:
            name = "Mats Hummels"
        elif match[9]:
            name = "Christoph Kramer"
        elif match[10]:
            name = "Bastian Schweinsteiger"
 
        face_names.append(name)

The next code will label the results and draw a box around the recognized faces:

# Label the results
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        if not name:
            continue
# Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
# Draw a label with a name below the face
        cv2.rectangle(frame, (left, bottom - 25), (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6), font, 0.5, (255, 255, 255), 1)

This line will generate all the frames with the labeled faces and add them to the output video file:

# Write the resulting image to the output video file
    print("Writing frame {} / {}".format(frame_number, length))

We can now check the final video!

# All done!
input_video.release()
cv2.destroyAllWindows()

This is the result of using the Face_Recognition Python library:

Use this API and share your results!

Resources:

Facial Recognition API for Python

ageitgey/face_recognition