Skip to content
Nat Lee

Keyboard shortcuts

Turn these off if they get in the way of a screen reader or voice control. The setting is saved in this browser.

1
Home
2
Work
3
Projects
4
About
5
Experience
/
Search projects
?
Show this panel

Computer Vision

Work · Infortrend

Persona Tracking

Audience Analytics for Digital Signage

A camera next to a digital sign tracks the people in front of it, guesses gender and age group, notices who is actually looking, and picks the ad that fits them. Most people, it turns out, walk past.

Built at Infortrend. The code belongs to the company, so there is no source link, and the figures come from internal experiments at the time. Read the chapter on the Work page

Role
Image Processing Software Engineer
Period
2019.10 – 2021.03
Status
Completed
Categories
Vision & imagingAI & machine learning
Company
Infortrend Technology, Inc.
Persona Tracking, screenshot 1

Overview

Built at Infortrend for an embedded digital-signage system. A camera beside the screen detects faces, follows each person from frame to frame with a track ID, estimates gender and age group, and checks whether the face is turned toward the screen. Each person's estimates are averaged over frames, so guesses made on small, distant faces improve as people walk closer. The results go into a daily SQLite database and a small HTTP API, and a web service scored every ad against the audience in front of the screen and played the best match. It ran on Nvidia Jetson edge devices next to the screens. The cover shows Open Persona, a later personal rewrite that tracks whole bodies and infers interests; it is not public.

Why I built it

Built at Infortrend for an embedded digital-signage system, so a screen could choose its ad from the people actually standing in front of it. Faces far away are small and blurry, so the system keeps refining its guess as people walk closer. Patience, implemented as an average.

What it's good for

Lets a digital sign choose its ad for the people actually in front of it, and leaves daily records of who passed and who looked. It all runs on a Jetson beside the screen and keeps compact records instead of video, so nobody has to sit through the footage.

The problem

  • Faces far from the camera are too small and blurry to judge
  • Detection, tracking and estimation had to fit on an edge device
  • One ad has to suit a mixed group of people

What I did

  • Averaged each person's estimates over frames, so early guesses are corrected as the face grows
  • Kept the whole pipeline on the Jetson beside the screen, writing compact daily records instead of video
  • Scored every ad against the audience's gender and age groups and played the best match

Results

  • Shipped an embedded ad recommendation system that reads the demographics of the crowd in front of a screen
  • Later rebuilt as a personal project, Open Persona, with whole-body tracking and interest estimates

Features

  • Face detection with a track ID that stays with each person
  • Gender and age-group estimation, averaged per person
  • Checks whether each face is turned toward the screen
  • Daily SQLite records and a small HTTP API
  • Ad selection scored against the audience in front of the screen
  • Runs on Nvidia Jetson edge devices next to the screen

How it fits together

Simplified on purpose. The real thing has more arrows and at least one config file nobody understands.

  1. Camera by the screen
    → frames to
  2. This oneJetson vision pipeline
    • Face detection and tracking
    • Gender and age group
    • Facing-the-screen check
    → writes
  3. Daily records
    • SQLite
    • Small HTTP API
    → read by
  4. Ad selection service
    • Scores ads per audience
    • Plays the best match
  • Vision & imagingAI & machine learning11 stars on GitHub

    Street Scene OCR

    Compare EasyOCR, RapidOCR and PaddleOCR on Traditional Chinese street signs, with text boxes, timings and a small public-domain benchmark.

    2026Completed
  • Spin-off · Infortrend
    Vision & imagingAI & machine learning11 stars on GitHub

    Face Wall Generator

    Tile a folder of face pictures into a seeded grid. Optionally crop faces with OpenCV’s YuNet detector before building the wall.

    2021Completed
  • Spin-off · Infortrend
    Vision & imagingAI & machine learning33 stars on GitHub

    OpenPose Wrist Heatmap

    Runs the body OpenPose model over a video and piles up where the wrists went into a heatmap. The repo name says hands; the model only ever knew about wrists.

    2020Completed