Alon Yakoby
All work
AI match analysis · in progress

Turning Match Video into Events, Positions and Players

Taking a recording or a live stream and detecting events, player movement and player identity with YOLOv8, my own training data and an MCP-connected vision model.

  • Computer Vision
  • YOLOv8
  • Label Studio
  • MCP
  • Vision Models
  • Sports Tech

What it will look like

The app I’m building towards: match video with detections, score and a live minimap, an event feed that jumps to each moment, clips, and every player’s moments one click away.

A frame from an amateur football match with players, a referee and the ball
#7 · 0.93#9 · 0.91#5 · 0.96#11 · 0.88#4 · 0.90REF · 0.97ball · 0.71
1st12:22HOME1 – 0AWAY
12:22 / 54:32 ● Stream or recording
Player moments
Concept preview with sample data on a frame from my own footage. Click an event to jump to it, or a jersey to see that player’s moments. Not the shipped product yet.

Overview

Systems like Veo put AI cameras at the pitch. I’m building the analysis side as software: take an existing recording or a live stream from any camera, and turn it into match data automatically.

The goal: detect shots and other key events, track where players move on the pitch, and identify exactly who each player is when the squad has been preloaded, to make our league feel professional without professional budgets.

It builds directly on the federation platform, which already knows every team, squad and fixture. The preview at the top of this page shows where it’s heading.

Training data

Label Studio project 'Goodfield Yolo': a frame of an amateur football match with bounding boxes around players and a referee, the class palette blank, player, goalkeeper, referee, ball and goal, and a frame counter 1537 of 98,834
  1. Players labelled frame by frame
  2. Referee as its own class, so officials never count as players
  3. Ball the hardest class: small, fast, often hidden
  4. Frame 1,537 of 98,834 a full match is close to 100,000 frames
  5. Six classes player, goalkeeper, referee, ball, goal and blank
My Label Studio project for the detection model, with footage from an amateur match.

Detection starts with good labels. I annotate match footage in Label Studio with six classes, then train YOLOv8 on our own pitches, kits and camera angles instead of relying on generic sports datasets.

How it works

  1. Recording or stream
  2. Frame sampling
  3. YOLOv8 detection
  4. Tracking
  5. Pitch positions
  6. Events
  7. Player identity
  8. Clips and stats

Target pipeline, from video to match data.

DetectionYOLOv8, trained on my Label Studio annotations: players, goalkeepers, referees, ball and goals
BackendDetections flow into my backend, where they can be joined with fixtures, teams and squads
MCP + vision modelAn MCP server gives a vision model on the stream access to tools and match context, so it can reason about what is happening, not just what is visible
EventsShots and other key moments, turned into timestamps and clips
PositionsWhere players move, mapped from the camera view onto the pitch
IdentityWho each player is, using preloaded squad data such as jersey numbers

Detection, labelling, backend and MCP are in place; events, positions and identity are what I’m building now.

How I’m approaching it

Use the data we already have

The federation already stores every squad, jersey number and fixture. Identity becomes a matching problem against known players, not open-ended face recognition.

Specialised models first, a general model on top

YOLOv8 does the fast, frame-level work. The vision model, reached through MCP, handles context and judgement, the same split I use for AI match previews: compute what you can, let the model decide only what it must.

Works with any camera

Recordings and streams from ordinary cameras, so clubs don’t need dedicated hardware.

Where it stands

  • In placeLabel Studio annotation project, YOLOv8 connected to my backend, and an MCP server with a vision stream model.
  • In progressShot and event detection, player tracking and pitch positions, and identifying players from preloaded squad data.
  • NextAutomatic clips and per-player stats, published to the league website next to live scores and AI match previews.

Relevant skills

  • Computer Vision
  • YOLOv8
  • Label Studio
  • Data Annotation
  • Model Training
  • Object Tracking
  • Video Streams
  • MCP
  • Vision-Language Models
  • Python
  • Backend Integration
  • Sports Tech