Home Media AI

Open-source ML system for organizing and curating 200K+ personal photo collections

Quick Takeaways

Problem: Finding “great photos” in 200K+ personal collections—combining objective technical metrics (sharpness, exposure) with subjective aesthetic judgment

Solution: Open-source ML system using computer vision, YOLO object detection, and hierarchical classification to surface best images

Philosophy: The intersection of art and engineering—translating professional photography judgment into algorithmic rules that scale

Key Insight: ML should multiply human capability, not replace it—review top 2,000 candidates instead of 20,000 images

Technical Approach: Python/SQLAlchemy, MariaDB, OpenCV/Pillow, YOLO for people/pet detection, scikit-learn for quality models

Status: Active development (43+ commits), verified on 700K+ file collections, open source on GitHub

GitHub → | Related: Photography Philosophy →

The Challenge: Finding Signal in 200,000 Images

Most families accumulate thousands of photos each year. My household averages 20,000+ annually—vacations, birthdays, pets, everyday moments. That’s over 200,000 images spanning a decade.

Every year, my wife goes through this entire collection to curate our family calendar. She’s looking for:

  • Technical quality - Sharp focus, good exposure, balanced composition
  • Aesthetic appeal - Compelling moments, good light, visual interest
  • Semantic relevance - Birthday photos for birthday months, holiday images for December, pictures featuring specific people or pets

It’s meaningful work—these images tell our family’s story. But manually reviewing 20,000 images is exhausting.

The core problem: How do you automatically identify “great photos” when “great” combines objective technical metrics (sharpness, exposure) with subjective aesthetic judgment (compelling composition, emotional impact)?

The Solution: ML-Powered Photo Curation

Home Media AI is an open-source system that combines computer vision, machine learning, and hierarchical classification to surface the best images from massive personal collections.

Core Capabilities

1. Automated Quality Assessment

  • Technical metrics: sharpness (edge detection), exposure balance, noise levels
  • Compositional analysis: rule-of-thirds compliance, negative space, subject positioning
  • Duplicate and near-duplicate detection (checksummed file tracking)

2. Rich Semantic Search

  • Hierarchical classification support (people, pets, places, events)
  • Works with any taxonomy: biological classification (World Flora Online integration), geographic hierarchies, custom family categories
  • Relationship tracking: “Find photos of [person] at [location] during [time period]”

3. Scale-Tested Architecture

  • Verified to handle 700K+ file collections
  • Files stay in original locations—metadata tracked separately
  • Relationship mapping between originals and edited versions
  • Cross-platform: Windows, Mac, Linux path support

4. Long-Term Reliability

  • Everything checksummed and versioned
  • Auditable: never moves or deletes files
  • Handles duplicate detection across drives and backups

The Goal: Multiply Human Curation

The system doesn’t replace human judgment—it multiplies it. Instead of reviewing 20,000 images, curators review the top 2,000 candidates the ML system surfaces. The human still makes the final decision, but with 90% less exhaustion.

Technical Architecture

Backend Stack

  • Python with SQLAlchemy ORM for flexible data modeling
  • MariaDB 10.11+ (UTF-8mb4 support for international text/emoji)
  • NumPy/Pandas for data processing pipelines
  • Scikit-learn for ML classification models

Media Processing & ML

  • OpenCV for image analysis (edge detection, histogram analysis, feature extraction)
  • Pillow for format handling and EXIF data extraction
  • exifread for metadata parsing (camera settings, timestamps, GPS)
  • YOLO (You Only Look Once) for real-time object detection and classification
    • Identifying people, pets, and objects in images
    • Pre-trained models fine-tuned on personal photo collection
    • Enables semantic search: “Find photos with our dog at the beach”

Development Environment

  • Jupyter Lab for exploratory data analysis and model tuning
  • Mamba for reproducible environment management
  • Phase-based development: Core discovery (Phase I) → ML models (Phase II) → Web interface (Phase III)

Database Design Philosophy

The system uses a relational database to track media and their relationships to classification hierarchies:

MediaFiles (checksummed tracking)
  ↓
MediaRelationships (originals → edits)
  ↓
ClassificationLinks (files → hierarchical taxonomies)
  ↓
TaxonomyNodes (hierarchical classification systems)

This design allows:

  • Multiple simultaneous classification systems per media item (e.g., a photo can be tagged with biological taxonomy AND family event categories)
  • Version tracking (linking RAW → edited JPG)
  • Flexible hierarchies (person/pet names, locations, dates, custom categories)

Real-World Impact

Current Use Case: Family photo archival and calendar curation

  • 200K+ images spanning 10+ years
  • 20K+ annual additions
  • Handles multiple cameras, phones, and formats
  • Preserves original files while tracking edits and versions

Potential Applications:

  • Nature photography organization - Integrate with biological taxonomies (World Flora Online, eBird)
  • Travel documentation - Hierarchical geographic classification
  • Technical equipment cataloging - Product photo organization
  • Professional portfolio curation - Quality-based filtering for client galleries
  • Digital archival projects - Long-term family/organizational media preservation

Technical Highlights

Modern ML: YOLO for Object Detection

  • Real-time object detection using YOLO (You Only Look Once)
  • Pre-trained on COCO dataset, fine-tuned on personal collection
  • Enables semantic search beyond simple keyword tags
  • Example queries: “Photos with both kids and the dog”, “Images featuring specific people at specific locations”
  • Bounding box data stored for future cropping/thumbnail generation

Scale-Tested Design

  • Verified on 700K+ file collections
  • Efficient database indexing for fast queries
  • Incremental processing (only analyze new/changed files)

File Integrity

  • Checksummed tracking prevents duplicates
  • Never moves or deletes original files
  • Separate metadata database keeps files untouched

Flexible Classification

  • Works with any hierarchical taxonomy
  • Example: World Flora Online (WFO) plant classification integrated
  • Custom taxonomies for family/personal categories

Cross-Platform

  • Native Windows, Mac, Linux path support
  • Handles network shares and external drives
  • Docker deployment for portability

Links & Resources

GitHub Repository: https://github.com/ericwait/home-media-ai

Technologies:

  • Python, SQLAlchemy, MariaDB
  • OpenCV, Pillow, NumPy, Pandas
  • YOLO (object detection), Scikit-learn, Jupyter Lab