Chewsy

Chewsy is a barcode scanner that answers two questions about any packaged food: how it's made, and what's in it. Point it at a barcode and, in a few seconds, you get a processing level and a nutrition summary, each with a reason.

The Problem

Food labels list ingredients and nutrients, but they don't say how far a food has been industrially processed. Researchers at the University of São Paulo proposed a four-level scale called NOVA for that: 1 is unprocessed or minimally processed, 2 is processed culinary ingredients such as oil or sugar, 3 is processed food, and 4 is ultra-processed food.

Open Food Facts is a free, open database of food products, built by volunteers who photograph packs and type in what's printed on them. It covers millions of products from around the world. Many entries have no NOVA label at all, because nobody has added one yet.

The Solution

Chewsy fills in the missing labels. Scan a barcode and it fetches that product's raw ingredients and nutrients from Open Food Facts, then a model we trained predicts the NOVA class from those inputs. The model never reads the community's label, so its answer is always its own, even when a label exists.

What It Shows

Two separate answers, on purpose. How it's made gives the NOVA class, the model's confidence, and the top SHAP features behind the call. What's in it gives NHS traffic lights for sugar, fat, saturated fat and salt. A candy bar can be ultra-processed and still low in sugar, so the two answers are never merged into one score.

Technical Implementation

Data and features

20,000 community-labelled products, 5,000 per class, cleaned to 19,998 rows. The 44 engineered features include nutrient ratios, brand and category frequency encodings, and 25 SVD components from the ingredient text. Imputation is fit on training data only, after I found and fixed a leak.

Models

Six model families were trained and compared, all tracked in MLflow. XGBoost won with a macro-F1 of 0.9465. I chose macro-F1 over accuracy because the classes are imbalanced.

All 26 tracked runs in MLflow: the six model families, the text-dropout sweep, and the packaging runs.

The model registry: version 4 of chewsy-nova carries the champion alias, and that is the version the app serves.

Experiment tracking with MLflow

Every run is logged to MLflow with its parameters, macro-F1, sparse-input metrics, and a confusion matrix. The tree models also log SHAP summary plots. Only the champion's model file is stored, because storing every candidate would have pushed the experiment history past GitHub's file size limit. The winner is registered as chewsy-nova with a champion alias, and the packaging step loads that alias to build the served model. A test confirms that the registry and the packaged file give identical predictions.

Data and model versioning with DVC

The pipeline is a DVC graph of four stages: fetch, clean, features and train. Each stage declares only the parameters it reads, so changing one value reruns only the stages downstream of it. The raw and cleaned datasets and the trained model live in a private remote, and git holds only the small pointer files. After a retrain, the new model is pinned and pushed in one step.

Serving and CI/CD

FastAPI loads the frozen model once, and the Next.js frontend calls it over HTTP. Every push to GitHub runs 129 tests through GitHub Actions. Pushes to main also build a Docker image, push it to Docker Hub, and trigger a Render deploy through its API, so the live service updates without a manual step. The frontend is hosted on Vercel. The API has structured logs, a /metrics endpoint, and a KS drift check against the training data.

The Challenge

Roughly three in four live Open Food Facts records have no ingredient list or categories. On that kind of input, the first model scored 0.41. Training with text-dropout augmentation raised it to 0.77 for a cost of about 0.002 on complete records. A thin record now shows a "not enough information" state instead of a confident class.

Key Takeaway

A model's score only means something if you measure it on the inputs it will actually see. Reporting the complete-record number alone would have hidden the problem.