Cup Up or Cup Down: Prototyping Deep-Learning Vision for Dried Tomato Sorting

A dried fruit processor dries tomato halves on large trays, cut side up. Halves that land cup down or on edge do not dry properly, so somebody has to walk the line and flip them by hand. The plan was a robot flipper - but before anyone builds a robot line, there is a cheaper question to answer: can a camera even tell which way a tomato half is sitting? This is the story of how we scoped and built that prototype, and why we think feasibility prototypes are a discipline of their own.

Rank the risk, then spend against it

The expensive part of the eventual machine was the robotics. The uncertain part was the vision. Dried and semi-dried tomato halves are about as unfriendly a vision subject as exists - no two are the same size, shape, or color, the surface is glossy and wet in places and wrinkled in others, and a cup-down half looks a lot like a cup-up half that has collapsed. In 2019, deep-learning vision for factory automation was young. Cognex ViDi - now VisionPro Deep Learning - was the first serious commercial package, and it was exactly the right tool class for a problem that could not be described with rules.

So the project was structured to spend money strictly in order of risk. Nothing got purchased that did not answer the current riskiest open question.

Step zero: prove it on the bench before you quote it

Before the proposal was even written, we ran an offline study in our own shop: cherry tomatoes on a scaled-down tray, a borrowed camera, and a loaner ViDi training license. Cherry tomatoes are not the product, and a bench is not a plant - but if the classifier could not separate cup up from cup down under ideal conditions, the whole concept was dead and nobody had bought anything. It worked, and that result is what justified writing a phased proposal at all.

Phase 1: the plant is the variable

The bench proved the algorithm class. It proved nothing about the plant. Phase 1 moved the experiment into the production environment, because ambient light is the thing that kills food-plant vision systems - sunlight through doors, shift-to-shift lighting differences, washdown, product residue on everything.

The phase 1 milestones were deliberately boring: build a camera and light mounting structure, set up a trigger sensor, lock the image acquisition parameters, and then gather images - across all ambient light conditions, across shifts, and before and after cleaning and maintenance procedures. Then train the model on those images and evaluate it against more of them. The deliverable was a number, not a machine.

Two deliberate cheats kept it small. One camera imaged half the tray, because the second camera answers no new question - if it works on half a tray it works on a whole one. And the training license and PC were loaners, returnable if the answer came back no.

Build the imaging chain for real, fake everything else

The parts that touched image quality were production grade from day one: a 5MP monochrome GigE camera (an Allied Vision Mako-based Cognex unit) behind a Moritex 12mm machine-vision lens, in an IP65 enclosure, over PoE. Diffuse LED panel lighting with plastic diffuser sheet, plus sheet-metal shrouding against ambient light - the shrouding was its own line item, and it earned it. An ifm diffuse reflection sensor supplied the trigger. The support electronics were the usual honest panel: a Hoffman stainless enclosure, a PULS 24V supply, Eaton breakers and operators, and a Patlite signal tower.

Everything else was cheerfully temporary. The first camera bridge quoted was a stage-lighting truss kit - the kind bands hang par cans from - before we switched to mounting on an existing plant frame, and only later designed a proper welded stainless frame. The deep-learning box was a consumer Dell desktop with a GTX 1070 gaming GPU. The purchase order trail includes a plastics distributor for diffuser sheet, buckets from Home Depot for handling product samples, and a rented truck for the install run. None of that would survive in a production machine, and none of it needed to. A feasibility prototype that looks like a production machine has spent money answering questions nobody asked.

Design the robot interface before the robot exists

The robot was coming from a separate mechanical integrator, later. Rather than wait, we defined the entire handoff as a one-page register map on the vision system's EtherNet/IP interface card: status bits (system ready, trigger ready, inspection complete, results valid), command bits (trigger, acknowledge, job load), and results as a count followed by X/Y coordinate pairs in millimeters, referenced to a fixed corner of the tray. A small Allen-Bradley PLC exercised the handshake long before any robot was connected, and soak-testing that handshake for extended periods was its own milestone. If the coordinates and the handshake are proven, the robot vendor's job collapses to "go to X,Y and flip."

The classes grow when the real product shows up

Commissioning image collection ran across several weeks, and the training workspace filled with hundreds of images and dozens of dated model checkpoints - retrain, evaluate, repeat. The most important finding was not a tuning parameter, it was a taxonomy change: the real failure population was not just cup down, it was cup down and on edge. A tomato half standing on its edge is a class the bench study never produced. Follow-on retraining after the system met real production volume was quoted as a small remote-service engagement - planned for, not apologized for. On organic product, retraining is maintenance, like changing a filter.

The shape of a good feasibility prototype

Looking back, the scoping rules that made this work:

• Spend in order of risk. The classifier was the risk; the robot could wait.

• Prove the algorithm on a surrogate before proposing field work.

• Move to the real environment as soon as possible - the plant, the shifts, the ambient light are the actual test.

• Build the imaging chain for real. Fake the structure, the PC, and the product handling.

• Halve the system wherever symmetry allows. One camera on half a tray answers the whole question.

• Put the acceptance number in writing, and stop the line to measure ground truth physically.

• Gate every phase. Loaner licenses and returnable hardware mean a "no" is cheap, and a "no" you can afford is the entire point of a prototype.

The distilled methodology - what to prove, what to fake, how many images organic variability really needs, and the kill criteria we use - is in the knowledge file below, formatted to drop straight into your AI assistant. Building feasibility prototypes like this is our day job; if you have a product everyone says is too variable for vision, get in touch through the contact form.

Downloads

Free, no signup. The knowledge file is designed to be dropped into an AI assistant's context. No customer information in any of it.

• knowledge.md - feasibility-prototype methodology for vision on variable organic product (for your AI assistant)

• Redacted drawing: camera support frame weldment (PNG)

• Redacted drawing: camera mount plate (PNG)