Can visual search identify a B2B part from a photo?

Sometimes. A legible label can be read with OCR and matched against the part dictionary, which works well when the dictionary is complete. A photo of an unlabeled part usually narrows the search to a family of similar items, and a person still has to confirm the exact part number.

Are CLIP or SigLIP models good enough for part numbers?

No. Contrastive image-text models encode appearance and coarse meaning. They do not reliably read exact characters, counts or serial numbers, so use OCR for text and treat the embedding as a similarity signal.

Is image-to-text search cheaper than image embeddings?

It depends on traffic and catalog size. Embedding a catalog is a one-off cost plus re-embedding when the model changes, while a vision-language call happens for every buyer photo. Compare the cost per successful identification, not the cost per call.

Can buyer photos be sent to a third-party AI service?

Check the provider’s data processing terms, your privacy notice and your customer contracts before any image leaves your systems. Photos can show people, documents or confidential equipment, so remove metadata and limit how long photos are kept.

Visual search for B2B shops

Visual search lets a buyer start from a photo instead of a typed query: a picture of a nameplate, a photo of a worn part, or a snapshot of a shopping list. For a B2B shop it is an attractive feature and an oversold one. A photo is not one problem. A label is mostly an OCR problem with a catalog constraint, an unknown part is an image similarity problem, and a worn part is a lookup in which the image only offers a hint. No single model handles all three well, and for some of them the text search you already run is the better tool.

This article covers the image modality end to end: what each job needs, how image embeddings and image-to-text search differ, how to read part numbers from a photo, how to combine image results with text and filters, what it costs, how to evaluate it, and when not to build it. It picks up where semantic product search for B2B stops, at text retrieval, and it assumes you have read how to choose an embedding model. Vector storage is covered in the pgvector article. Platform and price facts are as of October 2026.

What is visual search good for in a B2B shop?

Visual search pays off for three jobs. A buyer photographs a label or nameplate and needs the exact part number. A buyer photographs a part whose name they do not know and wants similar items. A field technician photographs a worn or broken part and wants the spare part it belongs to. Each job depends on a different signal: characters for the first, shape and appearance for the second, and context plus a spare-part catalog for the third.

The short answer is that visual search is worth building for the label job and the unknown-part job, and only once the typed-search path works well. It is weak as a replacement for a part number. A photo of a worn bearing rarely shows the detail that separates two sizes that look almost identical, so the result list needs a person to confirm the match, and the shop should say so on the page.

Job	Primary technique	Text path enough?	Metric to track
Label or nameplate photo	OCR, then match against the part dictionary	Often, if typed numbers are matched fuzzily	Exact-match rate after OCR
Whole part, name unknown	Image embedding and vector search	No; this is where images add value	Top-5 and recall@10
Worn or damaged part	Image-to-text hint, then reviewed lookup	No; the image is only a hint	Confirmed-match rate
Shopping list or cart photo	Vision model extracts lines, catalog match per line	Partly	Lines prefilled correctly
Packaging with a barcode	Barcode decoding and GTIN validation	Yes; a barcode decoder beats any model	Decode success rate

How do image embeddings make a photo searchable?

An image embedding model maps a picture to a vector, and text to vectors in the same space, so a photo can be compared with product vectors by cosine similarity. Most current options come from contrastive pre-training. CLIP learned by predicting which caption belongs to which image, using 400 million image-text pairs collected from the internet. It matched the original ResNet-50 on ImageNet in zero-shot transfer without using any of ImageNet’s 1.28 million training examples (Radford et al.). SigLIP 2 keeps that approach and adds objectives for semantic understanding, localization and dense features. It is released as a multilingual family from 86M to 1B parameters, with a dynamic-resolution variant (Hugging Face).

No single model wins everywhere. MIEB covers 38 languages, 130 tasks in eight categories and 50 models, and reports that no single method dominates across task categories, with clear weaknesses in some areas (MIEB). Benchmark on your own products rather than picking the leaderboard winner.

What the space encodes is coarse appearance and type: shape, colour, material, and whether something looks like a bearing or a hose fitting. What it does not encode reliably is exact characters, counts or a serial number. A nearest-neighbour search for a photo of one bearing can return a visually similar bearing with a different bore, and the distance will not tell you that. Design around that limit, because tuning will not remove it.

The pipeline is short. Embed each product image once and store the vector next to the model version that produced it. At query time, embed the buyer photo, run an approximate nearest-neighbour search, then filter. In PostgreSQL, pgvector provides vector and halfvec columns with HNSW and IVFFlat indexes. Its README states that HNSW has better query performance than IVFFlat, but slower builds and higher memory use (pgvector).

Is image-to-text search simpler than embeddings?

Often, yes. The shortcut that works well in 2026 is to turn the image into text and reuse the text search the shop already has. A vision-language model looks at the photo and returns a search term or product names, and the normal search does the rest. Spryker’s Search by Image follows this shape. It analyzes the uploaded photo, derives a search term, and redirects the buyer to the catalog results by default, or to the first matching product page. It accepts JPEG, PNG, WebP and GIF. Its documentation names two cases: identifying a worn or defective part on machinery on site, and identifying items when labels, barcodes or packaging are missing or unreadable (Spryker).

The trade-off is that the text step inherits every weakness of text search, and a wrong term gives the buyer no visible signal. Show the derived term, let the buyer edit it, and display the matched term next to the results, so a wrong guess costs one click to fix.

Spryker’s Visual Add to Cart applies the same idea to a shopping list on the Quick Order page. The model returns recognized products and quantities. Each name is matched against the catalog with a configurable text-similarity threshold, which defaults to 30% word overlap, and the form is pre-filled with SKUs and quantities. The defaults are JPEG or PNG, 10 MB at most, and 20 products per image, and the system prompt is editable in the Back Office (Spryker). Tune the threshold first: too low and a wrong SKU receives a quantity, too high and the buyer sees a list of misses.

The image-to-text path as Spryker documents it: the photo becomes a search term or product lines, and the catalog match decides the SKU.

The vector-index variant replaces the middle step with an embedding lookup. Both variants end in the same catalog and the same result page, so the rest of the shop does not change.

How do you read a label or part number from a photo?

Read the characters with OCR, then do not trust the raw output. Cloud OCR is the fastest route to a working prototype. Google Cloud Vision bills per feature applied to an image: the first 1,000 units per month are free, then $1.50 per 1,000 units up to 5 million units, and $0.60 per 1,000 above that, as of October 2026 (Google Cloud). A self-hosted engine makes sense when photos contain data you cannot send to a third party, or when volume makes per-unit pricing the wrong model. The cost then moves to engineering and GPU time.

The errors are predictable. Fonts confuse O and 0, I and 1, and B and 8. Embossed metal plates produce glare, and a label photographed at an angle returns half the string. Two checks help. If the number has a standard structure, validate it: a GTIN carries a check digit, and python-stdnum implements the EAN, UPC and GTIN calculation (python-stdnum). Manufacturer part numbers usually have no check digit, so the practical correction is matching against the catalog. SymSpell performs dictionary lookup within a set edit distance and is MIT-licensed, so it can run against your own part list (SymSpell). Return the top candidates with confidence scores and let the buyer pick, instead of accepting the first guess.

How do you combine image results with text and filters?

Pure image similarity returns things that look right and cannot be bought. The quality lever is the attribute filter: category, thread standard, material, availability, approved supplier. Retrieve candidates from more than one signal, the image vector, a text vector from any typed words and a keyword index, and apply the attribute filters inside the same query. Keyword matching matters because an exact bore size or a standard code is often the decisive token, and a vector blurs it. The two tracks below show where the image and the text signals meet.

Two tracks read one photo: the OCR track validates characters against the part dictionary, the embedding track searches the vector index with filters, and both feed one result list.

Check the model licence before a model reaches a shop. jina-clip-v2 is a capable multilingual model, but it is released under CC-BY-NC-4.0, which does not permit commercial use without a separate agreement (Jina AI). The same vendor’s multimodal reranker carries the same licence. Choose a model with a permissive licence, or obtain written permission first.

Put a multimodal reranker at the end, not the beginning. It re-scores the top 50 candidates from the first stage against the photo, so its cost scales with 50 rather than with the catalog. Measure whether it changes top-5 on your labeled set. On a small catalog it may only add latency. The query below shows the first stage with the filters applied in the same statement.

-- Candidates by image vector and attribute filters; the reranker only sees the top 50
SELECT p.sku, p.name, e.embedding <=> $1::halfvec AS distance
FROM product_image_embeddings AS e
JOIN products AS p ON p.sku = e.sku
WHERE e.model_version = $2
  AND p.category_id = $3
  AND p.is_orderable
  AND p.thread_standard = $4          -- optional attribute filter
ORDER BY e.embedding <=> $1::halfvec
LIMIT 50;

Test with the filters switched on. An approximate index combined with a selective filter can return fewer rows than the limit, so check the result count and recall under realistic filter combinations, not only on the unfiltered query.

What does it cost, and how do you measure it?

Embedding is cheap per photo, and storage is the larger item. Voyage AI’s multimodal models list $0.12 per million text tokens and $0.60 per billion pixels as of October 2026, with the first 200 million text tokens and 150 billion pixels free. Images under 50,000 pixels are charged as 50,000, and images above 2 megapixels are downsampled (Voyage AI). One million 1-megapixel product photos is about one billion pixels, or roughly $0.60 at that list price. A one-off backfill is therefore small next to engineering time. The recurring costs are query traffic and re-embedding when you change models.

Storage is arithmetic. pgvector stores a halfvec in 2 × dimensions + 8 bytes and a vector in 4 × dimensions + 8 bytes (pgvector). A million SKUs with 1,152-dimension halfvec embeddings come to about 2.3 GB of table data before the index. That is a calculation, not a benchmark, so measure your own index size. Keep one embedding table per model version, so you can run two models side by side during a migration.

Evaluate on your own labeled photos, not on a public leaderboard. No public benchmark covers industrial part catalogs, and reported OCR accuracy on nameplates varies a great deal between datasets, so a number from a paper does not transfer to your shop. Collect a set of photos for each job, label the correct SKU, and report top-1, top-5 and recall@10 separately for each job, together with the no-result rate. Add a negative set of photos of products that are not in the catalog. A search that always returns something fails that set.

When should you not build visual search?

Build the text path first. A part number typed or pasted into the search box is a fuzzy-matching problem. Elasticsearch’s fuzzy query already matches terms within a Levenshtein edit distance, with fuzziness: AUTO and limits on prefixes and expansions (Elastic). Add synonyms for old and new part numbers, a proper attribute model and search analytics: exit rate from search, zero-result queries and reformulations. Those three reports show whether images would fix a real identification failure or only a data gap.

Skip visual search when most zero-result queries are about missing products or missing attributes, because no model finds a part the catalog does not contain. Skip it when buyers rarely have a photo to hand, or when the typical photo would be taken on a phone in poor light with no readable label. Build it when the text path demonstrably fails at identification and at least one of the three jobs is frequent in your traffic.

My advice, from Spryker and Pimcore catalog work, is to start with part-number tolerance and synonyms, and to measure search exit before buying any embeddings. A vision feature on top of a weak catalog only makes the weakness harder to see.

What does a build checklist look like?

Treat the list below as the minimum for a first release.

Capture: a file input that accepts images, a preview before upload and a clear retake path.
Validate on the server: check the real file type rather than the extension, cap the size and reject anything the decoder cannot open.
Strip metadata: remove EXIF data, including GPS coordinates, before the photo is stored or sent to a model.
Queue: index product images in a background job, and embed buyer photos through the same queue when the latency budget allows.
Versioned embeddings: one table per model version, with the model name on every row, so a model change is a backfill rather than a rewrite.
No-result path: show the derived term, offer a text box and link to a category page instead of an empty grid.
Catalog-constrained OCR: match OCR output against the part dictionary with confidence scores, and never add to the cart automatically on a low score.
Privacy audit: list what buyers photograph, how long photos are kept and which providers receive them.
Evaluation gate: the labeled set, including its negative cases, runs before every model or threshold change.

On my own Nuxt site, where LLM features run, I treat model output as untrusted input: validate the format, limit the size and always keep a path for a no-result answer. The same discipline applies to a camera button in a shop. If you are scoping search for a B2B catalog and want the text path measured before an image model is added, see my B2B e-commerce work.

Visual search for B2B shops: image embeddings, part numbers and honest limits

Key takeaways