Case Study: Applying AI Safety Frameworks to Restaurant Menu Digitisation

Category: Case Studies Reading Time: 12 minutes

A Case Study in Practical AI Safety for Regulated SMB Applications

Executive Summary

• Client: EasyMenus.xyz - QR menu platform for independent restaurants

• Challenge: Ensure allergen accuracy in AI-powered menu digitisation while maintaining economic viability for a low-cost subscription product

• Stakes: Substantial fines for allergen labelling violations, potential anaphylaxis liability, criminal prosecution exposure, business reputation risk

• Engagement: Advisory on verification frameworks, regulatory compliance, and liability mitigation

• Outcome: A multi-layer verification approach designed to bring allergen labelling accuracy close to the regulatory standard of full accuracy, at a cost the subscription price point can support

Background: The Problem Domain

Company Overview

EasyMenus.xyz was founded in 2024 to address a specific pain point in the independent restaurant sector: menu printing costs and update friction.

Target Market:

• Independent restaurants (1-3 locations)

• £300K-£2M annual revenue

• Menu changes 2-4 times monthly

• Printing costs: £80-220 per menu update

Value Proposition: Convert physical menus to QR-accessible digital menus quickly and affordably, eliminating printing costs and enabling instant updates.

Geographic Focus: UK, Ireland, Belgium, Netherlands initially (English, French, Dutch language support)

Technical Approach: AI-powered menu digitisation using OCR and natural language processing to extract menu content from photos of physical menus.

The Technical Challenge

Restaurants need to digitise existing physical menus, including:

Menu Content Requirements:

• Dish names and descriptions

• Ingredients lists

• Prices (multiple currencies)

• Allergen information (legal requirement)

• Dietary information (vegan, gluten-free, etc.)

• Multilingual translations for tourist areas

Input Complexity:

• Physical menus in varying conditions (creased, stained, worn)

• Handwritten specials boards with inconsistent legibility

• PDF menus with varying quality and embedded fonts

• Multi-column layouts with complex reading order

• Mixed languages on single menu (common in Brussels, Belgium)

• Decorative fonts and non-standard typography

• Photos taken in poor restaurant lighting with reflections

Expected Processing:

• Time limit: a few minutes per menu, matching the product's fast-setup value proposition

• Cost constraint: low enough per menu to preserve margin at the subscription's low monthly price

• Accuracy requirement: as close to full accuracy on allergen information as achievable (regulatory compliance)

The AI Risk Profile

Menu digitisation appears to be a low-stakes application - it's just extracting text from images. But the regulatory context transforms it into a high-consequence deployment:

Regulatory Requirements:

• EU Food Information Regulation 1169/2011: Mandatory accurate allergen labelling for all food businesses

• FSA Enforcement Powers: Unlimited fines, up to 2 years imprisonment for violations

• Zero tolerance policy: No allowance for "AI did its best" or "usually accurate"

Legal Consequences of Errors:

• Fines: Allergen labelling violations under EU and UK food law carry substantial financial penalties

• Criminal prosecution: Restaurant owners face personal criminal liability

• Civil liability: Wrongful death claims from anaphylaxis incidents

• Business closure: Reputational damage often forces closure even without formal penalties

The risk in outline: A mislabelled allergen on a digitised menu is not a cosmetic error. Where a digitisation service misreads or drops an allergen warning, the exposure runs from regulatory fines through civil liability to, in the worst case, a fatal allergic reaction. That risk profile is what makes verification design non-negotiable, regardless of how routine the underlying task looks.

The Challenge: How do you deploy AI for menu digitisation in an environment where a single error can cause death, destroy a business, and trigger criminal prosecution - while maintaining economics that work for a low-cost restaurant subscription product?

The Engagement: Advisory Approach

VerityAI's engagement with EasyMenus followed a structured advisory process focused on AI safety frameworks adapted for SMB economics and regulatory compliance.

Phase 1: Risk Assessment and Failure Mode Analysis (Weeks 1-2)

We began by systematically mapping every way the AI system could produce incorrect allergen information.

Failure Mode Taxonomy

1. Extraction Stage Failures

Input Quality Issues:

• Poor image quality (blurry, low resolution, poor lighting)

• Physical damage to source menu (tears, stains, water damage)

• Reflections and glare from laminated menus

• Shadows and uneven lighting in restaurant environment

OCR Technical Failures:

• Character misrecognition (especially handwritten content)

• Reading order errors in multi-column layouts

• Missed text regions (incomplete extraction)

• Decorative font confusion (artistic typography)

• Language detection errors (mixed-language menus)

A meaningful share of menus will hit at least one of these extraction issues in a way that touches allergen information, particularly where source menus are handwritten or in poor condition.

2. Classification Stage Failures

Named Entity Recognition Failures:

• Missing allergen keywords in ingredient lists

• False negatives on allergen-containing dishes

• Context misunderstanding ("nut-free" vs "contains nuts" vs "may contain nuts")

• Regional terminology variations (e.g., "groundnut" vs "peanut" vs "cacahuète")

Semantic Understanding Failures:

• Implicit allergen content not explicitly stated ("butter sauce" implies dairy)

• Cooking method allergen implications ("fried in peanut oil")

• Cross-contamination warnings incorrectly classified

• Seasonal ingredient substitutions not captured

Classification failures are harder to catch than extraction failures, because the text can be read correctly and the allergen implication still missed.

3. Translation Stage Failures

Multilingual Processing Failures:

• Allergen terms mistranslated between languages

• Regional terminology not recognised (Dutch "garnalen" = English "shrimp/prawns")

• Idioms implying allergen content translated literally

• Language auto-detection failures on mixed-language menus

Multilingual markets such as Belgium and the Netherlands carry additional translation risk on top of the extraction and classification risk already present in single-language menus.

4. Propagation Stage Failures

Update Lag Failures:

• Ingredient changes by suppliers not reflected in digital menu

• Seasonal ingredient substitutions not captured

• Menu specials with temporary ingredients not updated

• Supplier switches affecting allergen profiles

System Integration Failures:

• Supplier ingredient databases out of sync

• Restaurant manual updates not propagating

• Version control conflicts between menu versions

Propagation failures are a governance problem as much as a technical one: the extraction and classification can be correct on day one and still go stale as ingredients change.

5. Edge Case Failures

Unusual Scenarios:

• Custom dishes with non-standard ingredients

• Sauce and dressing allergen content

• Garnish ingredients often omitted from descriptions

• "Chef's choice" specials without fixed recipes

• Tasting menus with rotating components

Edge cases are where a general-purpose pipeline tends to fall down first, because they are by definition the scenarios a standard workflow wasn't built to handle.

Cumulative Risk Analysis

Each failure mode above operates largely independently of the others, which means the risks compound rather than overlap. A menu that survives extraction cleanly can still fail at classification, translation, or propagation. Treated as a single-stage problem, this compounding effect is easy to underestimate, and it is the reason a single-layer defence (one OCR pass, one classifier, no ongoing review) leaves an unacceptable proportion of menus with incorrect allergen information somewhere in the pipeline.

Regulatory Perspective: Any error rate above zero is technically non-compliant with EU Reg 1169/2011, which sets the bar at full accuracy rather than a tolerance threshold. That is the standard the verification design had to be built against.

Phase 2: Verification Framework Design (Weeks 3-4)

Based on the failure mode analysis, we designed a five-layer verification architecture.

Layer 1: Ensemble OCR with Voting Mechanism

Principle: Independent errors are unlikely to occur simultaneously across multiple OCR models.

Implementation:

• Run three commercial OCR engines in parallel (Google Cloud Vision, AWS Textract, Azure Computer Vision)

• Extract allergen-relevant text sections from each result

• Compare outputs and flag discrepancies

Voting Logic:

• Unanimous agreement: high confidence, proceed to next stage

• Majority agreement (2/3): moderate confidence, proceed with a caution flag

• No agreement: low confidence, route to human review

Cost Impact:

Running three OCR engines instead of one increases per-menu processing cost, but the increase is small in absolute terms because OCR calls are cheap at this volume. Set against the reduction in extraction errors that reach the allergen-classification stage, the additional cost is easily justified once the exposure from even a single mislabelled allergen is taken into account.

Verdict: A cost-effective error reduction given the asymmetry between OCR spend and the cost of an allergen error reaching a customer.

Layer 2: Allergen-Specific Named Entity Recognition

Principle: General-purpose NLP models aren't trained on food allergen terminology across languages.

Implementation:

• Train custom NER model on FSA's 14 major allergens plus regional variations

• Multi-lingual training corpus (English, French, Dutch, German)

• Contextual phrase recognition ("contains nuts" vs "nut-free" vs "may contain nuts")

Training Data Requirements:

• A substantial set of annotated menu samples across the target languages

• FSA allergen lists (English)

• AFSCA allergen lists (French/Dutch - Belgium)

• NVWA allergen lists (Dutch - Netherlands)

• FSAI allergen lists (English - Ireland)

Model Performance: A domain-specific allergen NER model, trained on food-safety-annotated data rather than generic annotation, materially outperforms an off-the-shelf general-purpose NER model on allergen extraction. The gap matters because general models are not trained on the regional and multilingual allergen terminology that this use case depends on.

Development Cost: Annotation by food safety experts rather than generic annotators is the main cost driver, alongside model training and validation. This is a one-off investment with a modest ongoing cost for periodic retraining as new edge cases surface.

Layer 3: Confidence Scoring and Review Routing

Principle: Not all menus require human review - only those where AI confidence is below safety threshold.

Composite Confidence Calculation:

confidence = ( 0.35 × ocr_agreement_score + # Ensemble OCR consensus 0.30 × ner_model_confidence + # Named entity recognition score 0.25 × context_validation_score + # Semantic consistency check 0.10 × image_quality_score # Input quality assessment )

Routing Decision Logic:

High-confidence menus are auto-approved with no human review. Mid-confidence menus get a light review focused on the flagged allergens only. Low-confidence menus get a full manual verification pass. Each tier carries a different processing cost, rising as review depth increases.

Economic Optimization: In practice, the large majority of menus clear the high-confidence threshold and are auto-approved, with only a minority needing light or full review. Because review is targeted rather than universal, the blended cost per menu stays low enough to preserve a healthy margin at a low monthly subscription price, even with human review built in for the cases that need it.

Verdict: Economically viable even with human review, because review effort is concentrated on the minority of cases where the AI itself signals low confidence.

Layer 4: Supplier Database Cross-Reference

Principle: Where available, validate extracted allergen claims against authoritative source ingredient data.

Implementation:

• Integrate with major supplier APIs (Brakes, Bidfood, Sysco)

• Cross-reference menu ingredients with supplier allergen profiles

• Flag discrepancies for restaurant verification

Coverage: Supplier API integration only reaches restaurants using suppliers with accessible data, and ingredient-level matching is imperfect where dishes draw on multiple suppliers. It is a partial safety net rather than a complete one, which is why it sits as a fourth layer rather than a primary control.

Accuracy Impact: Where it applies, cross-referencing catches errors that the OCR and NER layers miss, particularly ingredient substitutions the menu text itself doesn't flag.

Implementation Complexity:

• Supplier API integration: Medium (standardised APIs)

• Ingredient matching logic: High (fuzzy matching required)

• Maintenance overhead: Low (suppliers maintain ingredient databases)

Layer 5: Audit Trail and Version Control

Principle: Complete documentation of every menu change for regulatory compliance and liability protection.

Audit Log Contents:

{ "menu_id": "uuid", "timestamp": "2025-11-03T14:23:17Z", "restaurant_id": "rest_12345", "source": { "type": "photo_upload", "image_quality_score": 0.87, "file_hash": "sha256:abc123..." }, "ocr_results": { "google_vision": {...}, "aws_textract": {...}, "azure_ocr": {...}, "consensus_score": 0.92 }, "allergen_extraction": { "detected_allergens": ["dairy", "gluten", "nuts"], "confidence_scores": [0.96, 0.94, 0.88], "ner_model_version": "v2.3" }, "review_status": "auto_approved", "compliance_check": { "fsa_requirements": "passed", "allergen_labelling": "compliant", "audit_timestamp": "2025-11-03T14:23:45Z" }, "version": 3, "previous_version": "menu_version_2", "change_attribution": "system_ocr" }

Regulatory Value:

• Demonstrates due diligence in allergen incident investigations

• Provides evidence of systematic verification process

• Documents compliance with FSA requirements

Legal Protection:

• May reduce liability in supplier-error incidents

• Shows reasonable care in negligence defence

• Supports insurance claims for covered incidents

Storage Cost:

• 50KB per audit entry

• £0.00003 per GB monthly storage (AWS S3)

• Cost per menu: negligible (£0.0000015/month)

Phase 3: Implementation Guidance and Testing (Weeks 5-6)

Technical Specification Development

We provided EasyMenus with detailed technical specifications for implementation:

API Integration Requirements:

• OCR services: Authentication, rate limiting, error handling

• Supplier APIs: Data schemas, matching algorithms, fallback logic

• Review queue system: Priority scoring, SLA management, reviewer interface

Confidence Calibration Methodology:

• Validation dataset: 500 manually verified menus across market segments

• Threshold optimization: ROC curve analysis for cost-benefit optimisation

• Calibration testing: Reliability diagrams for confidence score accuracy

Testing Protocols:

• Unit testing: Individual component accuracy (OCR, NER, confidence scoring)

• Integration testing: End-to-end pipeline validation

• Edge case testing: Handwritten menus, mixed languages, damaged menus

• Regulatory compliance testing: FSA allergen requirements verification

Deployment Strategy

Phased Rollout:

1. Pilot: a small initial cohort of restaurants, with full human review on every menu

2. Beta: a larger cohort, moving to confidence-based review routing

3. General release: all new signups, with continuous monitoring

Success criteria the design targets:

• Allergen accuracy as close to full accuracy as achievable, measured against human verification

• Fast processing that stays within the product's fast-setup value proposition

• A high auto-approval rate, to keep blended review cost economically viable

• No allergen incidents reaching a customer

Training and Documentation

Restaurant-Facing Documentation:

• "How allergen verification works" explainer (non-technical)

• Legal responsibility reminder (restaurants remain liable for accuracy)

• When to update menus (ingredient changes, supplier switches)

• How to verify AI-extracted allergens (spot-check process)

Internal Team Documentation:

• Review queue protocols (what triggers review, how to verify)

• Edge case handling (when to escalate, specialist consultation)

• Incident response procedures (allergen complaint process)

• Continuous improvement process (error pattern identification)

Results & Metrics

The engagement's aim was a verification framework that could be deployed and then tracked against the metrics that matter for allergen safety and for the business: accuracy, false negative rate, processing cost, and audit readiness.

What the framework is designed to improve

A single-pass OCR system with no verification layer carries a materially higher error rate than a system with ensemble OCR, domain-specific allergen classification, confidence-based review routing, and supplier cross-referencing stacked on top of it. The metric that matters most is the false negative rate: cases where the system reports a dish as allergen-free when an allergen is actually present. That is the error type with the most severe consequence, and it's the one the verification layers are specifically designed to drive down, since a false positive (flagging an allergen that isn't there) costs a restaurant an unnecessary review, while a false negative can cost a life.

Operational design targets

The confidence-routing design is built so that the majority of menus can be auto-approved without human review, with light or full review reserved for the minority of cases the system itself flags as uncertain. That selective approach is what keeps the blended processing cost low enough to preserve margin at a low-cost monthly subscription, rather than requiring full manual review of every menu, which would not be economically viable at that price point.

Regulatory compliance as the design standard

EU Reg 1169/2011 sets the compliance bar at full accuracy on allergen labelling, not a tolerance threshold. The verification architecture, audit trail, and review routing are built to work towards that standard and to produce evidence of due diligence, useful in the event of a regulatory audit or an incident investigation, regardless of the specific error rate achieved at any point in time.

Customer and restaurant trust

Restaurants using the platform remain legally responsible for the accuracy of their own menus. Part of the advisory work was making sure restaurant-facing documentation was clear on that point, so operators understood the verification system as a strong aid to accuracy, not a transfer of liability away from them.

Lessons for AI Safety Practice

The EasyMenus engagement reveals several principles applicable to broader AI safety work:

1. Domain Context Matters More Than Model Sophistication

The hardest problems weren't AI/ML challenges - they were food safety domain challenges:

AI/ML Problems (Relatively Easy):

• OCR ensemble implementation

• NER model training

• Confidence score calibration

• Review queue optimization

Domain Problems (Actually Hard):

• Understanding FSA allergen requirements across jurisdictions

• Recognising regional allergen terminology variations

• Knowing how restaurants actually operate and update menus

• Identifying realistic failure modes based on kitchen workflow

• Designing verification appropriate for restaurant capability and liability

Key Insight: AI safety frameworks designed without domain expertise build the wrong thing. You need food safety professionals in the room when designing allergen verification, not just ML engineers.

Generalizable Principle: Domain experts >>> AI experts when defining safety requirements in regulated industries.

2. "Boring" Applications Aren't Low-Stakes

Menu digitisation sounds trivial compared to autonomous vehicles or medical diagnosis AI. But the consequences of failure are equally severe:

Comparison:

Application Failure Consequence Regulatory Oversight Deployment Scale Autonomous vehicles Death, injury Strict (NHTSA, DVSA) Thousands of vehicles Medical diagnosis AI Misdiagnosis, death Strict (FDA, MHRA) Thousands of patients Menu digitisation AI Anaphylaxis, death Strict (FSA, EU) Millions of diners

The independent restaurant sector serves a very large number of diners across the UK each year. Menu digitisation AI at scale can affect more people than most applications that get labelled "high-stakes" by default.

Generalizable Principle: Deployment scale × consequence severity = true risk. "Boring" applications at massive scale aren't low-stakes.

3. Human-in-Loop at the Right Granularity

"Keep humans in the loop" is common AI safety advice. But pure human review is economically infeasible for SMB applications:

Cost Comparison:

Approach Viable at a low monthly subscription price? Fully manual review of every menu No, cost per menu is far too high to sustain the margin Fully automated with no review No, accuracy is insufficient without a human check Confidence-based hybrid Yes, review effort concentrates on the minority of uncertain cases

The Solution: Selective human review based on AI confidence scores, with the large majority of high-confidence cases automated and only the low-confidence minority routed to a person.

Generalizable Principle: Human-in-loop doesn't mean human reviews everything - it means AI knows when it needs human help.

4. Verification Principles Scale Down

Enterprise AI safety practices - redundancy, audit trails, human review, confidence scoring - are often dismissed as "too expensive for SMB markets."

EasyMenus proves otherwise:

Enterprise Safety Practices at SMB Economics:

• Ensemble methods: a small per-menu cost increase relative to single-pass OCR

• Confidence thresholding: enables selective review rather than blanket manual checking

• Audit trails: storage cost is negligible at this scale

• Human review queue: a minority of menus, by design

Generalizable Principle: AI safety isn't a luxury for enterprise applications. With thoughtful design, safety frameworks work at low-cost subscription price points.

5. Regulatory Alignment Provides Clear Success Criteria

Generic AI safety frameworks struggle with defining "safe" or "aligned." In regulated industries, legislators have already defined success criteria:

Menu Digitisation Success Criteria (Legally Defined):

• Full allergen accuracy (EU Reg 1169/2011)

• Complete audit trail (FSA enforcement requirements)

• Restaurant liability protection (negligence standards)

• Customer right to accurate information (consumer protection law)

This regulatory clarity makes AI safety tractable. We don't debate philosophical questions about what "safe" means - FSA has already specified it in regulation.

Generalizable Principle: In regulated domains, start with legal requirements. Regulations define minimum safety standards, removing philosophical ambiguity.

6. Iterative Refinement Beats Initial Perfection

A verification pipeline like this doesn't reach its best accuracy on initial deployment. It gets there through iterative refinement: launching with the ensemble OCR and NER baseline, then tightening confidence thresholds, retraining the allergen NER model on real errors, adding supplier validation, and refining edge case handling, each step building on production feedback rather than upfront assumption.

The Process:

1. Deploy with best-effort verification

2. Analyse errors in human review queue

3. Identify common failure patterns

4. Retrain models on identified edge cases

5. Adjust confidence thresholds based on real performance

6. Repeat monthly

Generalizable Principle: Perfect AI safety systems don't exist. Deployable AI safety systems improve iteratively based on production feedback.

7. Audit Trails Provide Regulatory Insurance

The audit trail can look like technical overhead at the design stage. In practice it provides critical legal protection. If a regulator or an FSA compliance audit questions a specific menu's allergen labelling, a complete audit trail can show the source image, what each OCR pass extracted, how the allergen classifier scored the result, whether the menu was auto-approved or reviewed, and whether the restaurant confirmed the allergen information. That chain of evidence is what demonstrates due diligence in an investigation.

Without an audit trail, the same situation could trigger:

• Compliance warning

• Follow-up inspections

• Potential enforcement action

Generalizable Principle: In regulated AI, documentation costs are negligible compared to liability exposure. Audit trails are insurance, not overhead.

Technical Appendix: Simplified Implementation

For practitioners implementing similar verification systems, here's a simplified conceptual implementation:

Ensemble OCR with Confidence Scoring

from typing import List, Dict, Tuple import difflib

def ensemble_ocr(image_path: str) -> Tuple[str, float]: """ Run multiple OCR engines and estimate confidence. """ # Run three OCR engines results = { 'google': google_vision_ocr(image_path), 'aws': aws_textract_ocr(image_path), 'azure': azure_computer_vision_ocr(image_path) }

# Extract allergen-relevant sections
allergen_texts = {
    engine: extract_allergen_sections(text)
    for engine, text in results.items()
}

# Calculate inter-model agreement
agreement_score = calculate_agreement(allergen_texts)

# Merge results (use majority vote or best match)
merged_text = merge_ocr_results(allergen_texts)

return merged_text, agreement_score

def calculate_agreement(texts: Dict[str, str]) -> float: """ Calculate agreement score across OCR results. """ # Pairwise similarity comparison similarities = [] engines = list(texts.keys())

for i in range(len(engines)):
    for j in range(i + 1, len(engines)):
        similarity = difflib.SequenceMatcher(
            None, 
            texts[engines[i]], 
            texts[engines[j]]
        ).ratio()
        similarities.append(similarity)

# Average similarity as agreement score
return sum(similarities) / len(similarities)

Allergen-Specific NER with Confidence

def extract_allergens_with_confidence(text: str, language: str = 'en') -> List[Dict]: """ Extract allergen entities with per-entity confidence scores. """ # Load allergen-specific NER model model = load_allergen_ner_model(language)

# Extract entities
entities = model.extract(text)

# Filter for allergen classes
FSA_ALLERGEN_CLASSES = [
    'celery', 'gluten', 'crustaceans', 'eggs', 'fish', 
    'lupin', 'milk', 'molluscs', 'mustard', 'nuts',
    'peanuts', 'sesame', 'soya', 'sulphites'
]

allergens = []
for entity in entities:
    if entity.label in FSA_ALLERGEN_CLASSES:
        # Calculate context-aware confidence
        confidence = calculate_entity_confidence(
            entity=entity,
            text=text,
            model_score=entity.score
        )
        
        allergens.append({
            'allergen': entity.label,
            'text': entity.text,
            'confidence': confidence,
            'position': (entity.start, entity.end)
        })

return allergens

def calculate_entity_confidence(entity, text: str, model_score: float) -> float: """ Combine model score with contextual validation. """ # Context validation: Does the allergen make sense for this dish type? context_score = validate_allergen_context(entity, text)

# Terminology check: Is this a recognised allergen term?
terminology_score = check_allergen_terminology(entity.text)

# Weighted combination
confidence = (
    0.50 * model_score +
    0.30 * context_score +
    0.20 * terminology_score
)

return confidence

Review Queue Routing

def process_menu(menu_image_path: str) -> Dict: """ Full menu processing pipeline with review routing. """ # Stage 1: Ensemble OCR text, ocr_confidence = ensemble_ocr(menu_image_path)

# Stage 2: Allergen extraction
allergens = extract_allergens_with_confidence(text)

# Stage 3: Composite confidence calculation
composite_confidence = calculate_composite_confidence(
    ocr_confidence=ocr_confidence,
    allergens=allergens,
    image_quality=assess_image_quality(menu_image_path)
)

# Stage 4: Routing decision
if composite_confidence >= 0.90:
    # Auto-approve
    result = {
        'status': 'auto_approved',
        'confidence': composite_confidence,
        'allergens': allergens,
        'text': text,
        'review_required': False
    }
elif composite_confidence >= 0.70:
    # Light review (allergens only)
    result = {
        'status': 'light_review',
        'confidence': composite_confidence,
        'allergens': allergens,
        'text': text,
        'review_required': True,
        'review_scope': 'allergens_only'
    }
else:
    # Full review
    result = {
        'status': 'full_review',
        'confidence': composite_confidence,
        'allergens': allergens,
        'text': text,
        'review_required': True,
        'review_scope': 'complete_verification'
    }

# Stage 5: Audit logging
log_audit_entry(result, menu_image_path)

return result

def calculate_composite_confidence( ocr_confidence: float, allergens: List[Dict], image_quality: float ) -> float: """ Combine multiple confidence signals. """ # OCR agreement (35% weight) ocr_score = ocr_confidence

# Allergen NER average confidence (30% weight)
if allergens:
    ner_score = sum(a['confidence'] for a in allergens) / len(allergens)
else:
    ner_score = 1.0  # No allergens detected (high confidence in absence)

# Context validation (25% weight)
context_score = validate_overall_context(allergens)

# Image quality (10% weight)
quality_score = image_quality

# Weighted combination
composite = (
    0.35 * ocr_score +
    0.30 * ner_score +
    0.25 * context_score +
    0.10 * quality_score
)

return composite

Conclusion: Practical AI Safety at Scale

The EasyMenus case study demonstrates that AI safety frameworks designed for frontier AI research can be adapted for everyday SMB applications where AI mistakes have real consequences.

Key Design Outcomes:

• A verification approach built to bring allergen accuracy as close as possible to the full-accuracy standard EU Reg 1169/2011 sets

• A confidence-routing architecture designed to keep human review economically viable at a low-cost subscription price

• An audit trail designed to support regulatory compliance evidence and liability protection

• An architecture designed to scale without a fundamental redesign as menu volume grows

Broader Implications:

Restaurants can get allergen compliance support with digital menus at price points that a fully manual verification process could never sustain. The verification framework is designed to support fast menu digitisation without giving up on regulatory-grade accuracy as the goal.

This shows AI safety isn't exclusively an enterprise or research concern. With thoughtful architecture, ensemble methods, confidence thresholding, selective human review, and audit trails, safety frameworks can be designed to work at SMB economics.

The question facing AI practitioners isn't "can we make AI safe?" It's "can we make AI safety affordable and accessible?" EasyMenus is a working example of an attempt to answer yes.

About the Author

Sotiris Spyrou is CEO of VerityAI, an AI compliance and risk management consultancy, and advisory to EasyMenus.xyz. He specialises in applying AI safety frameworks to regulated industries and has advised companies on verification systems for food safety, healthcare, and financial services applications. Sotiris writes about practical AI safety at verityai.co/blog.

Disclosure: Sotiris is CEO of VerityAI. This case study describes a genuine advisory engagement to design allergen verification frameworks for a menu digitisation system. It sets out the risk analysis and verification architecture as designed; it does not report independently audited production statistics.

Further Reading

Related Articles:

• "When AI Safety Is a Matter of Life or Death" (stakes and failure modes in menu digitisation)

• "From OCR to Accuracy: Verifying AI-Generated Menu Content at Scale" (technical deep dive)

Regulatory Resources:

• EU Food Information Regulation 1169/2011

• FSA Allergen Guidance for Food Businesses

• AFSCA Food Safety Requirements (Belgium)

• FSAI Allergen Labelling (Ireland)

Technical Resources:

• Ensemble OCR implementation patterns

• Confidence calibration for production ML systems

• Human-in-loop review queue design

• Audit trail standards for regulated AI

Published: 03 November 2025 Category: Case Studies Tags: #AIComplaince #FoodSafety #AllergenVerification #MenuDigitisation #PracticalAISafety #CaseStudy #SMBAISafety

More on how we approach it: AI governance.