AI Search

Overview

A search application for a telecommunications product, built on a vector database with GenAI-based tokenization so that results are matched semantically rather than by keyword.

The Challenge

Semantic search has no single correct answer to assert against. A keyword search either returns the matching row or it does not; a vector search returns a ranked set that shifts with phrasing, tokenization and the state of the index. Testing it meant moving from pass/fail assertions to judging whether results stayed relevant and stable as inputs varied — and being able to tell a genuine regression apart from acceptable variation.

My Contribution

• Search accuracy validation across query variations
• API testing of the search services
• AI/ML behavior validation
• Edge case and input variation testing
• Regression testing across model and index changes

Interesting Problems

Input variation was where most of the value sat. The same intent expressed with different wording, spelling, casing or partial phrasing should converge on the same set of results, and probing that boundary is what surfaced the behaviour worth reporting.

Testing Areas

• Search Accuracy
• API Testing
• AI/ML Behavior Validation
• Edge Cases
• Input Variations
• Regression Testing

Technical Environment

• Vector Database
• Generative AI
• Tokenization
• Semantic Search