Using Query Rewriting
The recall of product search queries can be drastically improved by rewriting the search queries to generate alternate equivalent forms (called “rewrites”) in some product search applications. Idilia’s technology performs this automatically.
- Idilia’s Product Search Rewrite API improves matching and recall from any product index
- Queries are automatically and instantly rewritten into semantically equivalent queries
- Idilia’s software understands the precise meaning of each keyword in the original query and rewrites the query using synonyms, categories, and sub-categories
- Users don’t need to spend time reformulating queries to find what they’re looking for
Let’s say a shopper is searching an online shopping site for table tennis rackets. So, the shopper searches with the keywords “table”, “tennis”, and “racket”.
But unless the seller used the same keywords to describe their product, the query won’t match the product listing. So, a product listed as “ping pong paddle” won’t match, even though it is what the shopper is looking for.
Idilia’s Product Search Rewrite API determines the exact meaning of the query keywords and rewrites the query into semantically equivalent queries. So, in this case, “table” doesn’t mean “furniture” but is in fact a constituent of a compound, “table_tennis_racket”.
|1.00||table tennis racket|
|0.94||table tennis paddle|
|0.94||Ping Pong racket|
|0.94||Ping Pong racquet|
|0.50||table tennis racquet|
|0.50||Ping Pong paddle|
The resulting rewrites can be expressed as separate queries, each submitted to the search engine, or the output can be combined using Boolean logic into a single query, “(“table tennis” or “ping pong”) + (racket or racquet or paddle or bat)”.
How it Works
- The Product Search Rewrite API routes a query to Idilia’s Sense Analysis software.
- A specially trained recipe determines the precise meaning of each keyword in the query
Then, the sense-annotated query is routed to Idilia’s Paraphrasing software where the query is rewritten into several semantically equivalent queries.
- The number of rewrites is configurable using several parameters including a maximum number, weighting for proximity to the original query, confidence of the sense analysis, and variations on the paraphrasing recipe (selecting whether to rewrite adjectives, verbs, nouns, etc.)
- Finally, the rewritten queries are returned with proximity weighting, and the original query (and all rewrites) are returned with sense-annotation, including confidence scores (read more about the API here).
- Depending on your product search engine, you can process the rewrites individually or combine the unique keyword rewrites into a single query using Boolean logic.
Three Ways to Deploy Query Rewriting
- Cached Product Searches – If you maintain a cache of queries and use auto-complete to suggest queries, then the entire cache can be sense-annotated, each query rewritten, and each set of rewrites turned into a Boolean query, all off-line
- Real-Time – Individual shopper queries can be routed to the Idilia API in real-time, or rules relating to the number of results returned by the original query can determine whether to route the query for semantic processing
- User Control – The interface can allow the user to specify precise senses for keywords, helping improve precision when the original query returns too many incorrect results
Customize Your Rewrites
The Idilia Product Search Rewrite API allows you to control how queries are rewritten by selecting a part of speech (e.g., a verb), and specifying how that part of speech will be managed by Idilia’s paraphrasing software.
Consider the query made up of the keywords “nice”, “walking”, and “shoes”:
Let’s see how this query is rewritten in two different customization scenarios:
Default Scenario – Rewrite all POS
|1.00||nice walking shoes|
|0.79||nice walk shoes|
|0.69||nice walking footwear|
|0.69||nice walking footgears|
|0.67||niceness walking shoes|
|0.64||nicely walking shoes|
|0.60||good walking shoes|
|0.60||well walking shoes|
|0.60||pleasant walking shoes|
|0.56||nice walk footwear|
In the example above, the customized scenario yields a better set of rewrites that can be combined into “(walk or walking) + (shoes or footwear or footgear)” using Boolean logic.