Two possible products sit on an operator’s research list. One has a high apparent margin and little evidence of demand. The other has stronger demand signals but requires more work to describe and fulfill. A model can rank them instantly. The operator still needs to know what the rank means.
A product opportunity score is a decision aid for choosing the next investigation or bounded test. It is not a prediction of profit. A useful score exposes its factors, sources, assumptions, and sensitivity. It keeps eligibility gates separate from attractiveness and allows uncertainty to change the proposed action.
This chapter of The Age of AI Leverage develops a hypothetical scoring method for a small content-led store. The product candidates, weights, and numbers are original teaching assumptions. No Salars inventory, sales history, or tested scoring system is represented.
Start with the decision the score must support
A score can help choose which product to research, which sample to inspect, or which small eligible purchase to consider. Those are different decisions. A candidate may deserve further research without deserving an inventory commitment.
Define the action first. For the illustrative store, suppose the immediate decision is which of two ordinary adult-use product candidates to investigate with a limited amount of staff time. The score should reflect that decision’s costs and possible learning, not assume a large future launch.
A ranking can reduce cognitive burden by placing several considerations in a visible structure. It can also conceal disagreement if the operator treats the final number as authority. The factor-level record is therefore more important than the appearance of mathematical precision.
Write what the score will not do. It will not approve product eligibility, establish demand, verify supplier claims, set a price, or authorize spending. Those decisions require their own evidence and permissions. The permission architecture keeps the recommendation separate from the action.
The score should also allow a result of insufficient evidence. If critical information is missing, a model should not fill the gap with a confident estimate merely because every row needs a number. The next useful action may be to verify one fact rather than rank the whole catalog.
Eligibility is a gate
Some conditions determine whether the proposed activity is acceptable at all. The operator may lack authority to sell an item, rights to its images, adequate safety information, or a credible fulfillment route. A high score on other factors cannot repair that failure.
The CPSC’s resale guidance explains that consumer-product safety responsibilities also apply to resellers and that inventory should be screened for unsafe or recalled products. A model-generated opportunity rank does not substitute for the actual product checks. CPSC resale information.
Use explicit states such as eligible for investigation, requires verification, excluded, and eligible for a defined test. The states should explain what is known and what must happen next. Avoid one approved label that seems to cover every stage from a promising idea to a lawful ready-to-sell unit.
For a hypothetical tool candidate, missing identity can block a relevant safety or condition review. The operator may investigate the identity before deciding whether to buy it. Another candidate may have a clear identity but uncertain delivery cost. That uncertainty can lead to a package measurement rather than a complete rejection.
Record who owns each gate and which evidence establishes it. AI can assemble candidate information and flag missing fields. The responsible operator should not interpret the existence of a completed scorecard as proof that the underlying gate passed.
Choose factors that describe a mechanism
A useful factor explains how a product might create value or impose cost. “Hot opportunity” names a conclusion. “Estimated contribution after defined direct costs” describes one economic input. “Evidence of a fitting reader need” describes another.
For this illustrative research decision, use four factors: economic room, demand evidence, operating fit, and learning value. Economic room asks whether plausible prices and full relevant costs leave space for worthwhile contribution. Demand evidence asks what supports interest from suitable buyers. Operating fit asks whether the store can describe, handle, and fulfill the product. Learning value asks whether a bounded test answers a useful uncertainty.
These factors can overlap. A product with high handling cost may score poorly on both economic room and operating fit. Avoid counting the same burden twice without understanding that choice. Define the factors so their different jobs are visible.
The true profit engine explains the economic inputs. The product-content graph can supply evidence of editorial relevance and product facts. Neither turns the opportunity score into a forecast.
Keep the factor set small enough to maintain. A dozen vague scores can create more subjective labor than a direct comparison. Add a factor only when it changes a decision and has a reasonable way to be evaluated.
Use scales with an interpretable meaning
A zero-to-five scale is familiar, but a number needs a definition. A demand-evidence score of four should not mean the model sounds enthusiastic. It might mean several independent, relevant observations support investigating the candidate, subject to stated limits.
Create anchored descriptions for the factors. A low operating-fit rating could mean major unresolved fulfillment or expertise requirements. A middle rating could mean manageable work with a documented unresolved condition. A high rating could mean the proposed test fits existing verified capacity.
Do not imply that the distance from one to two has a measured economic meaning if it does not. A weighted sum of ordinal ratings is a heuristic. It can organize judgments without becoming a scientifically calibrated probability or an expected dollar return.
Keep raw evidence next to the rating. A record of comparable transactions, a current supplier quote, a measured parcel, or a received reader question allows review. The score alone does not reveal whether the input was reliable or merely plausible.
When reviewers disagree, preserve the reason rather than averaging immediately. One may be evaluating general demand while another considers only the store’s actual audience. Resolving that difference can improve the question before any arithmetic begins.
Work through an explicit weighted example
For a teaching exercise, assign weights of 40% to economic room, 25% to demand evidence, 20% to operating fit, and 15% to learning value. The weights sum to 100%. They are proposed priorities for this imaginary research decision, not validated predictors.
Candidate A receives ratings of four, two, five, and three. Its weighted total is 0.40 × 4 + 0.25 × 2 + 0.20 × 5 + 0.15 × 3 = 3.55. Candidate B receives three, four, three, and four, giving 1.20 + 1.00 + 0.60 + 0.60 = 3.40.
The nominal rank favors A by 0.15. That small difference should not justify a major commitment. A has stronger assumed economics and fit, while B has stronger demand evidence and learning value. The factor view explains the tradeoff more usefully than a declaration that A is the winner.
The rating inputs are invented. A real operator would need current evidence for costs, suitable demand, and capacity. The score does not establish that either product is eligible, that a customer will buy, or that the business can afford the test.
The practical output could be to investigate A’s uncertain demand first, because one inexpensive check might resolve the ranking. It could also be to test B if its experiment is cheaper or more informative. The score helps expose the decision; it does not remove the operator’s responsibility to make it.
Show uncertainty as a range
Suppose A’s demand rating could reasonably fall between one and three, while its other assumed factors remain fixed. Its total then ranges from 3.30 to 3.80. Suppose B’s demand rating lies between three and five. Its total ranges from 3.15 to 3.65.
The ranges overlap. The initial 0.15 advantage does not establish a robust ranking. It may be more useful to resolve the evidence behind demand than to refine the final score’s decimal places.
These ranges are sensitivity exercises, not statistical confidence intervals. They show what changes under selected assumptions. They do not quantify the true probability of sales or account for every uncertainty in the operation.
Test the weights too. An operator short of cash may place more emphasis on affordable learning and recovery time. An operator with expertise and spare capacity may evaluate a more demanding product differently. A score is conditional on the priorities it encodes.
The ecommerce capital-allocation chapter examines those constraints directly. A high opportunity score can coexist with a decision not to buy because cash, space, or responsible attention is already committed.
Distinguish demand signals by what they establish
A search query suggests interest in a topic. A question about a product feature suggests an unresolved need. A completed comparable sale suggests someone accepted a particular offer under particular conditions. None independently proves that the store’s candidate will sell profitably.
Public asking prices are especially easy to misuse. A high listing price does not show a completed transaction. A sold comparable may still differ in condition, included parts, seller trust, delivery, and timing. Record those differences before using it as evidence.
AI can organize the observations and identify possible comparables. It should not count copied listings as independent sources or infer hidden transaction outcomes. The operator needs to inspect the actual supporting material.
A small local sample can be informative without being representative. Several received reader questions may justify a useful explanation or a modest test. They do not establish a market size. The score should label the observation and its scope rather than borrowing confidence from a large number elsewhere.
A negative signal can be valuable too. Repeated questions that reveal incompatibility may reduce the product’s fit while pointing toward a better guide or another candidate. The system should learn from reasons not to offer the item.
Let the next test answer the weakest consequential assumption
Choose a test with a stated hypothesis and a result that could change the action. If parcel cost is uncertain, measure and quote a representative packed example under the actual relevant conditions. If identification is uncertain, resolve it through appropriate evidence. If reader need is unclear, investigate that question without pretending interest equals an order.
Microsoft’s experimentation guidance recommends clear hypotheses, success measures, suitable randomization units, and guardrails. A small store must adapt that discipline to its scale; it may lack enough observations for a reliable conversion estimate. Microsoft guidance.
A test should have a time and resource limit. The operator might authorize a small research effort or one eligible sample. The result can be proceed, revise, pause, or exclude. Define what evidence would support each state before the model generates another enthusiastic recommendation.
Keep factual corrections separate from performance experiments. If a product description is wrong, fix it. Do not preserve an inaccurate version merely to measure whether the corrected one sells better. The test operates within truthful offers.
This article does not report an executed scoring experiment. Its proposed tests show how to convert a heuristic rank into a useful next question. The evidence comes from what a real operator subsequently observes, not from the elegance of the initial score.
Calibrate with retained outcomes
Over time, compare earlier ratings with the results of appropriate completed tests. Did a high economic-room rating survive actual fulfillment costs? Did strong demand evidence lead to suitable retained orders? Did the operating-fit estimate account for support work and returns?
Keep the cohort and definitions stable enough to interpret. A product’s sales can change with season, price, traffic, and availability. If those conditions shift, the score’s apparent success or failure may have several explanations.
Review both selected and rejected candidates where feasible. A system that observes only the products it chose cannot learn much about alternatives it never tested. It may repeatedly favor familiar categories and become overconfident in their superiority.
Do not tune the score to make past winners look inevitable. Preserve some later cases for evaluation and record rule changes. A retrospective fit can improve the appearance of the model without improving its next decision.
The useful outcome is a better local prioritization process under documented conditions. It remains an operating aid. Successful calibration on a limited store history does not establish general validity across products, audiences, or businesses.
Consider the cost of resolving uncertainty as part of prioritization. Two candidates may have similar attractiveness but very different investigation burdens. One needs a single verified measurement. Another requires an expensive sample, specialized inspection, and a new fulfillment arrangement. A ranking of their apparent potential can miss the better immediate use of scarce research time.
Keep test affordability separate from the product score when it clarifies the decision. Record the proposed expense, the question it answers, and the action the result could change. An inexpensive test with a consequential answer can deserve priority over a slightly higher-ranked product whose uncertainty cannot yet be resolved responsibly.
Give the score a responsible owner
The person who uses the score should understand its inputs and limitations. Assign responsibility for source freshness, gate decisions, factor definitions, and approval of spending. A generated ranking without an owner can become an unofficial purchasing policy.
Log why an operator overrode the score. A lower-ranked candidate may fit a known seasonal need or require less scarce capacity. An override is not automatically a defect. Its reason can reveal a missing factor or an inappropriate objective.
Also examine repeated overrides. If operators regularly reject the highest-ranked products, the score may be asking the wrong question. Revise it or stop using it rather than preserving the model because the dashboard looks impressive.
Use permissions to contain the output. A research assistant can suggest the next investigation. It should not receive purchase authority because its score exceeds an arbitrary threshold. Spending needs the relevant eligibility, budget, and approval conditions.
AI Leverage in Practice
What changed? AI can help collect, organize, and compare candidate evidence quickly. That makes broader investigation possible, while increasing the risk of treating weak observations as precise ranks.
What can you do today? Choose one decision and two candidates. Apply eligibility gates, define a small factor set, preserve raw evidence, and calculate a transparent score. Vary the consequential assumptions. Choose the smallest appropriate test that could change the decision.
What becomes possible later? A maintained local record can improve factor definitions and help prioritize more candidates. Calibration requires retained outcomes and protected evaluation cases. Even a useful score should remain subordinate to eligibility, capacity, and verified economics.
A number that opens an investigation
Candidate A’s 3.55 and B’s 3.40 are useful when they reveal the assumptions that deserve attention. They become dangerous when they disguise those assumptions as predicted profit.
The better score ends with a question the operator can investigate, a bounded action, and a record of what would change the decision. That is a modest but valuable form of leverage: less time arranging uncertainty and more time resolving the part that matters.
Return to the AI section, or continue through the series hub.
Sources
- CPSC, Resale/Thrift Stores information center.
- Microsoft ExP, Patterns of Trustworthy Experimentation: Pre-Experiment Stage.
- True ecommerce contribution.
Source guidance checked October 7, 2026. The scale, weights, ratings, and ranges are illustrative heuristics, not a validated statistical model or financial forecast.
Loading comments…