The short answer: a changed webpage is not proof that a price changed. The system filters the work; I still check the meaning before anything becomes a fact or triggers an alert.
After the first complete test run, the monitor returned a list of supposed price changes. I opened each official source and checked it. Not one billing term had changed.
The differences came from layout, scripts and dynamic page content. The monitor had done exactly what I first asked: find changes on the page. I had asked the wrong question.
A noisy alert is not harmless. After enough false alarms, people stop checking the one that matters.
So I stopped trying to make the project fully autonomous. The useful version is smaller: it reduces what I need to inspect, then waits for confirmation.
THE FALSE POSITIVE
I was comparing the wrong thing
My first plan was straightforward: save each public pricing page on a schedule, compare it with the last copy, and alert on any difference.
But a webpage is not a price list. It also contains navigation, notices, scripts, recommendations and layout. Any of those can move while the billing terms stay the same.
A better scraper would still compare the wrong object. Before tuning the code, I needed to define a price record that would stay stable between runs.
DEFINE THE RECORD
A page is evidence, not the thing I compare
I ended up storing one confirmed record for each billing item. It needs enough context to survive the next run:
Webpage changes
- Layout
- Scripts
- Dynamic content
- Billing terms
Stable price record
- OBJECT
- Service and billing item
- VALUE
- Price and unit
- CONDITIONS
- Region, plan, usage
- EVIDENCE
- Source and confirmed time
If a source fails, a unit will not parse, or two records do not line up, the output is needs review. The system is not allowed to upgrade uncertainty into “price changed.”
THE NEAR MISS
One clean record nearly sent the wrong all-clear
The next problem was the notification logic. Each service could report its own result, so one successful check could send “no issues” while another service was unfinished or had already failed.
The message was accurate for one row and wrong for the run. I removed notifications from individual records and added a separate run summary.
Before the system can say everything is fine, the summary checks three things: every required step finished, nothing failed, and no candidate is waiting for review.
- 01Collect and structureRead confirmed sources only
- 02Compare with baselineSummarize after the full run
- 03Submit candidate or failureDo not turn uncertainty into fact
- 04Human reviewConfirm meaning and next action
CONFIRMEDUpdate the baseline and alert when needed
UNRESOLVEDKeep it in review and leave facts unchanged
“No change” is a result only when the full run completed.
PROTECT REVIEW STATE
The refresh also had to leave my decisions alone
The review sheet exposed a quieter problem. A refresh could update extracted fields, but it could not overwrite confirmation, notes, or the decision to accept a new baseline.
Appending a full duplicate on every run was not useful either. The sheet filled with noise, and the latest state became harder to find.
I split the data in two. One record holds the current state for each billing item, with review fields protected. A separate audit log records only changes that someone confirmed.
WHO DECIDES
I split the work by consequence, not capability
The useful question was not “Can the agent do this?” It was “What happens if this judgment is wrong?”
The system handles
- Read confirmed public sources
- Extract the fields I defined
- Compare them with the baseline
- Record incomplete and failed runs
I still review
- What a billing change actually means
- Conflicts in units, plans or conditions
- Whether a candidate is now a fact
- Whether to update the baseline or alert anyone
The system can submit a candidate or a failure. Only a change that someone checked enters the baseline.
WHAT SHIPPED
The version that worked did less
The final workflow reads the sources I approved, extracts a fixed set of billing items, compares them with a confirmed baseline, and sends failures or candidates to a review queue. The baseline and alerts change only after review.
My part was defining what to monitor, rebuilding the data structure, deciding where the system must stop, and changing the rules after the test runs exposed weak spots.
AI helped me write and revise scripts, organize fields and debug the flow. It did not decide what counted as a confirmed change.
I also wrote the operating notes: when to step in, what to check, and where to leave an unresolved case. Otherwise the workflow would work only while I was there to explain it.
FIVE CHECKS
The five questions I use now
- SourceWhere does the evidence come from?
- ObjectWhat stable unit is being compared?
- OutputIs this a fact, candidate or failure?
- OwnerWho has authority to confirm it?
- ProofHow does the system show that the full run happened?
I do not judge this kind of automation by how completely it removes people. I judge it by whether fewer things need manual attention, without making failures or uncertainty disappear.
Official pages will change. New billing models will appear. Parsing rules will go stale. Review is where those exceptions wait instead of being guessed through.
PUBLIC NOTEThis article comes from a real price-monitoring project. Specific details have been anonymized.