In partnership with
AI research, explained in one morning email
Most of what shapes AI next year is sitting in a research paper today. Keeping up with them is a full-time job, and the abstracts don't help.
TLDR AI is the newsletter that does the translating. Each morning you get the handful of papers worth knowing about, curated by Anthropic and ex-Google engineers, with a plain-English summary of what they found and why it matters.
You come away smarter about AI research in the time it takes to finish your coffee.
Free, and delivered to 1.1M+ inboxes every morning.
Part two of two.
Last week was part one: a quiet monitor and a dead one look exactly the same. This is what that looked like from my inbox.
Day 1 through Day 18. Zero errors. Every dashboard green.
Day 19, 6:30 a.m. First error. Then another. And another.
About thirty minutes later. Roughly forty-seven of them.
What went off? Nothing. Well. My inbox did.
I found them myself, this August, scrolling my own inbox. On an integration I'd built the monitoring for.
Every tenth issue, an honest one.
Ten issues ago, I told you about the AI chatbot we shipped and then killed in eight weeks. It was the first time this newsletter said, plainly, we got this wrong. Issue #20 is the second. This time it isn't "we." It's me.
And it gets better. Back in July, issue #7 ran under the title "Your AI dashboard is green. Your deployment might be dying." I wrote that. I meant it. A month later, I was looking at a wall of green while an integration I'd built monitoring for was failing underneath it.
Last week was a smoke detector with no battery. This week, I'm the guy who forgot to put the batteries in.
What happened
The setup is ordinary. An integration platform moves records between the ERP and the CRM at a mid-market manufacturer. It has error handling. It has dashboards. For eighteen days it logged zero errors, which is exactly what you want, and exactly why nobody was looking.
At 6:30 one morning, records stopped updating. Statuses that should have changed on the hour didn't. Each miss fed the next one, so every hour the pile got bigger, and every failure sent its own email. One error, one message. The inbox didn't get quieter as the morning went on. It got louder.
At 6:30, the coffee hadn't kicked in. The first few messages didn't register as a problem. They registered as email.
I read my inbox every morning. It's habit: see what came in, sort what's urgent, act on it. That's how I found it. Not because anything alerted me. Not because a dashboard changed color. By chance, during a routine I happen to keep. The errors kept mounting. Hours later, customer service called in with the sentence nobody in IT wants to hear: "We might have a problem."
Here's what "a problem" meant. Orders weren't getting placed. The ones that were didn't make it to the warehouse. Nothing shipped. No invoices went out. Every order was fulfilled by the end of that day. But some customers waited longer than they should have, and the only reason it wasn't worse is that I happened to be doomscrolling. That's luck. Luck isn't a design.
Why every dashboard was green
The dashboards were doing exactly what they were built to do. That's the uncomfortable part.
They tracked the standard numbers: orders placed, orders fulfilled, invoices paid. Those are the right business measures. They're also lagging ones. They tell you how the day went, after the day goes. Not one of them measured whether records were actually moving through the pipe right now, in the moment.
Green meant "nothing in the totals looks wrong yet." Everyone, me included, read it as "working."
What actually broke
The cause was almost boring. Someone who wasn't familiar with the running service turned it off at one end. No malice, no mystery. Just a switch that looked unimportant to the person holding it.
Once that service was off, the integration and the ERP stopped talking. Nothing was watching whether they talked. So we found out hours later, the hard way.
This isn't a story about the person who flipped the switch. Anyone can turn off something they don't understand. The miss was that nothing told us it had been turned off. A heartbeat would have. That's not a guess. It's what happened the next time.
A notification is not an alarm
A notification tells someone. An alarm needs an answer.
Every one of those errors sent an email. The system did its part. It reported every single failure. But an email shows up, sits there, and waits for someone to read it closely. Forty-seven of them look a lot like one of them if you're skimming at 6:30. Last week I called that pile “noise.” This is what noise costs: the real warning arrives, and it looks like everything else.
An alarm is different in three ways:
It knows what normal looks like. Eighteen days at zero is a baseline. Forty-seven in thirty minutes is a break from it. An alert on rate would have fired on the first burst, before the coffee.
It escalates until someone acknowledges it. Not "sent." Answered.
It has an owner by name. Not a distribution list. A person, and a backup.
Zero for three
Last week I gave you three checks: a heartbeat, a fire drill, and a volume floor.
This integration had none of them. No heartbeat. No fire drill. No volume floor. Zero for three, on the exact checks I'm now telling you to run.
How did it feel? In a word: ugh. Every new system has its critics, and I'd just handed them one more reason. That's the cost nobody puts on the incident report. The orders were recovered by the end of the day. The trust lost takes longer.
What I've changed
Afterward, we fixed what we could fix right away:
A heartbeat. A check that confirms the integration and the ERP are actually talking, not just that each one is up.
A volume floor. Volume is watched daily for outliers, and anything unusual goes straight to a chat channel the team actually watches. Not an inbox.
More of the workflow covered. Monitoring now follows the records end to end, not just the totals at the finish line.
The fire drill and the tabletop are still on the list. Not done yet. I'd rather tell you that than pretend.
The battery worked
A couple of weeks ago, it happened again. A small change to how one integration account was named stopped orders cold. Same kind of break. Same quiet start.
This time it was caught and corrected in under thirty minutes.
Not because someone happened to be doomscrolling. Because the heartbeat we'd added caught it. The outliers started piling up, the monitor noticed, and the right people heard about it immediately. August took a lucky inbox review and a call from customer service. This one took a working alarm.
That's the whole issue in one comparison. Same failure. Different instrument.
The smoke detector went off, because this time it had a battery.
Twenty issues in
Twenty Tuesdays. The two issues I'd point a new reader to first are the two where I got it wrong. That's not an accident.
The wins teach you what worked once. Enjoy them. Then be careful, because a win is where complacency starts. Eighteen days of zero errors felt like a win. It was the most dangerous stretch of this whole story, because it gave everyone, me included, permission to stop looking.
The misses are the diamonds in the rough. Nobody goes looking for them, and nobody hangs them on the wall. But a miss is the only thing that shows you exactly what you were taking for granted. August showed me three checks I didn't have. I have two of them now. That's what a miss is worth, if you're willing to pick it up and look at it.
It's the same reason the podcast looks the way it does. The mic's in the shot, and I read off a script. If the vendor pitching you the next platform won't show you their seams, I won't hide mine. This issue is me showing you mine.
Celebrate the wins once in a while. Study the misses every time.
Try this on yours
Pick one integration you'd call stable. Find out three things:
What's its normal error count on a normal day? If you don't know, that's the first finding.
Where do its errors go? If the answer is "an inbox," who reads it, and how closely, at 6:30 in the morning?
If it threw fifty errors in the next ten minutes, what would fire? Not what would be sent. What would fire.
Next week gets harder. Errors at least leave a record. The tougher problem is measuring the thing that never happened: the alert that should have come and didn't, the answer that should have been there and wasn't. That's where AI assistants make the problem worse before they make it better.
Where do your error emails go? Reply, I read every one.
Go hug your ERP. Then check its pulse.
Plant Floor to Cloud: The Newsletter goes out every Tuesday. The podcast runs monthly. Same charter, more room to think out loud. Watch or listen on the website, or wherever you get podcasts: Apple Podcasts, Spotify, YouTube, iHeart, and more.
Views are my own and do not represent any employer. Nothing in this newsletter is attributable to or sourced from any specific company. All case content is generalized to protect employer and client confidentiality.
Plant Floor to Cloud · PO Box 122, Alto, MI 49302