Skip to main content
949

October 27th, 2025 ×

Web Dev HORROR Stories + Spooky Trivia! (Spooky Stories Pt. 1)

or
Topic 0 00:00

Transcript

Wes Bos

I think we're live.

Wes Bos

Yep. It's being pulled in. What's up, everybody?

Guest 1

I'll tell I'll tell you what, man. The makeup, it's a little tough. All those all the all the things are a little tough. I gotta use this. I spent all morning doing the makeup. I'm just rubbing it on my head and then trying to blend it, and then I would add too much black. And then it's like what you learn is that it's hard to go back when you add too much black to the mix. So then you're like and then and on top of it, I gotta wear glasses. So I'm doing the whole thing. I look like this, which is arguably a bit better. But when you put the glasses on, it's hilarious.

Wes Bos

Oh my goodness. That is so good. Alright. Just tweeted it out. Let me get it. You your title says at

Guest 1

Scott end tax.

Wes Bos

Oh, I see. I see. That's fine.

Wes Bos

That's fine. That's a spooky story in itself.

Guest 1

That's a spooky story in itself. Speaking of spooky stories, I'm happy that Riverside is even up today, given that Amazon scared the pooch. I I was wanting to test some stuff yesterday

Wes Bos

on Riverside, and I couldn't get it to load. Man, I I think I missed that whole outage. All I saw was everybody talking about it. I think I was just happily doing my local dev and didn't see a thing about it, but, Wes seemed like a major outage.

Guest 1

Didn't affect any of my stuff. It might have affected syntax dot f m. Yeah. Yeah. Definitely did. We're hosted on Vercel, which is hosted on AWS.

Wes Bos

So the whole the whole, the whole Internet went down. And the the funniest parts were people tweeting about their, like, Internet connected beds going down. Oh, yeah. Oh, well the the future of us living in a place where your bed stops working if the Internet goes down, which is is absolutely hilarious.

Wes Bos

But, people people are are piling in. Welcome, everybody. So we do this episode every single year where we do spooky development stories. These are stories that make you want to stick your hammerhead in the ground, and you people submit them every year. If if you have not submitted yours, please go to syntax.fm/spooky, and on probably next year, we'll read it. But we collect them all year round, and and we read them off once every year. So I'm excited to get ESLint it. How are you doing today, Scott?

Guest 1

Oh, I am simply dead. I I could not, go outside in the sun, so I I'm having a great time today, in fact, in my nice little little cave here.

Guest 1

But, folks, before we do that, let's talk about something that doesn't suck. Let's talk about Century, century.io, the perfect place to fix all of your, blood curdling, bone chilling problems on your applications.

Guest 1

Yes. We all know that these things pop up all the time, and it can simply make us scream.

Guest 1

So what you wanna do is head on over to century.i0/syntax.

Guest 1

Sign up and get two months for free. You can fix all of your bugs in production, all of your bugs in development, and you can do so with, like, the best tools in the entire world. And I say that as somebody who uses these tools every single day. So shout out to Century, for for presenting this podcast.

Wes Bos

You know that probably about four or five of these spooky stories GitHub our prompting have had this probably wouldn't have been an issue if we had Century enabled.

Wes Bos

Yeah. Right? Yeah. Those will probably write themselves, but you wanna grab the first Node? Let's get on into it.

Guest 1

Let's grab the first Node. PHP my headache.

Guest 1

On my very first job, we had a state of the art tech stack, PHP back end, MySQL database, and brace yourself, Pnpm MyAdmin in production.

Guest 1

We used it for everything, browsing tables, running queries, breaking things accidentally, you name it. Oh, Wes of the days of Pnpm my myadmin. PHP myadmin is like the OG Sass. You know? OG Sass. If or not the OG Sass. The OG, like, CMS. CMS back end. Yeah. Yeah. Yeah.

Guest 1

One fine day, I decided to refresh my local database with a copy from production.

Guest 1

Yeah. I know. Solid DevOps.

Guest 1

I opened phpMyAdmin, and I clicked the big red drop that database button and got the classic warning, are you sure you want to do this? Hell, yeah. I'm sure, I said confidently.

Guest 1

Click. Two seconds later, and our Slack and Sentry exploded like a fireworks show of despair.

Guest 1

Table does not exist. Query fail. Everything is on fire.

Guest 1

My soul left my body. I had been logged into production, not local. I had just deleted the entire production database.

Guest 1

Hey. The site's down. Yeah, I said with a forced smile. Working on it.

Guest 1

Meanwhile, one of my vacationing coworkers messaged me probably from a beach somewhere. Hey. What the hell is happening? My phone is blowing up. Nothing to worry about. I lied through my teeth.

Guest 1

Miraculously, I had a five year old MySQL dump on my Five minute not yet. Five year old. Yeah. Major difference.

Guest 1

Major difference there. A five minute old MySQL dump on my laptop.

Guest 1

I imported it back into production like my life depended on it, which pnpm sense it did.

Guest 1

Within minutes, the site was back online.

Guest 1

I leaned back, took a deep breath, and told my coworkers everything's fine now. Nobody ever knew how close we came to digital extinction that day. Just imagine, you do not have a five minute old backup here.

Wes Bos

It's wild that so many of these stories are people that don't have automated backups in place. They don't have any sort of guardrails where you can just YOLO Pnpm my admin in production. At the very least, change the background of that production. Pnpm might end in to be, like, red or something. If you're gonna have scary places Yeah. No kidding. A few guardrails in place.

Guest 1

Okay. How many how many, shaking bones do you give this one? Oh, man. I'm gonna give that a a, six out of seven for sure. Six out of seven. I'll give it a Six. I'll give it a five out of seven. If this didn't have a backup, you could go higher on this one. But definitely one of those stomach in your, pity your stomach moments for sure.

Wes Bos

Next one's called a browser hack. I worked for a small literal mom and pop shop. Wife was the programmer. Husband was the designer and head of company as my first job out of college, and it's had its own proprietary CMS ecommerce platform, the class of LAMP stack. I was a jack of all trades doing all the technical stuff from programming the front end design work to database and server administration. We hosted hundreds of our clients' websites on a server we rented. This was before the cloud was big, and it was obvious from my comment about the LAMP stack. Yes. So many of these these horror stories come from just shoddy Pnpm LAMP stack. Yeah. Scott that it's a bad programming language, but just, like, the the, like, hygiene around being safe with production Wes nothing. Everyone's just FTP ing and working in production. You're just YOLO winning all.

Wes Bos

One day, our phones started ringing off the hook with a bunch of customers saying their websites weren't coming us. For us in the office, though, the sites were coming up fine. There was an utter dread for maybe half an hour, which felt like an eternity as we were struggling to figure out what was going on. Eventually, we determined that it was affecting our users below a certain version of Internet Explorer.

Wes Bos

Remember those days? After investigating further, I eventually realized I had accidentally saved a semicolon at the very configuration shared by every site on the server.

Wes Bos

JS a result, a semicolon at the very beginning of the HTML for every website before the HTML tag, most browsers completely ignored it. Yeah. Browsers are super forgiving about mail form markup. But, of course, IE threw a fit about it. Fortunately, it ended up being an easy fix. We figured out what was going on. I am forever grateful. I don't get nightmares from having to support Internet empire.

Wes Bos

Literally, a semicolon took down their entire empire.

Guest 1

Oh.

Wes Bos

Five trumps out of five.

Guest 1

Yeah. Oh, five trumps. Yeah. I I'm gonna give this, pretty scary stuff here. This will also be five from me.

Guest 1

Who's here? I hit I think I just heard a doorbell.

Guest 1

Wes, did you hear something? Yes. Who's here? Maybe you should check the, the waiting room.

Wes Bos

Oh, Yarn you kidding? I didn't even know about this. No chance.

Wes Bos

Please let him be dressed up. Yay, Erg, ladies. Woah.

Wes Bos

Woah. No chance. No chance.

Guest 1

CJ,

Scott Tolinski

my computer was frozen for a second. Hello. Hello. How are you how are you doing?

Guest 1

Do you just casually have a pirate costume? I hear something else.

Guest 1

What's going on here? Let's wait a second.

Guest 1

What is that? What?

Scott Tolinski

I was working in my code late last night when my eyes beheld a creepy sight, for my code was working with no reason why, and then I sought to my surprise. It was a hack.

Guest 3

It was a browser hack. A browser hack.

Guest 3

With macromedia flash. Oh my gosh. It really saved my ass. A browser hack.

Guest 3

It was a browser hack.

Guest 1

Gentlemen, boys and ghouls, it's that spooky time of year.

Guest 1

And in the spooky time of year, anything can happen, so I've invited our friend CJ over for a bit of trick or treat.

Guest 1

And, see, we all know that a well timed browser hack can really, really save our ass.

Guest 1

So I prepared for you a soul sucking game where we have eight browser hacks.

Guest 1

You all have to tell me, is this a real browser hack, or did I make it up? The first browser hack, the backslash nine trick.

Guest 1

This targets IE 5.5 and IE six specifically by doing a backslash nine at the end of a CSS property.

Wes Bos

Do we both get to guess? You both have to guess.

Wes Bos

That I'm gonna say that's a legit one. There was backslash. I don't know if nine was the one. For whatever reason. Riverside is messed up. But can you hear me, Scott?

Guest 1

I can. CJ cannot. Yeah. Oh, it's alright. I if you need to be CJ? Yeah. I can hear you, Scott. I will tell you when you should talk. Wes says this is a real hack. CJ, what do you say?

Scott Tolinski

I don't think it's a real hack. I say no. CJ says fake. It is fake. Yes.

Wes Bos

Oh, but there was one, wasn't there? There was, like, a weird IE five and six one. Maybe.

Guest 1

Maybe.

Guest 1

JS and CSS, You can write JavaScript inside of CSS to extend CSS's capabilities using the expression function.

Wes Bos

That's fake.

Scott Tolinski

Wes says fake. See, CJ, what do you what say you? I mean, this seems really useful. Like, maybe it existed somewhere and then didn't become a standard. I'm just I'm I'm gonna say real. Real.

Guest 1

This is real. No way. This is a in all versions of Internet Explorer 5.54.

Guest 1

Apparently, you can this blog post is really interesting. Apparently, it worked really well. It was just slow as hell. So, people didn't use it.

Guest 1

Real.

Scott Tolinski

Fascinating.

Wes Bos

Alright. Number three.

Guest 1

CJ, two for two. Wes, o for two. Conditional comments used to deliver HTML or CSS to different versions of IE if l t I e seven.

Wes Bos

This is real.

Wes Bos

This was the move in, HTML five boilerplate. You had it and it added those conditional classes to the HTML element.

Guest 1

Yes. CJ, what say you?

Scott Tolinski

I believe it's real. I saw Wes saying a whole bunch of stuff, but, yeah, I've I've used this in the past. It's real.

Guest 1

It is, of course, real. For those of you who never worked in IE, this magic actually worked, and we used it all the time for various stuff. Even, like, banners that said, get the hell off this browser.

Guest 1

Alright. Number four, the ping fix for IE six. A shim, a PNG alpha in IE six by replacing images using Microsoft's filter ProGaDx image transform Microsoft alpha image loader.

Guest 1

Wes.

Wes Bos

This is very much real.

Wes Bos

I actually went down a rabbit hole several years ago to figure out, like, how did that actually work? To me, it was magic at the time that it figured out how to just punch out all the alpha pixels of a PNG, or all the, I guess, white Vercel, but it's now it's a now I know how it works. It's very real.

Scott Tolinski

CJ, what about you? What say you? Okay. I didn't hear what Wes said. He said a lot. I'm gonna say fake just because the colon in the syntax

Guest 1

seems like that'd be part of the process. It's real. Woah. Woah. Okay. Yep. Here's the CSS tricks article for it. Pnpm hacks from I six. I think this article is from, like, 2009 or so, and this is a blast from the past for sure. Alright. The comment float fix. This stops IE five Mac from double float margins because it forced the parser into a phantom quirks neutral mode. Wes.

Wes Bos

That you you tried to make it sound like that's actually something, but, IE five was not on the Mac. There was an IE version on the Mac at one point, and this is no. I've never seen this slash thing ever. This is fake, fake, fake all the way down. CJ.

Scott Tolinski

Real.

Guest 1

CJ says real. It is fake.

Guest 1

The margin co I made that up. Margin collapse fix. This prevents IE six from collapsing margins, which collapsing margins Wes you have two margins that butt up against each other. Wes, real or fake?

Wes Bos

Margin collapse fix. Margin collapse is a thing, but the the code you have is that's that's not real. This is that's fake.

Wes Bos

Wes says fake. CJ. I'm gonna say fake too. Oh, events collapsing. Yeah. Okay. Good.

Guest 1

The star and the underscore hacks. You can use the star or the underscore to target specific browsers because, I I I suppose it would, be whatever, the progressive fallback there.

Wes Bos

Okay. My this is real all the time. My I old IE knowledge is immaculate, and these are these are very much real. I've very much written many of those in my career.

Scott Tolinski

CJ, Wes says real. What say you? Oh, don't tell myself. I I now believe the syntax can be anything, so I'm gonna say real too.

Guest 1

It is, of course, real.

Scott Tolinski

West is mad that I'm telling you what his answer was. I just if I could hear his mic, I would know what he said.

Wes Bos

Well, you can.

Guest 1

Okay.

Guest 1

IE float double margin fix. When you float an element in IE, it doubles the margin.

Guest 1

So, this is that that's not the the that's not the real or fake part. IE actually does that. So if you do float left and then margin left, it will do 40 pixels.

Guest 1

So the fix would be throwing display in line to fix it.

Guest 1

Real or fake, Wes?

Wes Bos

Wow. That sounds like something that would be be real. And and putting something in line also sounds like the move because probably one of those margins is from it being Bos, and and the extra one is because of it being floated. So I'm gonna say real.

Guest 1

Wes says real. CJ, what do you say? You're not telling what the answer is.

Guest 1

But CJ says fake.

Guest 1

It is real.

Guest 1

Yep. You gotta do that. And thank you for joining us for,

Wes Bos

spooky stories, real or fake. You guys like Halloween way too much. Like, not not too much, but, like, such such impressive costumes.

Wes Bos

And mine is just, like I just grabbed one from the kids' bin, like, ten minutes before this started.

Guest 1

CJ, thank you for joining us. You're more than welcome to stick around or go off and do whatever you have to do for the rest of the day. I got

Scott Tolinski

some booty to be plundering, I guess. I don't know. Yes.

Wes Bos

Oh, CJ. This is a family show.

Guest 1

Oh, I believe I also have some booty to be plundering.

Guest 1

Okay. So let's get into the next one, ghost password.

Guest 1

So I'm a maintainer of a fairly popular npm package. Nothing huge, but it recently crossed the 2,000,000 monthly downloads mark. Nothing huge. 2,000,000 That's pretty big. Monthly. That's huge.

Guest 1

Give yourself a round of applause, of which I was super proud of. For a long time, I was the only dev behind it until a few months ago. I even wrote in with a potluck question about how to start sharing ownership. I do remember that.

Guest 1

The vampires never forget.

Guest 1

Then about a month ago, the spooky part started.

Guest 1

I'm on my phone, and 1Password suddenly stopped authenticating me. You know, face ID fails, and it asked for the master password.

Guest 1

I type in my master password.

Guest 1

Denied.

Guest 1

Oh, man.

Guest 1

That's the worst.

Guest 1

Okay. Weird typo. Try it again. Denied. Little Scott forms in my stomach. The next day, I try it on my Mac. Same thing. Now I'm officially locked out. I'm thinking, okay. Don't panic. I'll just contact support, spend a few hours going back and forth with one password support team, but they're firm. There is nothing they can do on their end to help me recover the account. Is that serious, one Password? Oh, man.

Guest 1

And here's the kicker.

Guest 1

The part that felt like ESLint from a horror movie, they told me, sir, we can confirm from our logs that your master password has not been changed in five years.

Guest 1

It was like being locked out of your own house with a butler telling me the key that I've used every day for five years is the wrong one. I eventually had to give up. I started the soul crushing process of resetting every single password for every account I own. Oh my god. In the end, I managed to reset every single account except for NPM. Due to their security policy, an account cannot be recovered without the two FA codes or the recovery codes. They tell you to save those recovery codes for a reason. Wes. Yeah. Do you save your recovery Node? Yes. Like, there's man,

Wes Bos

you should especially your one Password recovery codes or, like, your your password manager recovery codes, you need to store those for in the first place, and you need to store them somewhere safe because you

Guest 1

are screwed if you lose those. So with extreme regret, I had to say goodbye to my Npm account and package I built from the ground up. No.

Guest 1

No. Source project for billions of downloads Node orphaned because of a password I thought I knew.

Guest 1

I feel like I feel like you're just typing in that password a little bit wrong, don't you? Did that, like try see if caps lock was on maybe? Yeah.

Guest 1

Or did you try from another phone or computer or something? Because, like, I Node if I'm on, let's say, a new version of macOS, something might now always work the right way. I was on, like, Helium browser, and my Reddit login didn't work for two days. But I didn't change the password. I just tried it again in two days, and it worked. Sanity

Wes Bos

makes me wonder if this guy's if something else spooky is going on. You Node? Like, the guy's computer has been compromised because, like, everybody right now wants to get ahold of npm packages. And one with 2,000,000 if you got a npm package with 2,000,000 monthly installs, you are certainly being targeted.

Wes Bos

So I would probably wipe your computer because something funky is is being put in between you and the sign in there. I think think that's what's happening. Yes. Yikes. This was very specific. Us the name of it because yeah. That's that's odd.

Wes Bos

Yeah. But, also, like, props to 1Password for not giving anybody random people access to your account. I would not ever want them to be like, oh, sorry. I forgot it. That's the whole point of of backup codes and and all these awful there's there's many layers of of Node password if you if you get locked out. And if you can't get them all, you're probably a hacker. And, unfortunately, that means some good people get caught in the toying of it because they had bad password hygiene, but it's worth

Guest 1

it. Word.

Guest 1

How many spooky,

Wes Bos

bats do you give this one? That one's more of, like, a a sad one. I'm gonna get that four out of seven spooky bats.

Guest 1

This one's eight spooky bets for me because it's very spooky, sad or not. Like, sometime That's true. Sometimes, Wes, things that are are are sad are spooky too.

Wes Bos

That's true. Very yeah. That's that's deep.

Wes Bos

Next one we have here, just a patch. When I started web development, I did not know everything about how SQL patches were made. There were a few requests that I worked on. Bugs and new features worked fine for me, and then I was asked to patch it up to prod. I had new tables and some extra columns on existing tables, so I thought it would patch it all up at once. So I generated a script for every object in the database Scott to miss anything. Yes. Drop create scripts. The boss ran it on the prod DB, and then he asked me why every table was empty.

Wes Bos

That is when I realized what I did. Fortunately, there was a fresh DB backup, and it was fixed immediately. Man, awful.

Wes Bos

There's so many of these stories where people just YOLO some script. Like, at least run it, pull your day if it's if your database database is not so big, pull it locally and just run it just just to check. Or or Yeah. Or a lot of, like, ORMs will have, like, a, like, a test mode where it will tell you how many things it will update. Yeah. What are the dry run or something like that? Yeah.

Guest 1

Very, very common. And and I are you one of those people? Sometimes I see the dry run option, and I'm like, it's all good. I got this.

Wes Bos

Yeah. Well, I I've been there as well. Even just, my home assistant the other day, I needed to, like, update it.

Wes Bos

And I was like, should I back up my config? I was like, that'd be fine. It wasn't fine. It I I had everything go missing. Luckily, I had a backup from, I don't know, a couple months ago, but

Guest 1

always back up before you do anything where you say, it'll be fine. Yeah. Always do that. Hey, Wes. This next one is a little bit long.

Guest 1

I don't know if you've scrolled, but it it goes on and on. So, you think we should just read this in its entirety?

Wes Bos

Is it did you read this one already? Yes. It JS. It's spooky For the next show, I read them all to make sure they were all good. Is it worth it? Is it worth it? Sure. Let's do it. Yep. Alright. Well, this is a true story. If it's too long, we'll cut it out.

Guest 1

Folks, you're getting the live feed here. Yeah. The live Vercel of this.

Guest 1

Spooky stories.

Guest 1

Wes, I didn't hit you in any spooky voice today.

Wes Bos

Oh, I Does that make you happy? Yeah. Oh, I can tell you I'm getting really hot in this hammerhead shark costume. My wife's always talking about natural fibers.

Wes Bos

They're like,

Guest 1

don't wear plastic, folks. This is so hot. Oh, yeah. For JS Nation, we wore space costumes, and they were like it was like literally wearing plastic bags. I was just absolutely dying in it the whole time. Okay.

Guest 1

This is a true story, and it happened in 02/2019, about five months before the pandemic hit. It is a catastrophic production failure that I'm 100% guilty. Keep in mind, I was still under six months probation inflicted upon new hires. So this Wes a new hire, been there six months, and they have committed a crime.

Guest 1

The company I was working for at the time was releasing to a new market. With a new market comes new payment options. Of course, we wanted to allow every single payment method in every single county. Right? So we had multiple conditions to build our final list of available payment methods for each particular customer. That check was done in FE in in the FE. What is the FE? What why am I blanking?

Wes Bos

Finance? Front end? No. Front end.

Wes Bos

No. No. Yeah. Of course.

Guest 1

That check was done in the front end, my domain, while the implementation of the new payment method was done by a colleague in the payments team. I'll call him Steve.

Guest 1

He was a newer hire than I was. During the whole thing, no one else could really help us since every single senior there was too busy with some other part of the new market release. It was a whole company effort of the miracle releasing to a new market in a couple of weeks instead of months.

Guest 1

It was me and Steve figuring out our way through microservices.

Guest 1

We only had each other to rely on, so we pair coded most of it. Just to add some spice to my life, the requirements kept changing.

Guest 1

We were having issues with PayPal and Visa. So for around a little over a month, I'd get different requirements and conditions for which payment method should be available

Wes Bos

every other day. Awesome. So I'm there. It's a night. I hate working with this stuff. Like, the people always complain about Stripe and PayPal, and it's it's the reason people hate them is because they grow so large that it is impossible to figure all of this stuff out and keep it all straight as to what's available in which country.

Guest 1

The conditions were also fairly complicated depending not only on the customer country, but also how many purchases the customer had done in the past, whether it was a high ranking customer, whether or not they had submitted documents before, whether or not the customer was considered validated.

Guest 1

Since every time a new payment method was added, the norm had been touch as little as possible.

Guest 1

The conditional tree was an absolute mess with many functions scattered through the utils file, some of which add one line in them to call yet another function. Bro, we have all been there.

Guest 1

At some point, me and my partner in crime, Steve, couldn't tell if the new payment method was showing or not under the correct conditions because it was held just to figure out what the conditions for the requirement sheets were already added and correct or not. Steve and I decided to refactor the whole thing.

Guest 1

Beautiful.

Wes Bos

We look at it. Can we can we solve that for a second? The, like, the legacy code that have been tweaked year over and over and over to Vercel little edge case possible, it is almost never a good idea to just scrap it and start from scratch unless you have an extremely, well thought out test suite covering all of these, which it doesn't sound like they did.

Guest 1

Even for, like, markup. I I had a back in back in the day, I had an old HTML site that had, like, you know, 25 files in each directory that was, like, index forward slash hyphen hyphen new 20 you know, whatever. And and you'd update something. You'd be like, why JS this wrong? I was Scott using the right one or whatever.

Guest 1

And I decided to refactor that, and that was a nightmare too. And that was just straight up HTML and CSS.

Guest 1

Okay.

Guest 1

We looked at each other at this and say at the same time, screw it. We yanked out the whole component, gutted the so called conditional tree. We burned the utils file to the ground, and we added tests when there were none. Days of pair programming, blood, sweat, and tears, and all that's left is to confirm in the staging server.

Guest 1

I have cobwebs on my mouse pad, and it keeps messing up my scroll. I'm, like, going like this every single time that that I lose my spot because this one's so long. Okay. We hit a couple of simple problems. First, we couldn't properly validate test customers, so we set it to manual. We use a forced response that had to be configured in certain test API.

Guest 1

Everyone else was also testing stuff in that same staging server, and forced validation was getting in the way of the customer validation API test. Again, Steve looks at me like he's saying, we are nice to everyone who's going to be nice to us. Again, we go Scott it. We hard Node the customer is validated in the front end.

Guest 1

Hard coded that the customer is validated. Fine. Sure. They're all validated.

Guest 1

This is the first fork in the validation tree, and if the customer was not validated, no payment method JS, available.

Guest 1

We check, double check, triple check every situation, every country, every possibility in that FSM is accounted for, and we see and document everything is working and we test as far as we can test. See, there was another little thing. We couldn't complete payments with this new method because there was a test API for that method, and it never made it into the documentation.

Guest 1

Steve and I couldn't find a single soul who knew about it.

Guest 1

Maybe they were another they were a spirit.

Guest 1

Time was running out. This was Wednesday morning. We had to push to code before 3PM. There was one Wes left to run. The case where the customer is not validated.

Guest 1

The customer validation guys are still testing their code, so we can't use the forced response API again. We go, screw it, and we hard Node the whole thing. Fifteen minutes to deploy the code again to the staging server. We look at it. We see it's working. We harass both of our teams to get a PR containing all of the back end and front end modifications approved. Another fifteen minutes merge to master, another fifteen minutes to merge to production, and we are done. The release party is about to start. Crack open some brewskies. Right? Boss drags us all into the party lounge. Get in here, boys.

Guest 1

I bring the laptop to the party because my name was on the production of deployment, so I was ready to be on call, monitoring certain channels, error logs for at least two hours. Steve brings his laptop. We grab a couple of beers and split the task of monitoring. We sip our beers. We laugh. We talk about how crazy Owl JS, and we bask in the light of how much we managed to simplify that code. Then, oh, we see the curve showing payments transaction activity.

Guest 3

Flatline.

Guest 1

Oh, I that is like the people who don't know, if you work in payment processing anything, the moment that you make code changes and then all of a sudden in the actual payments window, you see a flat line, you you are going to panic mode immediately because you goofed something up. Right? I look at Steve. He's as pale as a ghost. I'm as pale as a ghost. Hand shaking. We go to the logs and try to find an inkling of a clue what the crap is going on. There are no errors. Nothing crashing. We dig deeper. We enable debug in production knowing full well we might just crash a couple of services. We see a bunch of logs. Customer not validated. Exiting out.

Guest 1

I banged my head on the table a couple of times. I know, I say carefully.

Guest 1

Steve, confused, just staring at me. I open the front end Node, and it dawns onto him. Crap.

Guest 1

We forgot to unhard Node the thing, didn't we?

Scott Tolinski

Classic. No. Oh, no.

Guest 1

He bangs his head on the table a couple of times. We look around to see if anyone had noticed everyone was partying.

Guest 1

I create the PR. We change one Boolean from false to true. Steve quietly approves.

Guest 1

I try to breathe deeply while looking at Jenkins, feeling those fifteen minutes stretch out over an eternity. I reach out Oh my gosh. To another beer, and split second, I crack the beer open.

Guest 1

It goes pnpm. I feel a hand on my shoulder. I look up, and I nearly gagged. It was our CTO.

Guest 1

Oh, CTO means business.

Guest 1

I fill out my soul leaving my body as he says firm calmly, please don't drink and deploy.

Guest 1

My brain was still trying to process it when the CTO Bos out laughing and leaves. Steve looks at me. We are so fired. We cheer with our beers as the deployment finishes. We drink and celebrate our own little party as the payment transaction curves go up.

Guest 1

Oh, man. We both had to sit there a postmortem of shame later that month to get scolded on how many millions exactly the company lost in this hiccup.

Guest 1

Bro, if that if that loss is a a milli or higher, that's a that's a scary situation.

Guest 1

That's scary, but also, like,

Wes Bos

if you have that much traffic and that much payment, unbelievable that that gets through.

Guest 1

But I think it's hard to come in these stories. Not just unbelievable that gets through, but unbelievable that that gets through and and gets put on the shoulders of a six month old higher. That's a a process failure right there.

Guest 1

Six month and a a less than six month old higher.

Guest 1

Oh, man. No. We didn't get fired because of that incident. The shame of sitting in that postmortem was probably more traumatizing than being fired.

Guest 1

The incident caused a solid bond between us. He was my favorite colleague until he left.

Guest 1

Wes, I give this one a good seven, a good seven creepy crawlies because Node if it starts with a million, that bumps it up in that,

Wes Bos

you know No kidding. Than seven territory. When the stakes are this high and the stories are so funny, it's a 10 out of 10 for me. It's a 10 out of 10. And, like, if there's anything that helps you get your deploys down to be as fast as possible, it's stuff like this Wes you know the fix is in or you hope the fix is in, but you gotta sit there for half an hour while you wait for the thing to to deploy and and be pushed through. You know? Because, like, some of these sites are down, and, hopefully, you can cut back.

Wes Bos

But it sounds like this was such a big one that there was no rolling back, and you could only only fix forward.

Guest 1

Yes.

Guest 1

By the way, these fangs are digging into my lip, and I'm gonna have, like Awful. Actual poke marks in my lip when I'm not I gotta take this off. Folks. That's not makeup.

Guest 1

It's about to be. I'll tell you that. That's about to be.

Wes Bos

Alright. Last quest or last story we have for this Node, and then, coming up on Wednesday, we have a full hour of them as well, plenty of them. It's called discount. You get a discount, and you get a discount, and everybody gets a discount. I work for a large company that offered multiple subscription based products, man. These these ones I like the ones where the database goes away, but I really like the ones where real money is involved. And I really, really like the ones where products go out before they realize it's that's okay.

Wes Bos

My task was to implement a discount feature in the CMS so that a content editor could add discounts to a product. Everything was tested and deployed to production without a hitch. Until twelve hours later, someone discovered that each and every product now had a discount. Products that had no discount set up, used the first available discount in the list, and it happened to be conveniently sorted from high to low. This was a cause of an API in our CMS that returned a field such as object dot discount dot value, but my code only checked if object dot discount exists, which always was a it's true. An empty object is truthy, folks.

Wes Bos

Although the financial loss was very extensive, my boss was relaxed about it and understood that mistakes happen. When I eventually left the company Yarn later, I got a postcard with the actual Oprah, you get a car meme as a goodbye gift.

Wes Bos

It's sometimes you see these, like like, pricing errors, and people, like, freak out.

Wes Bos

Like, Abercrombie had a had a major pricing error maybe about six months ago. And I'm still not sure if it was a pricing error or if it was just like a intentional, like, marketing campaign.

Wes Bos

But when when those pricing errors go out, that's nuts. Like, once Wes I when I was in university, we found a coupon. In Canada, we had this website called red flag deals. And anytime there was a pricing error or a coupon or a good deal, it would go up on that website, and people would just destroy it. So we had this one that was $5 off $5.

Wes Bos

So we went around to every it was called pharma plus, like a like a like a CVS in Canada. Yeah. We went around to every pharma plus because you're we're a college student, and we have more money than brains or sorry, more time than money in in brains. Yeah. Right. Yeah. And we just we went to every single one and bought $5 worth of stuff and then went back in and bought $5 worth of stuff and just kept doing it until they stopped us, and it just went to the next one. And I got, like, maybe, like, a $102,100 dollars worth of, like, deodorant and chips and all this hilarious stuff. And I just I feel bad now looking back. Like, some poor guy pushed that coupon out on a Friday afternoon. It was a Friday and just went home for the day, and then they by, like like, Sunday morning, they had stopped it. But, like, it was it was running rampant.

Guest 1

Yeah. One thing I always laugh about is when people are, like, talking about software bugs in, like, sports apps or something, and they're like, ESPN thinks this player plays for this team. So I was like, no. ESPN doesn't think that. It's like some poor intern who pushed a wrong value in the app and all of a sudden you know, it's like, some some some poor person is, like, now, like, getting the heat of the whole company because of that, even though it's like a Yeah. Like, you know, insignificant little thing. Right? Well, at least we can you give us a laugh. Many of these stories people send us years years later after they've the PTSD of the event

Wes Bos

has Deno. Man, I am hot in this shark costume.

Wes Bos

Mhmm. But, so that's the end of the first one. Should we just keep rolling? Because I don't wanna end the livestream.

Guest 1

No. We'll we'll keep rolling, but let's at least wrap the episode. Wes, thank you folks for joining us on this first spooky tales of spooky web developer stories where we found real scary situations that the real developers found themselves into.

Guest 1

We will catch you on Wednesday with a full hour for this Wes just a first taste, the first bite of these spooky stories.

Guest 1

Out.

Wes Bos

Peace. Just kidding. It's just a costume. I didn't get eaten. Oh, yeah. I was gonna ask that. You actually recorded that song? Like, that didn't sound like AI. That was really good. Would you like to hear it again? It folks, this song Wes not AI. I recorded this myself. You play it. I'm gonna grab a drink. One sec. Okay.

Guest 1

This was

Wes Bos

not

Scott Tolinski

I was working in my code late last night when my eyes beheld a creepy sight for my code was working with no reason why, and then I sought to my surprise. It was a hack.

Guest 3

It was a browser hack. A browser hack.

Guest 3

With macromedia flash. It was a hack.

Guest 3

It really saved my ass. A browser hack.

Guest 3

It was a browser hack. You doing the hives?

Guest 1

Yeah. That was me doing everything. I did one take on all the vocals, and then I did a second or third take to do some harmonies and backups.

Guest 1

But, yeah, that was me singing everything.

Guest 1

No auto tune, folks. I was just singing it. So that sometimes I forget that I was a music major.

Wes Bos

Sometimes I forget about that. It was a different life. That that is very good. I don't have any sort of talent like that.

Share