In this episode, Gabriel (Founder) and Thom (Security Team) discuss Scam Blocker. How it works, the types of scams it protects against, and why our ‘bad pages’ list is updated so often.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Show notes: See the full blog post on Scam Blocker.

Gabriel: Hello, welcome to DuckTales. I’m Gabriel, CEO and founder of DuckDuckGo. DuckTales is everything kind of inside DuckDuckGo. Today we have a new topic. I don’t think we have discussed much about security in our browser. I got Thom here. Thom, you want to introduce yourself?

Thom: Yeah, sure. Hi, I’m Thom. I’m one of the security engineers here at DuckDuckGo. I spend most of my time kind of in and out of browser security, product security, that kind of stuff. Yeah, that’s the kind of stuff I love.

Gabriel: Sweet. And I think we’re here today to talk about our Scam Blocker. If you follow our blog, we actually had a pretty big article about this when it launched a few months ago. And so you can always check that out too, but we’re going to tell you all about it here and some of the inside info on how it came together. Yeah, Thom, you want to just explain generally what it is? What is Scam Blocker exactly?

Thom: Yeah, sure. So I guess Scam Blocker is what we call our in-browser phishing and scam protection. It kind of runs in the background and checks websites as you visit them all locally in the browser. And we kind of have a pretty big data set here that we get from Netcraft. So we can protect against all sorts of scams — this isn’t your standard phishing protection. We try and protect against cloned e-commerce sites, fake crypto exchanges, scareware like fake virus pages, and advertising of fake products and stuff. So we have quite a lot that we’re trying to protect against, but this feature as a whole is that warning page that you get when you’re about to visit something that could be scammy or phishing related.

Gabriel: So let’s talk about that distinction a little bit. I guess backing up a little, how did this come together? How did we end up building this and then building it kind of differently than other companies?

Thom: Yeah, so it came from a long way back. Originally, we had this idea that we wanted to improve our tracking protection and all of this stuff — trying to make our browsers as safe as possible for our users. We knew that we wanted to do something in this space, but the challenge was that it’s quite easy to build a feature like this where it ends up looking like you need to check people’s browsing activity — and we can’t do that from a privacy perspective. So we knew that we had to do this in a privacy-preserving way, and we didn’t like the idea of sending any data to Google or Microsoft because they pretty much own this space in terms of browser protections. We weren’t comfortable with that, so that kind of led us down the path of building it ourselves.

Gabriel: Interesting. So like at a high level, our browser has a privacy protection list instead of blocking that we built ourselves because we didn’t believe anyone else was doing it up to the standard that we think it should be. But that’s all kind of behind the scenes on pages that you visit, assuming that was a page you actually wanted to visit. Privacy and security overlap, but as I understand it, some pages you visit are actually bad for you — not because there’s hidden trackers, but because the page itself has malware or scams. Those are the pages we wanted to cover. And in doing that, you need to have a list of bad pages.

Thom: Yeah.

Gabriel: Everyone else seems to be using Google or Microsoft, and all the other browsers are just kind of riding on Google Safe Browsing. But we wanted to go somewhere different. So we found this vendor Netcraft, who maintains a big list, and it turns out they have an even bigger list than Google’s because they cover these other categories, right?

Thom: Yeah, exactly.

Gabriel: Like some of these scam categories that you mentioned are not traditional malware phishing. They’re theoretically legitimate businesses that are scamming you. So for whatever reason, they’re not on Google’s list. Is that kind of how to think about it?

Thom: Yeah. That’s a good way of saying it. Some of these are quite unique. One of the interesting cases I like to refer to is that sometimes even a blog post could be a scam. If this is a blog post advertising a fake product that’s going to steal your money, that’s a problem. A lot of these scam sites start somewhere trusted, like a Medium article or GitHub page, and then send you down fishy paths until you end up somewhere meant to steal your money. That’s the kind of thing we’re looking at here with Netcraft. We get data that lets us look at the source of it rather than waiting for you to click through multiple times to get there.

Gabriel: So we license this data set from Netcraft who’s aggregating all of these scams from different signals. And then what do we do with it exactly? How does it work to be embedded in the browser?

Thom: Basically, we pull this data — it’s constantly evolving, which is one of the challenges. We have to update it pretty much every five minutes on the backend. We pull it, process it, filter out some of the lower-risk things, and then compress it.

Gabriel: Five minutes is so quick. So it’s really happening in real time. I didn’t realize we were doing it that real time.

Thom: Yeah, it’s rapid. If you take a random phishing link now and look again in five minutes, chances are it’s gone.

Gabriel: And that’s because all these people are reporting these things, right? It’s an arms race — things get blocked quick, they switch domains, and all sorts of crazy stuff.

Thom: Exactly. It’s this constant cat-and-mouse game.

Gabriel: Cool. Sorry to interrupt. Every five minutes, we’re updating this list on our backend.

Thom: Yeah, and then we compress this into a small format. Our browsers pull this data every 10 to 20 minutes depending on platform. That’s how the update mechanism works.

Gabriel: Got it. So once it’s sitting in the browser, the browser checks against the list. If you’re going somewhere that’s on the list, that’s when you see the warning page. Are we similar to others where you get a big warning page but can accept the risk? And do all these warning pages look the same or are there different types?

Thom: Yeah, pretty much the same. You get a warning page explaining the case. We have three types of warning pages — they vary slightly in iconography and copy. They’re for malware, phishing, and scam. Malware means you might download something malicious, phishing is about credentials or credit cards, and scam is broader — like a dodgy e-commerce site.

Gabriel: Got it. So any surprises in building this or challenges that arose getting it live to production?

Thom: Yeah, a few. The first one is that we have four browsers — four different platforms. The core part of the feature is constantly updating, but the other challenge is intercepting navigation requests. Every browser does this differently. So we had to map out how each does it and figure out ways to do it efficiently. We pride ourselves on our browsers being quick — we don’t want to affect load times. So we had to make sure the check runs quickly, just before a page loads. There’s a lot to consider. That was one of the biggest challenges.

Gabriel: Yeah, that makes sense. It basically seems like one project, but it’s four big projects — MacOS, Windows, Android, and iOS. Cool. So how has it gone? Any good response? I know we put out a blog post and got some press when it launched. It seemed positive from my view, but from your point of view, what did you think?

Thom: I think we had good positive feedback. One unique thing about this feature is that it’s in the background — its success hinges on people not really seeing it. If loads of people are seeing the error page, then we’ve probably done something wrong. But overall, it’s gone well.

Gabriel: Yeah, that’s a good point. It’s like our other privacy protections — always on, not breaking sites, contributing to peace of mind. It’s protection that’s there, not in your face.

Thom: Precisely. People who’ve come across it said it works well and gives them peace of mind.

Gabriel: Cool. So it sounds like it kind of went off without a hitch. Is there anything left to do now? Are we kind of in maintenance mode with it?

Thom: Yeah, pretty much in maintenance mode. We have about three or four people monitoring metrics. But we’re exploring ways to enhance