Tuesday, September 10, 2019

How to Mitigate the Firefox DNS-over-HTTPS Implementation


I was reading El Reg this morning whilst having some brekko and spotted an interesting article relating to DNS-over-HTTPS (DoH). It seemed to me that the approach taken by Mozilla is a double-edged sword of worrying proportions.

You can TL;DR to the Solution section at the bottom if you're already tired of hearing about the topic :)

On one hand I commend them for taking these steps to further ensure the privacy of the individual - DNS itself was designed in a time where data scraping, intel and spying were not immediate concerns of the internet (as was), but a way of translating machine-speak (IP addresses) into human-speak (domain names for URIs) was. It was critical to the newly born world wide web at the time and the emphasis was performance and simplicity.

Wind the clocks forward to the current climate some twenty plus years later and we now have our own ISPs hoarding our DNS requests, in order to augment their data pipeline to sell to advertisers. Of course there are many more data points the ISPs collect but this is the key focus on this issue. They'll also redirect you based on whose paid them the most that month, to sites selling wares or other paid-for content.

Now for the majority of people, Mozilla's approach here will drastically improve privacy  with respect to the ISP data collection and censorship. We need this type of approach to drive a change in behaviour within the advertising and data analytics industry.

Mozilla will do this by embedding a DoH client within their Firefox browser, and - apparently although I may have misunderstood - will enable this by default. There is the capability to opt-out and I'm sure the about:config page will inevitably allow a tweak in settings. Their chosen DNS provider is not OpenNIC or similar but Cloudflare, and whilst Cloudflare have pinky-sworn that they'll uphold good and only use the additional DNS data flow to support their resolver services, we should throw caution into the wind.

As I said, for the majority of people with simple connectivity needs who like seeing ads, and are happy being click-jacked to hostile domains this Firefox change will probably slightly improve their privacy. Especially in the US where ISP's openly sell customer data this closes off an entire pipeline for analytics.

An increasing number of people and businesses are running PiHole or similar LAN DNS configurations to exclude malvertising from all their devices on the entire network. The change being enacted by Mozilla has the potential to completely circumvent this anti-ad safe-space capability. "Free" DNS providers offering ad and parental filtering are no strangers to paid relationships with malvertising organisations so custom DNS configurations are really the only solid solution.

Of course DoT and DoH were designed to stop ISPs and other bad-actors doing exactly what PiHole is designed to do - which is why there's such a double edged sword here.

Now add in standard enterprise networks - how will this impact local (enterprise) resolved names for servers and services that are not exposed internally? Well future Firefox users will likely not be able to access these services without some serious per-device configuration by admins, and an additional group policy to disable user configuration to opt-in. The way Mozilla expects a canary domain workaround to operate, which will signal that DoH is not available / appropriate...although it can be overridden in a user-configured setting on the browser.

Some mobile apps embed DNS resolver clients to maliciously use their own HQ DNS, which is why it's often so difficult to block 'invisible' services such as Facebook APIs. We want to bring this back under control too.


I've been operating a network design for some time now which takes a caching domain name forwarder, various network security tools, custom scripts to acquire and parse ad-filtered domain rules, firewall rules (shorewall or iptables work) and some denting-of-desks-with head. But it works.

It also means that Firefox's new DoH implementation will work in my favour when I'm on the road - so it's a double bonus.

The aim is not to censor or inhibit but to remove malvertising and ensure no ISP-induced censorship is possible. In all likelihood this is exactly the type of approach your ISP will be using but for their own aims. As a side effect it also eliminates a large portion of tracking without acting like an overly protective parent. You could potentially modify your PiHole & router configuration to adopt this approach too - although I'm unable to test that statement.

During the development of this approach I also did a little research and decided I wasn't happy using Google DNS, Cloudflare, ISPs (e.g. BT / EE, Vodafone or Three) and anything to do with Facebook. So not only did I want the resolver to filter out malvertising but also ensure that no devices or apps were embedding DNSCrypt et al and ignoring this filter list.

What I've not got a full solution for yet is regularly updating what I'm calling the "hostile DNS" list e.g. FB and Google. On with the details.


Components of the solution:
  1. Unbound - caching domain forwarding resolver. Good bit of kit and low imprint mem / cpu
  2. iptables - rules configured on the routers of your network, ideally in the inner router if you have multiple
  3. script to download malvertising domain lists regularly, and parse these into unbound format (example script available here)
  4. a list of hostile DNS domain names. For this example I've used:
    1. onedotonedotonedotone.cloudflare-dns.com
    2. dns.google
    3. resolver1-fs.opendns.com
    4. resolver2-fs.opendns.com
  5. a list of hostile DNS ip addresses. Explanation of why the IPs and DN follows this list
  6. A script to resolve the domain names from #4 into a list of IPs on a regular basis, to account for the IP changes (template to work from is available here)
  7. (Optional) A VPN. Doesn't matter what else you do the ideal is to ensure that your ISP cannot track the connection to the DoT server in the first place. Whether or not this is necessary is debatable though.
I'm not going to go into detail about how to deploy and configure Unbound with DoT (properly) and DNSSEC, but that's what my baseline is in this example. Plenty of guides elsewhere on the web.

This post is going to focus on the acquisition of hostile DNS domains and IPs, plus the redirection of DNS traffic back to our known (friendly) DNS or resolver.

The basic tech-stack and process involved in delivering regular DNS management

Firewall Rules

In order to make this work we need to bring together a list of IPs that we consider hostile, the IP of our friendly LAN DNS which we've got up-and-running and a sprinkling of knowledge of firewall rule management.

What we're essentially going to do is redirect all traffic on DNS, sDNS and some (known) custom DNS ports back to our friendly DNS server. The DoT - and perhaps DoH - will not be able to authenticate and likely fail for port 853 requests, but embedded DNS in apps always has a downgrade or alternative. DoT & DoH authenticate the endpoint to ensure they're speaking to the DNS they're expecting to. If the hash or certificate does not match it assumes MitM and should return a SERVFAIL - I've seen some clients get NXDOMAIN in some cases, which I don't believe to be appropriate.

I've noticed one - the TSB banking app on Android - doesn't like this at all. But as the app is so s**t to begin with I tend to use the website on my mobile anyway. When I get time I'll take that apart with an EvilTwin on ParrotSec.

Anyway, we want to plug in some firewall rules and the first one is simple; we want to re-route all traffic on our DNS port(s) that isn't originating from our LAN DNS server back to our LAN DNS server. Confused? Here is some crayon work explaining it.

So flow #1 is the original DNS request from the client; the firewall rules on the router determine this to be a DNS request (e.g. plain UDP, TCP, DoT or DoH) and redirect it to the LAN DNS in flow #2; flow #3 is the logical response back to the router - but with the mangle in the firewall rules it'll appear to the client that the response is returned from the IP it requested, the LAN DNS will actually respond.

Breaking it into two parts then, the iptables rules that sort out flows 1 & 2 (one each for tcp and udp):
iptables -t nat -A PREROUTING ! -s LANDNSIP -p tcp --dport 53 -j DNAT --to LANDNSIP:53
iptables -t nat -A PREROUTING ! -s LANDNSIP -p udp --dport 53 -j DNAT --to LANDNSIP:53

And the critical bit following that is to masquerade the response back (again one each for tcp and udp):
iptables -t nat -I POSTROUTING ! -s LANDNSIP -p tcp --dport 53 -d LANDNSIP -j MASQUERADE
iptables -t nat -I POSTROUTING ! -s LANDNSIP -p udp --dport 53 -d LANDNSIP -j MASQUERADE

The above will deal with all plain DNS traffic, and can easily be broken out into a for loop in a shell script - most OpenWrt routers have a firewall script in their admin section, so it's easy to apply to home and SME routers. I decided to block a number of ports I'd tracked for DNS so ended up with the following:
for port in $dnsPorts; do
    iptables -t nat -A PREROUTING ! -s $targetLanDns -p tcp --dport $port -j DNAT --to $targetLanDns:$port    
    iptables -t nat -A PREROUTING ! -s $targetLanDns -p udp --dport $port -j DNAT --to $targetLanDns:$port
    iptables -t nat -I POSTROUTING ! -s $targetLanDns -p tcp --dport $port -d $targetLanDns -j MASQUERADE
    iptables -t nat -I POSTROUTING ! -s $targetLanDns -p udp --dport $port -d $targetLanDns -j MASQUERADE
For reference, those DNS ports are currently: 53, 5353, 853 (tcp only), 41895 (tcp only perhaps).

You'll end up with firewall tables full of:
DNAT  udp  -- !LANDNSIP  anywhere   udp dpt:853 to:LANDNSIP:853
MASQUERADE  tcp  -- !LANDNSIP       LANDNSIP      tcp dpt:853

We also need to ensure that if hostile DNS services have thought of this approach and decide to use custom ports, we block that route too - by redirecting all traffic to those servers to our LAN DNS port 53 (udp) or 853 (tcp).

Exactly the same type of approach as above but without the port filter, e.g.

## Known bad-actor DoT servers which may be embedded within mobile apps etc, should be redirected too for specific sDNS port
# Need a CA-backed TLS cert to do this properly
for dnsAddress in $dnsOverTlsServers; do
    iptables -t nat -A PREROUTING ! -s $targetLanDns -p tcp -d $dnsAddress -j DNAT --to $targetLanDns:853
    iptables -t nat -I POSTROUTING ! -s $targetLanDns -p tcp -d $targetLanDns -j MASQUERADE

Cron Jobs

Forgive the non-Windows parlance but this is job scheduling that can just as easily be done on your scheduled tasks cmdlets and Powershell (and just as well).

So I tend to have a daily reboot policy on routers in most environments in case mem. resident compromises have actually been effected unnoticed, with more frequent reboots on grids in more secure environments - make sure you plan your malvertising list fetches and hostile DNS resolver updates to avoid any high utilisation or downtime.

The latter script - which updates the IPs which are associated with the hostile DNS - doesn't need to be run daily as the IPs may only change once a week. Perhaps on a Friday morning or afternoon would be best before those providers clock-off for the weekend. No-one does releases on Friday, right? :)


So the above has been mostly operational here for almost six months. I enhanced it with some firewall rules back in May this year and I'm now looking at Android and iOS apps which people have reported "misbehaving". Suspect that's DNS related - either directly because private DNS cannot be accessed or the developers tied service calls and web applications into advertising scripts & libraries. Will take a look in more detail when I get time.

In other words, the secured domain name resolution and caching is worked as expected :) 

Thursday, September 05, 2019

Insecure Security at the DMA

It's been a while since I've had a chance to write up anything, having been so busy with consultancy work over the last few years.

A number of people have asked what steps I'd take to keep your phone numbers off spam call lists and the answer is simple - don't give it out in the first place! However that's not really a practical approach is it?

Having had to go through the pain of successfully chasing spammers and cold-callers in the courts over the last half a decade (because the regulators for advertising and data protection weren't willing to lift a finger), I've devised more manageable approaches.

More likely what will work is burner numbers from platforms like Hushed, which allow you to create a number (at a price) for a set period of time. I find this really useful for CVs (resumes & business profile documents for tender submissions) with companies I've not worked with before or jobsites I know not to trust; and also email signatures at client site.

Often the level of information security requires that I do not move client information off their own systems necessitating an email address on their platforms - they also usually prefer a signature so I supply my own company details and a VoiP / burner number.

Once I've finished the client visit or project there's no need to get in touch so the number is removed. Same thing with tender documents and resumes - once I have enough clients I close down the entry on the jobsites or notify the companies that those tender documents are out-of-date.

Of course with your own phone number it's a bit different so wherever possible use the Telephone Preference Service. It's a Direct Marketing Association-led initiative to help it's members adhere to PECR and the Data Protection Act. However that's not really true as PECR states that no-one can send you unsolicited messages (section 22) or cold call you (section 21) without your prior consent or purchase of goods. The Data Protection Act states that your data cannot be stored, processed or used without your prior consent either - which means that the DMAs explanation of the TPS mechanism dodges the issue at heart - where did they get your data and what makes them think you want to speak to them in the first place?

You see, if a spammer or cold-call centre had asked you for your permission to acquire (c.f. buy leads database) your data first and you'd said "no", the TPS list would be unnecessary.

The concept of TPS then must focus on the scenario where you've engaged with, for example, a company to buy their products but never specifically stated that they may use your details for marketing purposes. If, in this example, the sale does not complete you should always state that the company may not re-use your personal data for any reason. If they were happy to store your data received verbally they must also accept the notice to destroy that data verbally.

This stops the issue at point although if they later spam call you, you'll need proof of that original conversation - email or support tickets are usually easiest.

Once you're on the TPS lists DMA-linked organisations must make it their priority to ensure that they regularly update the copy of the TPS phone numbers lists they keep - this should ensure that you don't get cold-calls from UK-based or UK-operating organisations.

Every year they'll send you a reminder email to renew your TPS registration - which is ridiculous. If you haven't spoken to a company for over a year why would you suddenly want to start getting cold calls and spam again? It's bad enough we're playing cat-and-mouse with the malvertising platforms that we shouldn't have to deal with the direct approaches too.

This years re-registration was a little more worrying - the email I'd received not only explained why and what... but included the username and password I'd need to log-in and verify re-registration. As an aside this is terrible practise - instead of account details I could register with initially and tidy away into a nice, secure password manager - they're not mentioning any account creation and sending me a username-password combo in a plain-text email! Added to that is the fact that the password is entirely numeric and less than ten characters. It'd take one of my Raspberry Pi's & John The Ripper the grand total of about 30 seconds to brute force that (assuming a hash had been acquired).

Looks like a Perl-based form submission - I hope it's appropriately secured but I'm not going to even think about taking a look without prior engagement with their SOC.

The domain they're using for renewals is "secure.dma.org.uk". Normally I'll use Tor for the unsubscribe / renew links I get to minimise the intrusion - although each link will have an embedded tracking mechanism Tor will prevent OS, screen res, lists of plugins etc. being available for scraping and trend analysis. There's no lawful reason to exclude Tor traffic for a service notification / required intervention.

Before I make the next statement I want to stress that this may be a Tor-specific issue and not necessarily related to the DMA at all, as the CA verification works fine in Firefox: The URL immediately gave me a "Unsecure [sic] site warning" and on closer inspection showed a self-signed certificate with issuer "Digicert Global CA G2". Bit odd. That's usually the middle cert in the chain back to the Digicert Global Root. The self-signed bit is what usually causes that red padlock in the browser address bar if you weren't aware already.

Not sure why but Tor seemed to have missed out the CA repository, but if I didn't know any better this would also look like a MitM attack.

I sparked up Firefox and used the same URL - all fine. Verified CA-issued cert matching the domain and domain name resolved via DoT + DNSSEC. And no, we don't use Google DNS or similar.

Ok, ok so perhaps a bit of click-baiting in the headline but quite a good example of what to look out for. I think it looks like there's some Tor exit nodes which are being proxied / filtered so I tried generating a new Tor circuit and then changing bridges but this didn't change the issue.

The DMA domains are not on the Tor blocking lists and other domains seems to be fine.

Regardless I'm not going to put any personal data into a web page that for some reason arouses suspicions. I used a different browser and checked the domain ownership vs. historic TPS emails and all seemed to check out.

TPS registration renewed via Firefox instead - perhaps one day the DMA will get their acts together and make this a permanent opt-in (on the basis of opt-in-by-default at their member organisations) with TPS as an elective opt-in for categories of products and services you want to hear about. This would even work well for consumers buying a new / previously used number.

I'm sure we'd all race to sign up for cold calls and spam emails after all :)