Hi,
What to do if the domain name of one of my webserver, that me and some lab members use for work related stuff, is no longer resolved by our university DNS? When I first noticed it, I could see no resolution at all while now the domain resolves to a wrong IP. The site can be normally reached on any other network so there is no problem on my side I think.
Should I just wait (now more than 24 hours) or should I try anything? I am entitled to complain to our IT even though the issue is only with this not-really-professional FreeDNS subdomain?
EDIT: apparently some automatism marked this domain as malicious (absolutely it is not, not willingly and not compromised) and somehow DNS resolves to CNAME sinkhole.paloaltonetworks.com.
You must log in or register to comment.
I would migrate the domain. Don’t bother with flakey services. Cloudflare free tier can do some amazing things.
In the meantime set it in your host file to the correct IP to get by.
I see your point, but now I do not think it is FreeDNS fault. DNSChecker.org shows my domain name properly resolved worldwide, and so it has been for months. I also created a second subdomain just now, exactly as the non-working one, and was properly resolved within seconds at my work pc. So I do not blame FreeDNS, I think it is our internal DNS server that is messed up or even hijacked.
Try changing your DNS server in that case!
I tried to set it to 8.8.8.8 but I have still the same result. Can it be overridden at the router level? So far the only solution is to manually add the damn line to etc/hosts.
Probably not your problem but if 8.8.8.8 has some wrong DNS record cached you can flush the cache for one name at https://dns.google/cache and for 1.1.1.1 at https://one.one.one.one/purge-cache/
There are also commands on each of the major operating systems to flush local caches.
It is also possible that DHCP or IPv6 router advertisements reset your manual DNS setting of 8.8.8.8 depending on how you set it.
Another thing that can be happening is that the router or firewall is redirecting all port 53 traffic to their internal DNS servers. (I do the same thing at home to prevent certain devices from ignoring my router’s DNS settings cough Android cough)
One way you can check for this is to run “nslookup some.domain” from a terminal and see where the response comes from.
What does it mean?
nslookup my.domain.com Server: dns.google Address: 8.8.8.8 Non-authoritative answer: Name: my.domain.com Addresses: ::1 xx.x.xx.xxx (wrong IPV4 address from the other side of the world)If I use 8.8.8.8 at home addresses is first of all “address” and is correct.
That looks like 8.8.8.8 actually responded. The ::1 is ipv6’s localhost which seems odd. As for the wong ipv4 I’m not sure.
I normally see something like
requested 8.8.8.8 but 1.2.3.4 respondedif the router was forcing traffic to their DNS servers.You can also specify the DNS server to use when using nslookup like:
nslookup www.google.com 1.1.1.1. And you can see if you get and different answers from there. But what you posted doesn’t seem out of the ordinary other than the ::1.Edit just for shits and giggles also try
nslookup xx.xx.xx.xxwhere xx.xx… is the wrong up from the other side of the world and see what domain it returns.
Interesting, thanks. I think this is what it is happening. Feels like I can put whatever DNS server and still end up with an internal one.
Your host sets it’s own DNS servers, if the router isn’t on the list, they don’t get pinged. Now they could try to man in the middle you, so you could try DNS over TLS, but it’s probably not your issue.
You’re DNS server settings likely never took hold. Like if you use a DHCP client, then override your DNS settings, that won’t take effect until you request a new DHCP connection.
Some Linux distros will have local DNS servers that you always point to which are a pain to update as well. Not sure about Windows and MAC.
good luck man!
Sounds like your university is using a Palo Alto Next Gen Firewall which is intercepting DNS requests and responding with the sinkhole FQDN for anything they deem malicious or suspicious. You can try to override this with DNS over HTTPS but they may also be blocking that. Standard security stuff. You can also probably try to open an IT ticket and request that they whitelist the domain.
So it seems. Do you think this was from the detected user activity? A colleague reported it was using it and it stopped working from one second to the next. Maybe some of his traffic looked suspicious? I am opening a ticket in any case today.
That is possible as well. Those firewalls are capable of packet inspection. If you are using personal devices it won’t be able to see much if you are using encryption in transit but if you are using University provided machines there is a good chance they can inspect all the data you are sending and receiving.
Why are you using a crappy uni DNS? Why not 1.1.1.1 or OpenDNS or even Google’s 8.8.8.8?
That doesn’t help when you run the server and other people have to access your website.
Well, the main point is I would need to manually change this for tens of pcs and its not my job, moreover other people should to the same on theirs. Nevertheless, I just tried 8.8.8.8 on a couple of PCs and I have the same issue! It appears that my DNS setting is irrelevant as it is overwritten down the chain, the only way I can reach the site is put the line in etc/hosts. Could it be?
Yes, your uni might intercept communication on port 53 and reroute it to their DNS servers. It’s possible.
In that case just use a DNS that listens on a different port
I think there are many levels to approach this problem. First off the obvious investigate why your org DNS is having issues. This is IT request they should fix that. They should have SLA on this critical service and not fixing it should escalate to management. There may be many reasons why resolver is not working specially in complex multi site setups. This is the best option as it solves this and probably other DNS related issues.
The rogue approach: On other side if you only host service for handful of users that you personally know and you have ability to edit your hosts file, you can bypass DNS completely. This isn’t ideal as it has to be done one every system and in case your IP changes you will have to do it again. It would largely depend on your level of access to system. If you even can change hosts file.
Alternative crazy idea is to host your own DNS. Change DNS setting on your network configuration. Then point your dns to your org dns. Same problem as hosts file you will need to do that for all systems that need connectivity.
Expanding on own DNS approach you could go as far as hosting your own network. WiFi or switch in case you need Ethernet cable connection. You can buy used enterprise equipment for cheap plug it in l, configure to point to your own DNS and anyone connected to your network would have your settings. Of course this is super shadow IT and I would discourage from pursuing that.
Less crazy and rogue option is to use something like tailscale (or similar) which would have DNS (magic dns). You would need agent installed on every client.
Thanks for the detailed answer, a lot of suggestions are great but unfortunately a bit unpractical. Changing etc/hosts is at the moment the only think working and if the issue is not fixed soon I will suggest to the users that are willing to do so. I would not go as far as asking people to install VPNs and I am pretty sure that buildin a rogue wifi/LAN network will be against any corporate policy and I will be fired :D
This may come down to details of their policies and how they interact and support each department. If it’s for your official work, and I’d say start with a ticket and if they resist then push it up the flag pole and don’t stop. (Assuming you’re not one,) Your PI ought to fight like hell to make sure their employees can do their jobs, and the chair fight to make sure their researchers can run their labs, and the dean much the same, but throwing heavier punches each step up. Really shouldn’t get to that point, but if you can’t do your job, rattle the cages until you can.
I already had contacts with our IT. I originally asked if they could host this service for us as it seemed the normal thing to do. They do not support anything custom (i.e. anything which is not a wordpress site) and just to give me a fourth level subdomain they wanted signatures from half the administration above me. That’s why I’m rogue with selfhosting also work stuff. But I think I can still complain just because their DNS gives back random IPs. This could even be hijacking, no?
I would probably send along the output of
dig your.domain @uni-ipanddig your.domain @8.8.8.8anddig your.domain @1.1.1.1anddig your.domain @your-domains-authoritative-dns-serverif you have that or some similar DNS client tool installed that allows direct requests to specific servers. If you don’t know the authoritative DNS servers for your domain, those are the ones in the NS records.Nice, I am routed to sinkhole.paloaltonetworks.com I am a malicious domain apparently.
Are you hosting a service that is not under your organizations official domain or something? It is common security practice to block newly created domains which may be why your domain is blacklisted if you only recently stood it up.
Do you have a static IP? If not, have you tried some kind of dynamic DNS like DuckDNS?
The IP is static, and is resolved properly everywhere outside my university network
An the issue is only inside the network? I’d complain to IT about that, yeah. Maybe they are overriding the DNS record with their own DNS server or something.
Can you set your own DNS servers on your client devices? Does cloudflare or quad9 resolve it?
I think this is exactly the case, they have some issues with the DNS server and, as some other comments indicate it is possible, they reset my settings for DNS servers at router level. So nor cloudflare or others can help, only the line in etc/hosts works
Cloudflare tunnel is the easy solution here. It’ll cost you a couple bucks a year for a domain name but you’ll have no more DNS issues.
