#StackBounty: #nginx #ssl #lets-encrypt #ios #safari Website suddenly doesn't open in Safari and iOS devices

Bounty: 100

I have this website that was working fine until recently. Now users are reporting that it’s not opening in their iphones and ipads.

Doesn’t matter what browser you try on iOS, it just won’t work. Also it doesn’t open in Safari when browsed on a Mac machine. Other browsers work fine though (Mac OSX only, not iOS).

Here is the server configuration:

DigitalOcean + Ubuntu 18.04 + Nginx + PHP 7.3 (Laravel) + Let’s Encrypt SSL + HTTP2 activated

Before I explain further, I wanna mention that using the same type of configuration as above, I have another server which is running fine and is accessible from all devices. They are almost identical (in terms of config and setup) apart from domain name and ip address obviously.

Another tiny extra info which I think is worth mentioning, is that my users are located in Iran.

Here is a list of things that I’ve checked:

  • The domain is accessible using ping command from iOS and Mac.
  • The IP address itself is also accessible, and it can be used to browse the site, it will just show a “invalid ssl certificate” warning.
  • The DNS that were used for the domain are all ping-able from iOS and Mac.
  • If a VPN connection is used, the site can be accessed. Which is really weird because everything is technically reachable, the domain, the ip and the dns.
  • Aside from using a VPN, the site can also be accessible if SSL is completely turned off/disabled.
  • Other public websites that are using Let’s Encrypt SSL are all accessible. So it can’t be a Let’s Encrypt issue.

Here is what I’ve tried so far:

  • Tons of googling. I’ve even came across with a user with same type of issue on here and here and here
  • I’ve tried disabling HTTP2.
  • I’ve double checked my firewall settings, and even tried disabling it.
  • I’ve tried disabling GZIP.
  • I did a ssl test on ssllabs.com and it shows no problem and it’s identical to my working server.
  • Tried using keepalive_disable "safari" in nginx config
  • I’ve tried swapping ssl_protocols values, like only accepting TLSv1.3 or dropping TLSv1.0
  • Tried using ssl_session_cache
  • Tried changing ssl_ecdh_curve to auto and secp384r1
  • Tried changing ssl_ciphers to HIGH:!aNULL:!MD5; and EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
  • Tried changing ssl_session_tickets to on and off
  • Tried using Strict-Transport-Security
  • Tried commenting /etc/letsencrypt/options-ssl-nginx.conf which is used by certbot
  • Tried different HTTP to HTTPS redirect settings
  • I made sure to use private tab when testing to make sure i’m not looking at a sort of faulty cached attempt to reach the site.
  • I did service nginx reload & restart everytime I made a change to config files.
  • I did lots of sudo reboots to make sure it’s not a simple reboot issue.

As a final resort, I even went through the trouble of reinstalling the OS from scratch and setting up the server again. Still Doesn’t work. And no I didn’t copy my config files, I did every command manually and modified config files carefully to make sure I don’t bring any problems from previous setup. This also means I did a fresh SSL request from Let’s Encrypt (using cerbot). Although I don’t know if they generate a new one or the send you the one from last time.

EDIT 1:
I want to throw a random thought in here. I really don’t know about advanced networking but maybe something from Iran censorship firewalls is meddling with the SSL connection and that causes Apple devices to fail establishing the connection. I heard Apple is strict in it’s own way and requires a specific SSL connection unlike chrome and firefox. those 2 might ignore a slight error but Apple devices don’t.

EDIT 2:
I checked the Safari while Wireshark is tapping. It seems only few packets gets exchanged only no matter what I do. Also it seems Safari is using TLSv1.0 even though I’ve disabled it in nginx config. I really don’t know how completely turn it off so Safari won’t even try to use it to communicate with the server.

EDIT 3:
I tried using a different SSL (using comodo’s 3 month free ssl trial program) and the issue is still there… I read some people saying they resolved some similar issues after changing their SSL. I don’t know why it’s not working for me.

EDIT 4:
I decided to change DNS servers of the domain to see if it’s a DNS issue. I don’t know why and how but after I changed them, Safari could load the site briefly. But after sometime it stopped working again.

EDIT 5:
No luck with disabling external script codes in my site, like google analytics ( because their service is kinda blocked for Iranians ). Again, I read people reports implying that some fixed their issue by disabling external scripts.

EDIT 6:
I’m starting to believe this is not a networking or server config issue on my part. It really looks like a 3rd party interfering with the request. I don’t know about Apple and how it handles networking, but I think Apple connection/packeting negotiations is causing some sort of red flag so to speak and that makes Iranian censorship/firewall to drop the client connection.

EDIT 7:
I even changed the server’s datacenter, with a fresh new ip address. the issue still exists 🙁

EDIT 8:
This is the /var/log/nginx/error.log output when it’s set to debug. It shows these lines when I initiate a request from a Safari client.

nginx error.log output

I did try to disable php fpm to make sure it’s not a php bottleneck issue. Also I tried to tweak some random parameters, like bumping net.core.somaxconn to 1024 or increasing backlog in php fpm config files.
Also when I watch the system using htop everythink looks normal, no cpu usage spikes and no specific process jumping to top of the list.

EDIT 9:
I swapped Nginx to Apache2 to check if it’s a webserver issue. Well, it wasn’t.


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.