It's definitely the DNS server of your internet provider that is causing this issue, their DNS server is not updating fast enough. When you switch off the router and use the cell-tower connection it uses a different DNS server that's why it's usually working from cell service.
Question, when...