Wouldn't it be fun for clients to just appear on the LAN and find the server and be done with it? Well it turns out I happen to know a guy in networking....
Anyway, we don't need TCP first of all. It's great but we don't need it really. My suggestion:
Switch to a model where the server implements SSDP.
https://en.wikipedia.org/wiki/Simple_Service_Discovery_Protocol
And then there's good old multicast as well.
This would solve a number of problems and make this simpler. No more punching in IP address or port numbers. Just start the server on your LAN, fire up the client, and blam it works.