If location of the server does not matters and does not effect the downloading time then why it is recommended to download files from the nearest server?

In most cases, the location of the server (on your continent, or on a foreign continent) DOES matter. For example, if you are in New Zealand, and one of the download servers is in Sweden, your Internet traffic will need to cross two oceans, and that will INCREASE the download-time.
If you can find a server in Australia, then your Internet traffic will be "local", probably an underwater cable, rather than NZ-earth-to-satellite-to-USA-earth-to-satellite-to-European-earth, and will be faster.