When I use the webclient example to download around 16 MB file, the memory usage become around 50 MB and remain after the socket closing. [The same happen with my program of course :))] If I set the SecureProtocol to None it works well.

I wrote a multi threaded server and It can consume my resources quickly.

The behavior you're seeing is due to garbage collection. The .NET framework will not start releasing memory until the system is low on resources or until the GC.Collect() method is called.