What exactly does it actually do?
If I do -P 100 on the client side,
Does it open 100 ports and make 100 connections to a given server in parallel?
Is it what's normally used to simulate a server maintaining "X (simultaneous) connections in parallel"?
Note: this answer is iperf 2 related. iperf 3 is single threaded.
Yes, if the client computer supports threads there will be 100 traffic threads and 1 reporter thread. The traffic threads will send traffic to the server threads "in parallel" and the reporter thread will output statistics to stdout per the -i value. How much "in parallel" depends upon CPU cores and OS scheduling. More cores can allow more traffic threads to truly run in parallel. When the cores are exhausted the threads will be scheduled by the underlying OS scheduler. (Note: In 2.0.8 or greater and with Linux, the traffic threads can be set to realtime scheduling using -z or --realtime.)