This isn't exactly a programming question but it's relevant to programmers, and I'm looking for a data-backed, specific answer.
I'm working in a field where the size of the files we create is doubling roughly every 100 days. Nielsen's Law suggests that connection speeds are increasing by about 50% every year, which would mean doubling every 600 days. There is some talk of doing our processing on AWS or some other cloud computing service.
To me, this seems implausible, since the time required to upload the data will soon dwarf the time for processing. However, Nielsen's Law (original article by Nielsen) was made for end user connection speeds, so I'm not sure I can make my point with that.
Does anyone know of a public resource on AWS connection speeds, or institutional (e.g. university or corporation, not residential) connection speeds, over time? I'm wanting to know if it is just larger than residential, but still increasing at the same rate, or if for some reason connection speeds to institutional customers might be increasing faster than Nielsen's Law. Any help in finding evidence on the trend over time for this is appreciated.
I'd think just coming up with a nice pretty graph plotting estimated data download time into the future ought to make your point.
I can't speak for Neilsen's law, but your question in terms of AWS is nearly impossible to answer (in practical, "how should we spend this money" terms).
AWS's connection speeds vary depending on who you are, how much money you have to spend, and where you're located. For example, can you colocate a rack in Reston, VA? How about another datacenter provider used by Amazon? There are three per AZ. If you can, you can likely negotiate much more bandwidth between your rack and theirs than would be available normally. Are you a customer the size of Foursquare? Are you planning on running your jobs on 20,000 instances? I'm sure Amazon network engineering will help you squeeze out every drop of bandwidth you can, and probably write a white paper about it, too. There's rumor of dedicated, non-public-internet pipes between eu-west and us-east. The network map of today's cloud won't likely resemble 2017's at all.
This isn't to suggest your question is wrong or bad, just that it's difficult to answer. As a thought experiment, it's fascinating. As an argument in a discussion about long-term capacity planning and capital outlay, I'm not sure it's as useful.