Hmm well I suspect that would happen to some degree to many people as the modem is not sitting around waiting for someone to connect. Device number 1 might be using all the bandwidth and all of sudden Device number 2 is jumping in. I don't think too many modems load balance the way you expect and subsequently this will happen especially if device number 1 is doing something intensive.
I often run 2 or more devices off the one 4G connection and if I am downloading movies or torrents on one device my second device often times out. Keep in mind that if you have an average to poor signal the results are often worse.