Does body "turn down" its immune system?

I've found when I seriously work out, I get colds back-to-back. I've always felt my physical exertion has a direct correlation with my susceptibility to viruses. There are a bunch of logical arguments why... viruses spreading in communal fitness centers... physical exhaustion weakens the bodies immune defenses.

But I wonder if there isn't something else at play. When you're working out, you presumably increase the rate of new cell development and cell transformation. I wonder if the body has a natural immunosuppressant that tunes down the immune system perhaps with the objective of preventing it from potentially attacking new cells? Or perhaps the body is simply rate-limited (say by vitamins, available proteins, stem cell production, etc) in the number of new cells it can produce simultaneously and not as many immune response cells can be produced when muscle is being built?