I think you'd need to keep TDM, S&D and ranked up all the time, and then rotate the other modes. Sorry I didn't make that clear.
Overwatch do this by keeping the two standard game modes permanently available and rotating the other games in their 'arcade' weekly.
Great idea. I also had a concern about spreading the playerbase but i also think the game will be more appealing to gamers that currently play other shooters such as COD if they implemented these additional modes. I really stopped playing BO4 because they did not include CTF in this one. I agree that the special game modes could rotate in like a weekly arcade until the playerbase grows.