Would you have to play the other elder scroll games to understand skyrim

No, while it's nice to know the series backstory already, it has no real bearing on Skyrim's gameplay. You'll find books in the game which mention the history, so you can read about it from those if you are curious.