Where I live you just get a letter some weeks before the election. With that letter you can vote at the place that is named in the letter (or anywhere in the same city). If you lose the letter you can still vote with your id-card, but only at the place that was named in the letter.
Easy, isn’t it?
They generate no frequency so the situation you fear is not possible.