So the problem you're running into is designed to stop people shooting themselves in the foot by opening the vpn on a port that is already in use.
In your scenario, this is the case, but in the process you actually free up the port and use it again.
I'll have a think if there's anything that can be done to work around this.
Your use case is a little unusual, but I get it.
