I think you can remove the tags using the TortoiseSVN Repo Browser. Open the project in Windows Explorer > right click > Tortoise SVN > Repo Browser. It should display the list of tags on the left side.
There used to be a bug where you couldn’t reset the highest build number (if say you deleted a tag with a higher number than your “latest”), but that looks to have been fixed.
Update: Found a way to batch delete tags. Change the URL at the top to be /tags instead of /truck. Then you can multi-select and delete them.
It will prompt for a message, and it is nice to at least say something, like “Deleting old tags”, because your actions do show up in the TeamServer page of your project.