Of course the correct way to repair this is to determine which piece is badly worn and replace it. You could remove both right and left brake drums and exchange them -- that would confirm which part is worn (axle or hub or both). While they are off, use Vernier calipers or micrometers to measure all the parts.
Since you don't want to replace the parts, the second best way to repair this is to have the worn parts built back up using metal spraying technique. That works well and will make them like new, but it will be somewhat expensive.
The third best solution, and an old mechanic's approach to this, is to make a shim from thin sheet metal, cutting it exactly to size so that it does not interfere with the keyway for the square key. It will be a trapezoid shaped piece. Something about 0,5 mm thick should work. It is not a great solution, but it is an inexpensive one.
Good luck with your repairs.
All the Best, Chip
