You ask: What regulators do you use?
A lot of people here use the DVR2 Regulator supplied by
http://dynamoregulators.com/I have them on three dynamo machines, others here have many more than that, and their reputation is very good. Available in negative or positive earth and they work at 6v or 12v.
They require the dynamo to be wired the Lucas way, ie the field coil from F terminal to earth, one brush to earth and the other brush to D terminal.
If you wanted to stay with the dynamo wired the way you have it now (with the field across the F and D terminals), then a JG unit should be a straightforward fit, see
http://www.davelindsley.co.uk/jg-electronic-regulators.html. I used one of those for 15 years until replacing it with a DVR2, and it still worked fine.
If you wanted to look at another option, the Podtronics dynamo regulator is also highly regarded :
http://www.podtronics.net/DC_regulator.htm. This one looks rather like what you have got, but it is not the same thing inside. There are other units available, but I don't have any experience with them on my own machines. See, eg,
http://www.aoservices.co.uk/data/v-reg.htmThe one you have unfortunately does not have a great reputation. It does not seem to me it is doing what it should, whether because it has got damaged in the process of playing with it, or because it was defective from new. Price is relevant - you do tend to get what you pay for with these things.
If it were me, I'd get a DVR2 and go from there, but any of the above would be better, I think, than what you have.
Replacement mechanical regulators are also available - but their quality can be doubtful, both electrically and in terms of construction.
The most expensive option would be to swap the whole charging system for an Alton alternator (which now comes with a good Podtronics rectifier/regulator and has an improving reputation, although I have no first-hand experience. See
http://www.alton-france.com/generators.html).
So there are some good choices, which ought to provide plenty of power, protect the dynamo and battery - and enable you to put these woes behind you.