In 1954 BSA were still running the Plunger frame as the Swing Arm models were introduced. The battery mountings are different, so to define the actual model would be useful. If a Plunger model, has it still got the air filter housing and battery mounting? They are rare and quite pricey to replace with original parts. S/A models have a battery tray under the seat, between toolbox and oiltank.
Usual wiring scheme was to have the brake light on it's own little circuit. Getting this to work should be simple, but poor bulbs, failed switch, dirty contacts, rotted wiring, corroded connectors and a poor earth return all gang up. Start with a good charged battery, a circuit diagram and a multimeter and be prepared for non-standard bodges.
Yes, the head and tail lights will work without a battery, powered directly by the dynamo output. My guess is that the bike will be hiding a few surprises. One of which is a good magneto that won't stop sparking....now that is a GOOD surprise. The cure is simple, by following CB's advice. I'm sure a personal message will help sort this one.
Welcome to the Forum. Tell us how you fell into this bottomless maelstrom, over in the intro section. We love pictures.
Swarfy.