3 and 5BA have become a harder to get these days, and choice of lengths is pretty limited as you say, being confined to 3/4 and 7/8 in c/sunk or caphead from my usual go-to site, BA Bolts. If 7/8 is too short owing to the rubber spacer, 'tis a pain . . . .
3BA's major diameter is 4.1mm and the pitch is 35tpi in round numbers. M4 x 0.7 is 36tpi in round numbers. So I'd have thought would serve in what is a non-critical application?
If the different thread angles are a worry a dab of loctite should do it - or run an M4 tap down the holes if you've got one handy?