Hi Tomo,
I am grappling with a similar issue on my bike currently. This one has a rod conversion that seems to have been done quite well but there was virtually no contact between linings and brake drum when I started. (the bike is new to me). I found the hub/drum had been machined and new linings fitted and it now seems to be down to getting the shoes to bed in.
If properly set up the brake should be fully on in less than an inch of movement and this suggests there is some freedom in the system that should not be there.
I would take things apart and check the wear pattern on the shoes and the squareness of your drum to the back plate, also check everything is centralised (the pivot pin can float when released and the wheel spindle also has an input). Ideally there should be full contact between the shoes and the drum otherwise you will get that lost motion / spongy action. Other things that contribute are any loose fits between pivots and pins, rods through clevises wear in bushes etc etc. Operate the lever and watch all the movement checking for any lost motion i.e. instead of, or as well as, rotating the shaft the shaft moves perpendicular to its axis due to wear in the support bushing. Watch also for things twisting off or bending out of line too.
If you were preparing the brake for racing you would machine the drums true, fit oversize linings and machine them down assembled on the back plate, to just fit tightly in the drum. In this way you are aiming for full contact and minimal clearances. The brake should only improve with use... until it starts wearing out again!
Good luck with it all anyway.