also remember ricochets count as leaving if they leave, especially important if you're shooting knockdowns...
depending on your neighbours, it might be worth pre-empting any situation with a chat, perhaps showing them the safe backstop, and letting them know about how long and how often you intend to shoot. Most people i've heard that have had issues with neighbours have had issues with noise.
i'd also think about insurance as well... it's a small price to pay for peace of mind, and demonstrates that you're pro-active and covering all bases. If you can flash a club card as well, it all points to a responsible shooter just taking advantage of a bit of garden practice, rather than perhaps appearing as something else that people might worry about.
some years ago i had a visit from a neighbour about 4 doors up, who could hear the pellets in the air and thought they were coming his way. When I invited him to come and see the setup without any prior notice, on the spot, the offer seemed to reduce his fears... in the event it was just the noise of the pellets in the air, and the backstop he could hear... and i wasn't even shooting in his vague direction.