When you say your 'clicks are set for this', I take your dialling. If so, it's easy to check.
Set your mag to 20x, make a pellet mark at 55 yards. Mag up to 50x and aim at the same mark, if there's shift, expect to see a distinct right or left POI. Repeat to hearts content
If you hit the same mark it's fine
You might want to shoot from a bench just for the test.