Possibly the main reason the ladder was on the port side is because all of the starboard side panels are detachable for maintenance and of course a fixed ladder would interfere with this vital function.

I’ve just looked at a photo of the Lysander which shows the exhaust was on the starboard side – possibly another reason for a port side ladder.