It would have increased weight and cost at the time of change, basically when jest came of age, but we’ve overcome those problems now. There would probably be more flutter on a fin with an upright leading edge. The shorter the body length to diameter, the bigger the fin. Lok at A380, that outrageous fin on a dumpy body. If the Body was twice as long the fin would get smaller (If it’s further back from the turning moment it has to do less work).
Single engine prop jobs have a non uniform fin. i.e. in cross section the shape more resembles a wing, with one side nearly flat and one side curved. This counters the shear from the prop. In twins and upwards, the shape is the same at either side of the chord.

Isn’t there an a/c with the fin facing forward? Looks like it’s on backwards?

I think you are referring to the Mooney M20 series of machines where the fin leading edge is vertical and certainly does look like it is back to front. Lovely pitch trim system on them too, the whole tail cone complete with fin, rudder, tailplane and elevators pivots up and down. Never heard of it causing any problems though for all it is an unusual idea.