1. We know that the average BVR AAM is usually qouted with max speeds of some mach 4, perhaps going up to mach 5. Doesnt matter if its amraam, sparrow, mica or r-27. Now those speeds are usually qouted as theoretical maximum, when used at high altitude. Much thinner air means the given impulse/drag ratio of the rocket motor will give the missile greater speed which will give it greater energy, thus giving it greater range.

It is also said the same missile fired near sea level will have much shorter range. The difference in air density at sea level and at, say, 40.000 feet, is some 250%, does that mean that the maximum speed of the missile is also going to be reduced by that amount? Sea level density is 1, 40.000 feet density is little under 0.4 How much will max speed be at sea level if it was mach 5 at 40.000 feet? mach 2? Or something else?

Ok, let’s talk about projetile (missile) that achieves M4 at 40k ft.
So, at M4 (40k ft), such a projectile will have skin temperature of ~650°C and a Dynamic Pressure (essentially drag) it needs to overcome of ~210KPa. Since the temperature in tropopause is constant, we’ll work with Dynamic Pressure.
Now, if the launching aircraft descends to 0 ft (sea-level) and fires the same missile, it will most definitely have reduced range. The projectile speed that produces Dynamic Pressure on 40k ft of 210KPa would be ~M1.7, at sea level.

So yes, the M4 at 40k ft projectile would have top speed of ~M1.7 at sea level (outright speed depends on launching aircraft) and average mission speed much lower. The impact in range is even greater and falls to 1/3 against optimal (non-maneuvering) target, or way less, essentially putting AMRAAM (or other MRAAM) in WVR class weapon, at most extreme conditions.

This calculation has been given for standard atmosphere and generic missile and particular models will have different outright performance.