Derivation for Height of Distant Objects Obscured by Earth Curvature

Correctly calculating what they expect to see if the Earth is curved is the most common error made by the Flat Earth community that I have seen. In fact, I would almost say they get it wrong Every Single Time (or else they wouldn't still be Flat Earthers).

If you don't care about the math you can stop here and just use my calculator:

FEI Horizon Calculator

---

Full Derivation

The most common error made is to try to calculate the Drop Height from a tangent off the surface - this is shown below as height (D) and then use this as the value that would be "hidden" at that distance. This is just clearly wrong I find it astonishing that this is still going around.

Every time you see someone using "8 inches per mile squared" and talking about how much of some object or city should be hidden, you know that they don't know what they are talking about. This formula is just an old surveyor estimate that is only close for the first 100 miles, after that it's useless, but more importantly Drop Height is not the same as Obscured which must also take into account the viewers height and any terrain variations in between.

Unless the center of the lens of your eye OR camera is exactly half-way under water or buried in the dirt you are NOT viewing from height of 0, exactly at the surface. Not even if you sit your camera down on the ground. It literally has to be half-way buried/submerged.

Hopefully you can read this and understand the issue and at least stop using the wrong math.

What we want to calculate is shown as B - and then we also want to understand the impact of atmospheric refraction, which is a vastly complex subject but we will look at the simplest case considering just one sightline and just using a conservative curvature of refraction.

Figure 1.  Image from David Ridlen box.com - note that D should actually point toward Earth center)

Unless you have convinced yourself that measuring (D) is the wrong measurement there is really no point in proceeding - although I will show at the end that my formula and 8"/mi² give essentially the same answer for the first 100 miles (mine is slightly more accurate) at a height of exactly 0. But if you try to use a height of 0 to evaluate how much of a distant object is obscured then you are using the math to lie.

Pythagorean theorem

Before you proceed I recommend you (re)familiarize yourself with the Pythagorean theorem which says that, given a right triangle the area of side c is the same as the sum of the area for side a + side b:

Figure 2. (image credit wikimedia)

Mathematically that is: a² + b² = c² where c is called the Hypotenuse (the line segment across from the right angle corner, will always the longest line segment).

We are going to use a slightly more complex version in the form of:

a² + b² = (c+δ)²

and

a² + (b+δ)² = c²

where δ is our "delta" - some unknown measure added to one of the sides that we need to calculate.

We will be using the following diagram to guide our calculations. To keep this derivation as SIMPLE as possible I'm going to solve it in two parts for you, each one using the equations above -- and then we can just substitute the first one into the second equation and get a formula we can use.

I'm also going to give you links to Wolfram|Alpha so you don't even have to understand the math very much.

Here we go...

Geometry for Height of Distant Objects Obscured by Earth Curvature

Figure 3. Using two right triangles to find height obscured at a distance

First of all our observer is at h₀ and what we really want to know is h₁ -- this is the amount that will be obscured beyond the horizon at our total distance d₀

Think about what values on this diagram we usually KNOW.

We can see that we are working with two right triangles, and we have labeled the sides in a hopefully obvious way.

We can figure out the expected Radius (R) of the Earth using a formula given the latitude of the observation or we can use an average value if we aren't sure.

We often know about high up the observer is (h₀) - or we can make some educated guesses and then see if the calculation fits what we see. If we aren't pretty sure then we have to calculate the highest elevation we think the observer might be and the lowest -- that gives us a range of answers.

And we know the distance to the object (d₀), or we can usually get "pretty close" with tools like Google Earth. However, please note that Google Earth is giving you what is called the Great Circle Distance which is the distance along the Oblate Spheroid - straight-line distance would be less. There isn't much difference for distances like 100 miles but there is some. Just know that if you want more exact answers you have to use more exact inputs and use the straight-line distance to your object. YOU are often the source of most of the error -- do not trust your inputs to be super accurate which means any calculated values are going to have a significant error bar. Again, you need to do the calculations with a RANGE of values in mind and check to see just how much error that introduces.

That gives us...

Inputs

h₀ = elevation of observer
d₀ = total distance to distant object
R = Earth Radius (we will calculate this based on our latitude rather than using an average value)

Outputs

d₁ = observer eye-line distance to Horizon
h₁ = height of object obscured by horizon (positive when √h₀ √[h₀ + 2 R] > d₀)

Equations

The first equation we will need is to figure out the value for d₁ which is the distance to our horizon point. We will need this value in the second equation.

We can plug in these values into the Pythagorean theorem equations giving us:

(R+h₀)² = d₁² + R²

When we solve for d₁ we get equation (1) d₁ = √h₀ √[h₀ + 2 R]

We can be very certain of this equation because I've provided the Wolfram|Alpha link which solves it for you.

And the FIRST thing we need to do is see if our observer can see GREATER THAN your total distance. If this horizon point (√h₀ √[h₀ + 2 R]) is GREATER THAN d₀ then your total calculation needs to be a NEGATIVE value, meaning. ONLY where √h₀ √[h₀ + 2 R] is LESS THAN d₀ then it will be positive h₁ value.

Next we just plug in the values for our second right triangle (on the right):

(R+h₁)² = d₂² + R²

And we solve for h₁, giving us equation (2) h₁ = √[d₂² + R²] - R

We don't know d₂ but we do know that d₂ = (d₀ - d₁)

We can now combine these equations to give us our final formula (or you can calculate it yourself in the two separate steps since you need the first result anyway). You can simplify this equation further if you make a few assumptions but I don't want to do that.

h₁ = √[d₂² + R²] - R
h₁ = √[(d₀ - d₁)² + R²] - R
h₁ = √[(d₀ - [√h₀ √[h₀ + 2 R]])² + R²] - R

This is our formula!

Height of Distant Objects Obscured by Earth Curvature = h₁ = √[(d₀ - [√h₀ √[h₀ + 2 R]])² + R²] - R

We arrived at this equation using nothing but simple geometry and the Pythagorean theorem so it's very easy to verify this math.

Showing match to 8"/mi² at height of exactly ZERO

Just to show that this equation simplifies to the 8.002"/mi² equation AT HEIGHT OF ZERO (0) let's test it for 6 miles at a point where the Earth radius is 6371000 meters (this is the average value). I use 8.002 because it's more accurate than the 8" approximation.

8.002 * 6² = 288.72 inches [Wolfram|Alpha] or ~24.06 feet.

√[(d - (√h √[h + 2 R]))² + R²] - R, d=9656.06, h=0, R=6371000 [Wolfram|Alpha]

Which gives us 7.31749 meters or ~24.007 feet. We have a good match here. Which gives us some additional confidence that our math is correct.

Working with the equation, examples

Ok -- but let's say we have our camera at just 1 meter above the ground?

We check our horizon distance: √h √[h + 2 R], h=1, R=6371000 = 3569.59 and that's less than d₀

√[(d - (√h √[h + 2 R]))² + R²] - R, d=9656.06, h=1, R=6371000 [Wolfram|Alpha]

That shows that only 2.9 meters will be obscured - considerably less than the wrong equation.

And at 10 meters what happens.

We find that √h √[h + 2 R], h=10, R=6371000 = 11288.1 [Wolfram|Alpha]

From this, we  know that at 10 meters we can see the distant point all the way to the base and past it, because 11288.1 is > 9656.06, we get -0.21 meters from our equation (remember that when you take a square root you don't inherently know if it is negative or positive so you have to figure that out as I have described).

√[(d - (√h √[h + 2 R]))² + R²] - R, d=9656.06, h=10, R=6371000 [Wolfram|Alpha]

We haven't even added in refraction yet. And come on, you KNOW refraction matters - you see mirages, you see light bending in water, it is clearly messing with the path of the light.

Refraction

Calculating the EXACT Refraction is complex in practice because every TINY millimeter of air can be a different temperature, density, and humidity which all affect the actual amount of refraction. And Every Single sightline from you to another distant point can have a different amount of total refraction. However, the total path integral sums up to just one number and we can express that value in terms of % of Earths radius. 0% is no refraction and we just use the equations above exactly as they are. But to account for refraction we can also add a value to the Earth's radius -- that amount can be positive or negative.

We are ONLY going to consider fairly nominal conditions where the refraction is usually between about 7% and 14% - it can go outside those values under many conditions but these are conservative boundaries.

Let's add just 7% for refraction to our last equation, this 7% is simply added to the Earth Radius to account for the refraction:

Our distance to horizon point is now √h √[h + 2 R], h=10, R=6371000*1.07 = 11676.5 m [Wolfram|Alpha]

Conclusion

Therefore, at 6 miles, from 10 meters up under nominal atmospheric conditions we expect to see roughly 11677 meters distant. Refraction here added 388.4 meters to how far we can see.

We have derived an equation that we can feel confident about using to estimate the height of a distant object that would be obscured by Earths Curvature and we can see how it varies when the various factors are changed -- and we can also see that it is very sensitive to certain values, those first few meters of elevation really make a big difference, refraction matters more over longer distances.

And we've shown that 8"/mi² is absolutely not the correct equation to be using when you want to know how much of some distant object should be obscured (given the oblate spheroid model of Earth).

In other posts I will use these equations to show that the observational evidence is actually in very good agreement with the estimates we get from this formula.

Addendum

I've noted elsewhere that many people incorrectly use the distance as given by tools such as Google Earth to calculate the amount obscured of a distant object - but these tools give you the distance as the length of the arc from location to location. This is technically not the correct distance to use for the above calculation which expects the distance along some tangent.  If the distance is under 100 miles then the error is small enough that it usually doesn't matter more than a fraction of a percent.

If you want to be more accurate you can use this formula where d is the distance over the curve and you'll get back the amount obscured based on that.

Figure 4. formula: r/cos[d/r - asin(sqrt[h×(2×r+h)] / (r+h))] - r

You'll see a similar formula used in the horizon calculator when you specify the distance to the object is over the curve, in that version I use the horizon dip angle which I already have in hand but the formula are equivalent.

Other Tools

Here is a 'GeoGebra'-based calculator, based on Mick West's model but I tweaked it a bit.  It reads out both the straight-line distance and curve distance so you can use either one to position the Target.

Finally I note that to calculate a better estimate for Earth radius at some latitude you can use the following calculator or formula:

Radius at given Latitude on Oblate Spheroid


There is a VERY BASIC 'calculator' for height obscured BUT, it doesn't allow you to enter the actual Earth radius at your latitude, it cannot take into account refraction, and it ONLY works at EXACTLY sea-level (cannot take into account your elevation bias) - so it gives very slightly less accurate answers.  But you can use it to get a quick estimate:

Estimate Height Obscured at h=3.5m, d=63.634km = ~254.6m

But, for example, at latitude 43.340372 where ground elevation is 73m MSL, the effective radius is ~6368.183 km and with 7% refraction, we actually should get closer to 236 meters:

√[(d - (√h √[h + 2 R]))² + R²] - R, d=63634, h=3.5, R=6368183*1.07 = 236 meters [Wolfram|Alpha]

So these things do matter somewhat, use ANY estimator with appropriate caution and awareness of your limitations to accurately measure things. It is FAR more difficult to take accurate measurements than most people realize. Even a few thousands of a degree in an angle, when compounded over many miles, adds up to a big error.