Online Airline Management Simulation

My Account
E-mail:
Edit account
» Achievements
» Logout
Game Credits
Credit balance: 0 Cr
» Credit history
» Credits FAQ

### Author Topic: [-] Haversine Distance Formula  (Read 626 times)

#### LemonButt

• Members
• Posts: 2201
##### [-] Haversine Distance Formula
« on: February 14, 2014, 02:03:52 PM »
This is a HUGE bug.  Preface:  I'm a huge math nerd and have a degree in applied mathematics.

In trying to find a tech stop solution to fly from Paris to Tahiti, I used a great circle mapper: http://www.gcmap.com/mapui?P=ory-ppt&MS=wls&DU=mi

Flying from Paris to Tahiti is 8487nm according to AWS.  I threw in a tech stop at Ontario in Southern California as the great circle shows the line going straight through it.  Flying Paris to Ontario is 4891nm and Ontario to Tahiti is 3597nm.  If you add those together, the total flight with the tech stop is 8488nm--or only 1nm farther than flying direct.  I LOL'd as this is the most efficient tech stop I've ever seen.

The problem is that the great circle distance flying from Paris to Tahiti is not 8487nm--it is 9773nm.  I have developed websites and calculated a user's distance from a set location using a Haversine formula, so I immediately went over to a Haversine calculator and got the same result--9773nm.

A great circle, by definition, is the shortest path between two points on a sphere: noun a circle on the surface of a sphere that lies in a plane passing through the sphere's center. As it represents the shortest distance between any two points on a sphere, a great circle of the earth is the preferred route taken by a ship or aircraft.

So there is something wrong with the way AWS calculates distances and when you get out there really far, the error grows very large (1200nm in this instance).  I tried to come up with the 8487 value using other methods, such as the classic distance formula, but failed to get the same result.  The Haversine formula works because Earth is an oblique spheroid and while degrees of latitude are equally spaced, degrees of longitude fluctuate from being the widest at the equator to converging on zero at the poles.

For reference, below is my php Haversine equation for distance between two zip codes with corresponding lon/lat data stored in a database (\$row* = database row).  The return value is multiplied by 60 to go from seconds to minutes (1nm = 1 minute) and the 1.1515 value is to convert from nm to miles (take out the 1.1515 to leave it in nm obviously).

\$theta = \$rowzip1['lon'] - \$rowzip2['lon'];
return round(\$dist * 60 * 1.1515);

I have absolutely no idea how this change would be made to existing game worlds as it would absolutely destroy existing long haul route times/turnaround/slots/etc...meanwhile I'm going to hurry up and schedule my flight to Tahiti while it's short
« Last Edit: February 14, 2014, 02:11:41 PM by sami »

#### Sami

• Members
• Posts: 16973
##### Re: Haversine Distance Formula
« Reply #1 on: February 14, 2014, 02:11:14 PM »
Your cg mapper gives range of 9773 miles, not nautical miles.

in nm it is 8492 nm according to that site
http://www.gcmap.com/mapui?P=ory-ppt&MS=wls&DU=nm

not a bug.

#### LemonButt

• Members
• Posts: 2201
##### Re: Haversine Distance Formula
« Reply #2 on: February 14, 2014, 02:20:02 PM »
Your cg mapper gives range of 9773 miles, not nautical miles.

in nm it is 8492 nm according to that site
http://www.gcmap.com/mapui?P=ory-ppt&MS=wls&DU=nm

not a bug.

Oh I'm an idiot...sorry   You can delete this thread...

#### BD

• Former member
##### Re: Haversine Distance Formula
« Reply #3 on: February 14, 2014, 03:26:17 PM »
Oh I'm an idiot...sorry   You can delete this thread...
Don't feel bad...I have a few posts like this where I missed the not-so-obvious obvious.  I'm sure this club is fairly sizable, if people were honest about it.

I even started to write new posts and realized my error and, fortunately, never did hit the post button.

WARNING! This website is not compatible with the old version of Internet Explorer you are using.