[ok] Cargo not shifting on one leg

[Lennertlenor]

Hi Sami,

in MT, it seems like the actual cargo demand is not growing on routes originating from FRA. On the return leg however, the demand has grown close to it's potential. For instance:
FRA-LAX: 35910/116200  LAX-FRA: 190970/227410
FRA-ATL: 23180/82710  ATL-FRA: 27440/35990
It looks like this for every route i checked, so definitely seems to be an issue with cargo shifting on the routes originating in FRA. Could you look into this for me please?

[Tha_Ape]

This is most likely a bug I reported around a year ago, and that has been reported several times since then:
https://www.airwaysim.com/forum/index.php/topic,77996.0.html

Answer from Sami:

Appears that it might not be possible to fully fix the issue, without making bigger changes to the code which in turn would create potentially unwanted large changes to the demand figures. So changes will take place later with other larger updates.

 

[JumboShrimp]

I see it repeatedly as well.  Cargo "settles" in one ratio going one way, different ratio going the other way.

By ratio I mean ratio of actual as percentage of Potential.

[JumboShrimp]

Here is an example from MT world, using SIN (as an island, not sharing) and FRA (sharing demand with many).

SIN-FRA captures nearly all of the SIN -> FRA demand, but only less than 1/2 of returning demand:
https://www.airwaysim.com/game/Routes/Planning/X/WSSS/EDDF/

Now, looking at routes that can bring the rest of the returning cargo:
SIN-CGN: Nope
https://www.airwaysim.com/game/Routes/Planning/WSSS/EDDK/?go=1

SIN-DUS Nope
https://www.airwaysim.com/game/Routes/Planning/WSSS/EDDL/?go=1

SIN-STR Nope
https://www.airwaysim.com/game/Routes/Planning/EDDS/WSSS/?go=1

SIN-MUC?  has a small overlap.  But similar story to SIN-FRA.  Lot's of independent SIN->MUC volume, only a tiny fraction of returning volume is allocated as actual demand
https://www.airwaysim.com/game/Routes/Planning/EDDM/WSSS/?go=1

So this may somehow be the case of demand somehow disappearing somewhere, to some kind of leak when there are multiple airports sharing the demand.  The demand from Germany in FRA vicinity going to SIN somehow disappears.

[JumboShrimp]

Just to update this topic:  The leak of cargo happens only when there are flights to multiple airports within catchment area.

Let's say there is one flight from an One "island" (O) airport to a region with Multiple airports (M), and this is the only flight.  Under this scenario, 100% of the cargo gets allocated as "actual" both ways.  Let's say
O -> M1 = 100%
M1 -> O = 100%

Now suppose I introduce another flight from this One "island" airport to region with Multiple airports, let's call them M1, M2, M1 + M2.
O -> M1 + O -> M2 = 100%
M1 -> O + M2 -> O < 100%
(even though M2 may have added more catchment area)

So the 2nd flight introduces the leak.  More flights, greater the leak.

[JumboShrimp]

Example with numbers

TPE -> SEA + PDX area

1. I fly to SEA, and returning potential demand is 25k of cargo, all of it becomes actual, so I am bringing back 25k of cargl
2. I add a flight to PDX that has a little bit of overlap and also some independent demand.
3. Actual demand from SEA drops from 25lk to 15k, loss of 10k of actual demand
4. Returning flight from PDX picks up only 6k of 15k potential.  So 4k of demand just disappeared by adding a flight
5. Of the 15k PDX potential, 10k may be independent (approx.).  So the total returning Potential demand may be 35k.  However, only 21k (15k + 6k) shows up as actual, meaning loss of 14k of actual demand.  So 40% of demand disappeared.

I can produce 10s and 10s of examples like that, when actual demand disappears after flights are added.  So the re-allocation of demand results in a leak, in demand disappearing.

[JumboShrimp]

Here is another, even more extreme example (from the game world I am not playing, but with a lot of routes to illustrate the point) GW4.

Frankfurt has a number of airports in its catchment area that have both cargo and passenger demand, so a lot of flights.  Remember with this bug, each additional flight destroys demand.  There are a lot of flights destroying demand.

The main connection from FRA to TPE, 4 dedicated cargo flights, 2 combi flights, 2 pax flights (with good sized cargo space) ends up with about 5% of cargo demand.

The other leg of this flight, not affected by this bug, gets 33% of the demand.  So instead of 33% of demand being available, only 5% is available. a reduction of 84% from 33 to 5 that is lost to this bug.  As I said, Frankfurt is one of the more extreme examples affected by this bug.

[JumboShrimp]

Another fresh example:

Demand to HAM: 53k

Opened route to BRE, result:
Demand to BRE: 0 -> 15k +15k
Demand to HAM: 53k -> 28k, -25k

So opening a flight to BRE destroyed 10k of demand, that just disappeared (15k - 25k = -10k)

Since BRE should have some demand independent of HAM, the demand destruction is in fact greater than 10k (we have no tools to find this level of detail to verify).

[JumboShrimp]

Could this bug get fixed in the foreseeable future?

If it is unfixable, given the data that the system has at its disposal, can the system just keep resetting the allocation, as in, re-do the allocation between the airport from scratch periodically, if it is the procedure of incremental re-allocation causes the leak and disappearing cargo demand?

[Lennertlenor]

yes please, this bug is costing me a lot of money from FRA.

[JumboShrimp]

Another example:

- Flying to PHX, returning demand is 36k of cargo
- Added a flight to TUS with tiny bit of overlap in the catchment area results:
Demand from TUS: 0k -> +4k
Demand from PHX: 36k -> 22k = -14k

So again, adding an extra flight destroyed 10k of demand (or more, since TUS has some of its own independent demand).

For the player unaware of this bug, the player doubles his expenses and lowers his revenue.

[Sami]

- Flying to PHX, returning demand is 36k of cargo
- Added a flight to TUS with tiny bit of overlap in the catchment area results:
Demand from TUS: 0k -> +4k
Demand from PHX: 36k -> 22k = -14k

Round 1: Looking at the code and database, all cargo types:

#1) No flights at all:
KPHX-RCTP: Actual 13210 / Potential 35990
KTUS-RCTP: Actual 4150 / Potential 9030

#2) Flight only to PHX:
KPHX-RCTP: Actual 35990 / Potential 35990
KTUS-RCTP: Actual 850 / Potential 9030 (actual is never zero, not sure where you're getting that....)

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 16670 / Potential 35990
KTUS-RCTP: Actual 3950 / Potential 9030

Does not seem logical but I am not yet sure "what it should be", before I eliminate some other things and do more testing.



Round 2: On both TPE and USA there were other airports involved, but no flights between them. Eliminated all other airports apart RCTP, KTUS, KPHX and after that:
- RCTP catches 4 areas with 100% of demand drawn from them.
- KPHX catches 8 areas, of which 7 are 100% and 1 area (#11447) is shared PHX-56 / TUS-44%.
- KTUS catches 4 areas, of which 3 are 100% to this airport.

After that the figures are:

#1) No flights at all:
KPHX-RCTP: Actual 34950 / Potential 35570
KTUS-RCTP: Actual 8160 / Potential 8920
  => These are logical (no other airports in area, and one overlapping area is shared).

#2) Flight only to PHX:
KPHX-RCTP: Actual 35570 / Potential 35570
KTUS-RCTP: Actual 830 / Potential 8920
  => This too works as intended.

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 22130 / Potential 35570
KTUS-RCTP: Actual 3950 / Potential 8920
  => Beep! Total demand 26080, while it should be ~44k



Round 3: If we eliminate totally the one area (#11447) that is overlapping between PHX and TUS, and no other airports are involved, the results are as expected:

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 34160 / Potential 34160
KTUS-RCTP: Actual 7690 / Potential 7690
  => OK! The airports don't overlap so demand should be full to potential at both.



Round 4: If we swap the situation from previous and we have only one departing area (that #11447) of which PHX and TUS share in 56/44% ratio (and the 3 arriving areas RCTP has), we get:

#1) No flights at all:
KPHX-RCTP: Actual 1220 / Potential 2170
KTUS-RCTP: Actual 940 / Potential 2110
  => OK. Total demand 2160.

#2) Flight only to PHX:
KPHX-RCTP: Actual 2170 / Potential 2170
KTUS-RCTP: Actual 220 / Potential 2110
  => This too works as intended.

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 1310 / Potential 2170
KTUS-RCTP: Actual 850 / Potential 2110
  => Total demand is 2160, works fine.

=> As a minor detail; potential for both airports should be the same? Maybe; have to check
     => The 60kg difference in potential is caused by different distance from PHX and TUS to destination. The distance from dep to arr airport is one factor, and airport-to-airport dist. is used instead of area-to-area distance due to speed (area-to-area calculation for this would be too slow). So this is correct but room for minor improvement here if performance permits in the future.



Round 5: Situation of areas returned to "round 2" where issue was present, with one overlapping area, but catchment of the one overlapping area is changed to 50%/50% between the airports for easier calculation. All possible internal scaling factors and all game world specific adjustment settings are removed and calculation should be "pure" from any adjustments. (figures are low, but that's not the point)

#1) No flights at all:
KPHX-RCTP: Actual 13330 / Potential 13600
KTUS-RCTP: Actual 3290 / Potential 3550
  => OK.

#2) Flight only to PHX:
KPHX-RCTP: Actual 13600 / Potential 13600
KTUS-RCTP: Actual 360 / Potential 3550
  => OK.

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 8400 / Potential 13600
KTUS-RCTP: Actual 1580 / Potential 3550
  => Still incorrect, total demand is 10k (should be ~14-15k)

 => Any of custom settings or factors are not the cause.



Round 6: Figuring out what the figures SHOULD be. Situation and settings as on round 5.

#A) Only KPHX-RCTP, all other data eliminated. PHX gets 100% of all the areas it overlaps and area #11447 removed.
KPHX-RCTP: Potential 13060

#B) Only KTUS-RCTP, all other data eliminated. TUS gets 100% of all the areas it overlaps and area #11447 removed.
KTUS-RCTP: Potential 3020

#C) Calculating only the area #11447 to RCTP:
#11447-RCTP: Potential 570

Totals:
KPHX-RCTP: Potential 13060 + Potential 570 = 13630
KTUS-RCTP: Potential 3020 + Potential 570 = 3590
  => Comparing to 'round 5' the figures for potential match 13600 vs 13630 and 3550 vs 3590. Minor differences (+/-50kg) due to the way some score values are counted when number of results increase.

If the area #11447 is split 50/50 between the two airports, then the Actual demand should be:
KPHX-RCTP: Potential 13060 + 50%-Potential 570/2 = Actual 13345 (rounded to 13350?) / Potential 13630
KTUS-RCTP: Potential 3020 + 50%-Potential 570/2 = Actual 3305 (rounded to 3300?) / Potential 3590
  => Total expected actual demand should be 16650.

In the case there are no flights, the calculation is OK. We had 13330 and 3290, and here we calculated manually 13350 and 3300.




(to be continued)

[JumboShrimp]

Round 1: Looking at the code and database, all cargo types:

#1) No flights at all:
KPHX-RCTP: Actual 13210 / Potential 35990
KTUS-RCTP: Actual 4150 / Potential 9030

#2) Flight only to PHX:
KPHX-RCTP: Actual 35990 / Potential 35990
KTUS-RCTP: Actual 850 / Potential 9030 (actual is never zero, not sure where you're getting that....)

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 16670 / Potential 35990
KTUS-RCTP: Actual 3950 / Potential 9030

#1. Makes sense, since there are minor airports involved where the demand might be.
#2  But once there is a flight from PHX, all the moveable demand shifts to PHX.
#3 Yup, that's the problem.  The figures in your test are close enough to MT figures

[JumboShrimp]

Round 2: On both TPE and USA there were other airports involved, but no flights between them. Eliminated all other airports apart RCTP, KTUS, KPHX:
- RCTP catches 4 areas with 100% of demand drawn from them.
- KPHX catches 8 areas, of which 7 are 100% and 1 area (#11447) is shared PHX-56 / TUS-44%.
- KTUS catches 4 areas, of which 3 are 100% to this airport.

After that the figures are:

#1) No flights at all:
KPHX-RCTP: Actual 34950 / Potential 35570
KTUS-RCTP: Actual 8160 / Potential 8920
  => These are logical (no other airports in area, and one overlapping area is shared).

#2) Flight only to PHX:
KPHX-RCTP: Actual 35570 / Potential 35570
KTUS-RCTP: Actual 830 / Potential 8920 (actual is never zero, not sure where you're getting that....)
  => This too works as intended.

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 22130 / Potential 35570
KTUS-RCTP: Actual 3950 / Potential 8920
  => Beep! Total demand 26080, while it should be ~44k

That's great info.  We don't see the squares so we can't tell how much of the squares are exclusive.

But the sum (union) of the catchment ares appears to have combined demand 43k, and after flights are flying to both airports, demand drops to 24k

[DanDan]

GW (HaF): between CGN & FRA and DOH: both routes are being flown since years, by two different companies as Heavy Cargo, so no "movements" can be expected anymore.

Potential Heavy Demand
CGN-DOH 3860kg
FRA-DOH 3430kg

Actual Heavy Demand
CGN-DOH 1910kg
FRA-DOH 1700kg

1) So in total, the two overlapping airports are only doing 1910+1700=3610kg of Heavy Cargo, less than CGN-DOH potential demand

2) The Frankfurt airport was the sole Heavy Cargo airport flying to DOH in the region for years. Actual Demand was same as Potential Demand at that time.

3) The overlapping area is only about 16-18% of the total area for CGN, and 11-13% of FRA. Meaning: theoretically at least half of the production of Heavy Cargo for FRA and CGN should come from that area... yeah, right... considering that CGN has areas like the Ruhrarea included in the coverage, the area of Koblenz and Limburg do not seem to be all that industrial to justify such a high cargo generation.

So not only the distribution between airports is bugged, but also the distribution of Cargo-Generation seems rather random.

[JumboShrimp]

GW (HaF): between CGN & FRA and DOH: both routes are being flown since years, by two different companies as Heavy Cargo, so no "movements" can be expected anymore.

Looks like Sami already reproduced this bug:
https://www.airwaysim.com/forum/index.php/topic,82543.msg488519.html#msg488519

[Talentz]

Ahhh... movement! Very good Info to digest.


Talentz

[Sami]

Round 7: Issue is found and a sort of temporary fix has been coded. Not perfect since the system can't go into such details at the end of the calculation cycle that it would be able to 100% accurately assign the partial demand from an area to two airports. But basic concept is anyway done. Figures and setup continued from 'round 5':

#1) No flights at all:
KPHX-RCTP: Actual 13330 / Potential 13600
KTUS-RCTP: Actual 3290 / Potential 3550
  => OK.

#2) Flight only to PHX:
KPHX-RCTP: Actual 13600 / Potential 13600
KTUS-RCTP: Actual 2670 / Potential 3550
  => OK. Note that KTUS-RCTP demand is the base demand without the one shared area (only areas TUS catches). But this is not fully accurate at this time.

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 12960 / Potential 13600
KTUS-RCTP: Actual 3000 / Potential 3550
  => Nearly OK. Should have figures of about 13300-13400 for PHX and 3200-3300 for TUS, but it's still a bit inaccurate and not sure if the accuracy can be improved due to the original design.

#4) Bonus: Flight only to TUS (no flights to PHX):
KPHX-RCTP: Actual 11910 / Potential 13600
KTUS-RCTP: Actual 3550 / Potential 3550
  => OK. Same case as #2 but reversed.

The logic works better now and main issue is faded out, but not perfect/100% correct yet.

Will be implemented tomorrow probably.



And compared to the original situation (round 1):

-- OLD --

#1) No flights at all:
KPHX-RCTP: Actual 13210 / Potential 35990
KTUS-RCTP: Actual 4150 / Potential 9030

#2) Flight only to PHX:
KPHX-RCTP: Actual 35990 / Potential 35990
KTUS-RCTP: Actual 850 / Potential 9030

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 16670 / Potential 35990
KTUS-RCTP: Actual 3950 / Potential 9030

-- NEW --
#1) No flights at all:
KPHX-RCTP: Actual 13210 / Potential 35990
KTUS-RCTP: Actual 4330 / Potential 9030

#2) Flight only to PHX:
KPHX-RCTP: Actual 35990 / Potential 35990
KTUS-RCTP: Actual 6770 / Potential 9030

#3) Flights as of current MT game world to both PHX and TUS:
KPHX-RCTP: Actual 34310 / Potential 35990
KTUS-RCTP: Actual 7620 / Potential 9030

[JumboShrimp]

Looks definitely much better.  I compared the sums of no flights to flights to both destinations, and in both cases, the MT and the test scenario, the 2 flight Actual demand is 3.8-3.9% below full potential demand.

It might be worthwhile to fine tune it, because who knows if this error has a way to scale when there are 10 overlapping airports on each end.

[schro]

If this is implemented tomorrow then those of us bugged by this issue should start buying more freighters.... now...