Searching for tracks in the sky, using flight tracking for geolocation

28 October 2024 18 minutes Author: Cyber Witcher

The use of flight tracking opens up new opportunities for geolocation of objects, helping to identify locations by flight routes and aircraft tracks in the sky. Learn how OSINT analysts analyze this data to quickly and accurately identify locations based on real-world examples and case studies.

Let’s start

Using flight tracking for geolocation – Quiztime October 30, 2019

This post provides an explanation of the solution to the challenge from Tilman Wagner dated October 30th. It is worth noting that this was one of the most difficult tasks Quiztime managed to solve. However, Quiztime is not only about the answers to the quiz, but also about the new knowledge that can be gained in the process. A lot of new skills were learned during this assignment. Here is a photo of the task:

The only information Tillman provides is:

The plane in the sky is a Boeing 747 flying from London to Hong Kong
The photo was taken on October 30, 2019.
Tillman was at the hotel.

The challenge is to determine which hotel Tillman stayed at, but before that, there are a lot of other things to find out. There is also some information in the photo that will also help:

Today is a clear and sunny day.
The shadow indicates that the sun is probably behind Tillman.
The position of the plane in the sky. This will be decisive.

Attempts to solve this problem using reverse image search were unsuccessful. The building in the photo does not have enough distinctive features to be identified through a reverse search. Even after cropping and cropping the image, no matches were found.

Tillman’s Twitter account was also checked to find out where he might have been on October 30, but that too was unsuccessful. Tillman is known to travel a lot: he is in Germany, sometimes in Albania and even in Mongolia, so it is very difficult to predict his whereabouts at any given time. If he stayed in the same place every week, it might be tempting to just guess where he is rather than pinpoint the location. This once again confirms the rule of investigation: “Don’t assume anything.”

Planning approach – from general to specific

Where to start? One could try to identify the building, although that could take a very long time, or focus on the plane, which provided a bit more information. In solving such geolocation puzzles, it is advisable to first cover the general picture, and then gradually narrow the parameters. Theoretically, this shot could have been taken anywhere in the world, but the step-by-step detection of details allows narrowing down the search to a specific building. Knowing the date and route of the plane, one could guess which building Tillman was looking at.

Spoiler alert: he’s somewhere in this photo.

The building could be located anywhere in the world, which is of course a safe assumption to begin with. However, it is known that it is on the flight route between London and Hong Kong. This automatically excludes North and South America, Africa, and Oceania, narrowing the possible locations to Europe, Central Asia, or Asia. While this may seem like a minor detail, it actually narrows down the list of potential locations significantly.

In addition, it is known that the photo was taken during the day. So, once an aircraft is identified, its route can be traced and where exactly it was at a specific time to find out whether that place was illuminated by the sun or plunged into darkness during the flight. If a certain location was in the dark, it can be ruled out as a possible location. However, to begin with, it is important to identify the aircraft itself.

Aircraft identification

Tillman tells us that the flight was from London to Hong Kong on October 30 and that it was a Boeing 747-8F. To identify the aircraft, we need to know all the flights and aircraft types that traveled between London and Hong Kong on that day. The only problem is that London has five international airports (Heathrow, Gatwick, City, Stansted and Luton), while Hong Kong has only one. This means it will be easier to look at flights that have arrived in Hong Kong from any London airport, rather than trying to work the other way around with five possible departure points.

Historical flight data can be found in several sources. The most famous is FlightRadar, but there are also Airportia and FlightAware. The start of the analysis was delayed ten days after the image, so getting data from FlightRadar required a premium account. FlightAware also charges for access to historical data, while Airportia only provides information on scheduled passenger flights. Fortunately, Toby Blume had already determined that it must have been flight N617UP – a UPS cargo flight that departed London Stansted at 06:12 UTC on 30 October and was bound for Hong Kong. Thanks to this tip, it was possible to save time and possibly avoid additional costs.

This is the correct flight as determined by Toby.

It may not seem like it, but the identification of the plane means we have all the information we need to eventually determine which hotel (and room) Tillman stayed in and exactly when he took the photo (within minutes). It will be somewhere very close to this line:

With a little geometry, historical weather forecasts, and a sunset calculation, this flight path can be used to significantly narrow down the possible locations where Tillman was before he was eventually found.

Building a course using KML files

The task seems simple enough in theory, but how do you go from a flight route of over 6,000 miles to a specific hotel? Fortunately, both FlightRadar and FlightAware allow you to download flight tracks in KML format. KML is a file format that contains data about geographic points, such as GPS coordinates, altitude, etc. These files can be opened in Google Earth, which allows you to view and analyze the KML information. Downloading the KML file for the flight captured on the Tillman image allows you to see the exact coordinates, altitude and orientation of the aircraft at each leg of the route. By viewing this data in Google Earth, the location of the aircraft can be visualized and matched with a photograph to determine the location. Access to KML files requires a paid Flightradar account or a three-month free trial of FlightAware, which also provides access to KML.

When the KML file is opened in Google Earth, the entire flight path is overlaid on the map. Here is the plane taking off from Stansted:

Note that the KML overlay shows not only the position but also the height. The slider (top left of the image) can be moved to a specific time to see where the plane was. Here is the takeoff and landing in Hong Kong:

So how can we use this information to help find Tillman? First we need to understand a little geometry.

Geometry

Now that we have all the flight path data, we can combine it with sunset/sunrise information and a bit of geometry to calculate Tillman’s location. We know that the plane left London Stansted at 06:12 UTC. Entering this information into Suncalc shows that it was still dark in Stansted at the time:

Then the plane goes to Amsterdam. Maybe this city? If we repeat the same Suncalc method for Amsterdam, we see that it was also dark there when the plane flew by at 06:37 UTC, so it cannot be where Tillman was.

At this point, it’s important to remember that flight data is always in UTC, regardless of local time. This includes a time slider in Google Earth that can be used to show where the plane was. On the other hand, Suncalc always shows the local time relative to UTC, so Amsterdam is UTC + 1. It’s easy to get confused if you don’t remember this detail!

So Amsterdam can be ruled out, but is there a more efficient way to find out which city Tillman might have been in instead of checking every city along the flight path?

Geometry comes to the rescue here. After flying over Amsterdam, the plane changes course to 089 degrees, heading almost due east. If you look at the tracking log, you can see that the plane makes only minor course corrections through the Netherlands, Germany, Poland, Belarus and Russia. Only over the territory of Russia does the plane make a significant turn, changing its direction towards Kazakhstan, China and, ultimately, Hong Kong:

Excerpt from Flight Aware’s flight tracking log showing heading.

The plane crossed Kazakhstan at about 10:00 UTC (15:00 local time), but it was dark by the time it crossed China. That means Tillman couldn’t have been in China when he took the picture, so we can rule that out as a potential location. Also, since the plane’s course is usually due east, Tillman would have had to be facing roughly west for the plane to be heading toward him. The fact that the sun is behind him, combined with the angle of the shadow, suggests that the sun is approximately in the southeast, i.e. just above his left shoulder:

This indicates that Tillman took the picture quite early in the morning: evening time can be ruled out, as the sun would then be against him. So the possible locations remain the eastern part of the Netherlands, Germany, Poland, Belarus, or possibly western Russia, the only areas over which the plane flew on a clear day when the sun was in the southeast. While this still covers quite a few cities, there was a way to narrow down the search.

Given the size of the building in the image, which shows five stories and possibly one or two more, it can be assumed that the building is at least six or seven stories tall and is located next to the hotel where Tillman was staying. This indicates an urban location rather than a rural or suburban location. So it is necessary to find a city in one of the specified countries, located near the flight route and with the appropriate orientation in relation to the flight path.

So, next we need to do some geometry to help get the orientation right:

The real horizon is not as shaky as the blue line I’ve drawn here.

We know from flight data that this aircraft was heading almost due east for most of its journey across Europe. By superimposing the compass points on the original image, it can be concluded that Tillman’s position must be north of the flight path. If it were south of the eastbound plane’s flight path, the backtrack would look like this:

If it is difficult to imagine all this in your mind, do not worry – you can visually check the information using Google Earth. There is also the question of why an airplane appears to be flying vertically relative to an observer at ground level. We will consider this question later, as it will help to determine the location more precisely, but first we need to establish in which city Tilman is located.

The initial choice is Berlin

Given the angle at which the picture was taken, it can be assumed that Tillman is north of the flight path. This allows you to exclude a large number of other potential locations that are located in the wrong direction relative to the aircraft.

Це не може бути Ганновер чи Брауншвейг, бо вони на півдні.

It’s definitely not Poznań or Warsaw either. Volgograd could be a candidate, but it was probably too late for the sun to be in this position:

Anyway, the weather doesn’t match the sunny blue skies where Thielman was:

However, there is one city that stands out as a very strong candidate. It’s close to the flight path, and most of the city is on the right side as well:

Historical meteorological reports also confirm that there was not a single cloud in the sky over Berlin at that time:

The sun is also in the correct position (southeast) at the time the plane flew over Berlin:

So Berlin seems the most likely place where Tillman was. Starting with broad parameters—”somewhere between London and Hong Kong”—we gradually ruled out entire countries, checking where the plane had flown during the day. This narrowed the list down to a few possible locations. Then, given the orientation of the photo relative to the flight path, it was possible to discard cities such as Hanover or Warsaw. Berlin remains the only city that meets all the criteria: daylight, weather conditions, position of the sun, orientation and location relative to the flight path.

Searching for an exact location

If Tillman is indeed in Berlin, it is now worth using the available image and flight path data to locate him. Sure, you can just search for “Berlin hotels” and browse through them, or spend hours exploring the city on Google Maps, but these brute-force methods are tedious, slow, and inefficient. The KML file contains enough data to use Google Earth, which allows for very precise visual search parameters and to quickly locate Tillman, given that his location is within Berlin.

Before the final search, it is worth considering one more detail: why does the plane seem to be flying vertically upwards, if it is known that it flew overhead? This question may provide even more clues about Tillman’s exact whereabouts.

Tillman was not looking directly up; his gaze was slightly above the horizon, making the plane appear almost vertical. This is explained by the curvature of the Earth: when planes approach from over the horizon, their movement appears vertical. Their true horizontal position becomes apparent only when they fly overhead. And when the plane moves away, it seems that it descends to the horizon. This effect is quite common, and for the present case it is sufficient to know that the plane was approaching Tillman, but probably not yet directly over Berlin.

Photo reproduction

So Tillman was probably in Berlin, and the plane was approaching him. How to find an exact location without hours of searching on maps? With a KML file and the 3D Buildings feature in Google Earth, it is possible to recreate Tillman’s image almost perfectly, using the plane tracks as a reference point. First, you need to determine where exactly the plane was in relation to Berlin at the time of the shooting.

Since the planes are moving fast, the time slider shows that at 07:11 UTC (08:11 local Berlin time) the plane was a considerable distance from the city, roughly near Brandenburg:

After just five minutes, the plane almost disappeared from the field of view and already completely flew over Berlin:

Therefore, to reproduce the view from the ground, you need to capture the moment when the plane was heading over the horizon in the direction of Tillman’s location. Even if Tillman’s exact location is unknown, you can start visualizing it in Google Earth by trial and error. Keeping in mind that Tillman was probably north of the flight path, one can focus on the ground level near the Brandenburg Gate and face west, as Tillman did in the image:

Note that the plane’s path (gray line in the distance) is visible from ground level in Google Earth. The KML file also contains altitude data, so the aircraft will be displayed at the correct altitude relative to the ground. The time on this photo is 07:11 UTC. The plane was actually close to the city of Brandenburg at this point (about 60 km / 37 miles from the Brandenburg Gate), but you can see how Google Earth replicates the effect of the plane’s vertical position in the sky due to the curvature of the earth. . Here’s the same image just two minutes later at 07:13 UTC:

And again at 07:15 UTC. Please note that by this point the plane has completely flown overhead. This also means Tillman must have taken the photo before 07:15 UTC:

Thus, for the aircraft to appear to be flying almost vertically toward Tillman, the image must have been taken between 07:08 and 07:15 UTC. To reproduce the conditions as accurately as possible, setting the time slider to 07:13 UTC allowed for an optimal view. This allows you to see the plane on the horizon, moving towards Berlin, as in Tillman’s image, which makes the reproduction as close as possible to the original moment of shooting:

In the picture, Tillman is looking almost straight ahead, so in order to reproduce the same view, it is necessary for the plane to remain at the same level and at the same angle to the horizon as seen from ground level. (To “look” in Google Earth, you can hold down the Ctrl key and use the mouse to look around.)

These parameters are useful, but the location needs to be further narrowed down, as Tillman could not be near the Brandenburg Gate itself, but could not be very far away either. The most efficient method is to keep moving along the ground level in Google Earth until the angle of the flight path matches the angle of the plane’s wake in the original image. No complicated calculations were needed; just set up the view on the desktop and moved left and right (north and south) until the angle matched exactly.

Determining the angle greatly accelerated the process. To establish search boundaries, it was necessary to draw an imaginary square or “box” around the probable location of Tillman’s Hotel, and then check the area manually.

It was quite easy to establish the eastern and western borders. Since Tillman was near a tall building, it became clear that the suburbs with their low-rise buildings were not to be considered. Therefore, the suburbs of Berlin formed the eastern and western boundaries of the search area:

As it turned out, the east and west borders were a bit too big, but it’s better to play it safe and have wider borders than too narrow.

To determine the northern limit, I shifted “right” keeping the direction of view west until the angle of the plane’s track became too flat compared to the original view point. This showed that the position was too far north, helping to determine the northern limit of Tillman’s position:

The process was then repeated for the southern boundary, moving to the left. The decisive factor here was that the reverse angle of the plane’s track became too steep, which meant that the position was too far south. Thus, the southern limit of the search area for the place where Tillman could be was determined:

The southern border mark will be nearby.

At this point it became clear that I had gone too far south. The angle of the aircraft’s track has become almost vertical, which is different from the original image, confirming that the position is outside the Tillman zone. This also confirms the previously stated thesis that when moving south of the flight path, the angle of the wake changes so much that it appears completely inverted. This finally defined the southern limit of the search:

In this way, it was possible to be absolutely sure that if the plane was flying east, then Tillman should have been north of the flight path. So now Google Earth has a blue search area highlighted. Tillman should be in there somewhere, but you need to narrow it down even more before looking around at street level.

Areas without large buildings like the one in the image, such as parks or rural areas, have been highlighted in red. This narrowed Tillman’s probable location to the blue zone, excluding the red areas. Although the initial border may have been a little too large, this helped avoid the risk of missing something. Even with the reserve, the search area was only about 2 km wide, so it wouldn’t take too long to check the entire area.

Then it was necessary to turn back to face roughly west toward the approaching aircraft, as Tillman did:

The next step was a gradual advance through the city from east to west at the indicated height. This approach allowed us to scan the search area while staying within the outlined boundaries and the blue zone where a tall building like the one in the image could be located:

There is a familiar building on the left…

A perfect match! This means that the hotel Thielmann took his picture from was the H+ Hotel Mitte Berlin:

So, the accuracy in reproducing the picture of the reverse trace turned out to be quite high. In order for the aircraft to be in this position relative to Tillman, the picture must have been taken between approximately 07:13 and 07:14 UTC (08:13-08:14 local time). This time window allows you to almost pinpoint the moment of capture, confirming the calculations and orientation achieved during the simulation.