22.08.2024
7 min
1214
The article describes how mobile applications collect and transmit users’ geodata without their knowledge, and how anyone can check this for themselves. It explains how to set up an environment for traffic analysis using mitmproxy and mitmweb, how to intercept requests, filter them in a Python notebook, and find hidden transmissions of coordinates, IDFA, or other identifiers. The material shows examples of data visualization, collective collection of results via Google Sheets, and the possibilities of developing an open geoquery database.
The mass collection and transmission of location data by mobile applications has become one of the least controlled processes of the digital age. Every smartphone sends dozens of requests every second — not only to official application servers, but also to advertising, analytical or tracking platforms. This means that even a simple game or flashlight can silently transmit coordinates, unique device identifiers, information about the Wi-Fi network and nearby base stations. Such a stream of metadata allows you to create behavioral profiles of users, determine travel routes and predict human habits. The problem is that most users do not know where exactly this data goes, and do not have the opportunity to check it — neither through system settings, nor through official privacy policies, which often hide the real scale of information collection.
The purpose of the study is to provide users with practical tools for independent analysis of the outgoing traffic of their mobile applications. The project is not limited to just demonstrating the process — it shows how anyone can join the collective observation of how the advertising infrastructure works, what data is sent in real time, and who exactly is the recipient of these packets. The participants of the experiment receive step-by-step instructions for running their own local request analyzer, as well as access to open source code that helps filter out geolocation parameters, IDFA identifiers, or other sensitive metadata. In this way, a common knowledge base is formed — independent, transparent, and based on actual evidence, rather than assumptions about the behavior of applications.
The experiment uses the open source tool mitmproxy, which allows you to intercept and analyze traffic between a mobile device and a server. Its web interface mitmweb provides a visual visualization of each request: you can see the domain address, data type, HTTP headers, and the content of the transmitted information. Combined with a Python notebook, the system allows you to quickly filter huge amounts of data, highlighting only those requests that contain coordinates, tokens, or user IDs. All examples and ready-made code are published in an open GitHub repository, making the reproduction process simple even for beginners. Thus, anyone can not only test their own applications, but also contribute to the expansion of the shared library of intercepted data, increasing the transparency of the mobile ecosystem.
Preparing the system for traffic analysis is a key stage on which the quality of all subsequent observations depends. To get started, you need to have a computer on the same Wi-Fi network as your smartphone, since all Internet traffic will pass through it. The idea is simple but powerful: an intermediate control point is created – a local proxy server that accepts all requests from the phone, decrypts them and allows you to see the true content of the communication between the application and the server. This does not require special equipment: a modern laptop, a stable Wi-Fi connection and basic command line skills are enough.
Before starting, you should make sure that Python is installed and the system has up-to-date network libraries. Next, mitmproxy is installed – a flexible tool for intercepting traffic. For ease of visualization, the mitmweb web interface is used, which allows you to view all requests in real time, filter them, search for key parameters and save them in a convenient format for further processing.
The basic setup steps look like this:
Встановити mitmproxy через команду
pip install mitmproxy.
Запустити
mitmweb --listen-host 0.0.0.0 --listen-port 8080.
Переконатися, що комп’ютер і смартфон підключені до однієї мережі Wi-Fi.
Перевірити доступність панелі mitmweb через браузер (адреса: http://localhost:8080).
After completing these steps, a complete monitoring environment is created, where you can see all application traffic in real time. Every request sent by a mobile device is displayed in a web interface, opening the way to detailed analysis of the behavior of any application.
Once the proxy server is running on your computer, you need to configure your mobile device to route traffic through this server. To do this, activate a manual proxy in the Wi-Fi settings, specifying the computer’s IP address and port 8080. This allows you to route all requests from your phone through a local server, which now sees the full scope of communication between the application and the network. Next, you need to visit the mitm.it page — it opens automatically after connecting to the proxy. On this site, a special certificate is downloaded that you need to install in the system as trusted. This is a critical step: without a certificate, HTTPS connections remain encrypted, so even the most detailed analysis will not be able to show the content of the transmitted data.
After installing the certificate, the phone gets the opportunity to “see” decrypted traffic, and the user can track which servers receive requests, when and how often. Experience shows that even without interacting with the application, a huge amount of data is sent in the background: geolocation, system information, advertising identifiers. Using mitmweb allows you to not only see this stream, but also record it for further analysis, creating an archive that can be worked with in Python or shared with other researchers. Thus, the phone turns from an object of surveillance into an active tool for detecting hidden data transmission channels.
After completing the system setup, you can proceed to the main stage – recording network flows. The user opens the selected application, and all requests that he sends are automatically recorded in the mitmweb interface. At this point, the system saves not only the server addresses, but also the content of the requests, the time of sending, the protocol type, and even internal parameters transmitted within JSON or form data. This allows you to create a complete picture of the interaction between the application and the network – see how often coordinates are sent, whether sessions are saved after closing the application, and which third-party services are involved in data collection.
To keep the structure of the study organized, all flows are exported to files with unique names, for example, game_test.flow, maps.flow, or social.flow. Such files are subsequently imported into a Python notebook, where key parameters are automatically extracted: server address, packet length, request type, response code. This allows you to quickly compare different applications and determine those that use geodata the most. Gradually, your own evidence base is formed – live, structured and useful for further research into the transparency of mobile applications.
After all the collected traffic flows have been saved, the most interesting part begins – the filtering stage. This is where those requests that may contain the user’s personal data are highlighted from the general flow of technical information. The analysis is carried out using a Python notebook, which automatically scans .flow files, breaks them into individual requests and searches for certain keywords that most often indicate the transfer of geolocation or device identifiers. Such filtering helps not only to find specific risks, but also to see the systematic nature of data transfer – when dozens of different applications access the same domains.
The following parameters are usually checked to find anomalies:
latitude та longitude — координати користувача, які передаються у фоновому режимі;
IDFA або GAID — рекламні ідентифікатори, що дозволяють відстежувати дії між додатками;
location, geo, gps — змінні, які містять реальні дані про місцезнаходження;
advertising_id та session_id — маркери для відновлення профілю користувача при повторних запусках.
Once these elements are identified, queries are grouped by domain, and at this stage a map of relationships between applications, analytics platforms, and ad networks begins to emerge. This creates the basis for the next step — building a visual domain structure.
After filtering the data, it is important not only to see a list of domains, but also to understand how they interact with each other. For this, the PyVis library is used, which builds a graph model of all collected network activity. Each domain is displayed as a node, and its size corresponds to the number of requests received by it. Color is used to mark applications, thanks to which you can visually see which applications send data to the same servers. The graph allows you to instantly identify leaders — advertising networks, analytical services or cloud backends that collect the largest amount of information.
The following groups are especially common during analysis: Google Ads, Firebase, Appsflyer, Applovin, Unity Ads — they are the ones who most often receive user coordinates, session time and device data. The visual map helps to see not only the scale of the presence of these players, but also how small applications actually work as sensors, supplying data to large statistics collection systems. These are no longer just individual requests — this is an ecosystem that functions constantly.
To make the research as open as possible, a Google form was created through which anyone can add the results of their own analysis. The user uploads their .flow file, specifies the name of the application and describes what parameters were detected. All this data automatically enters a common table that does not contain personal information, only technical facts. This allows you to build a collective knowledge base – to see which domains are repeated most often, how they change over time and what new addresses appear after updates to popular applications.
This approach turns an ordinary experiment into a real network of public monitoring. Each contribution, even small, strengthens the overall picture, allowing you to identify patterns that were previously unnoticed by neither users nor even privacy researchers. This is no longer just a technical initiative – it is a step towards real transparency of the digital environment.
Despite the successful results, there are a number of difficulties when analyzing mobile traffic that prevent the process from being fully automated. Many applications use their own data transfer formats or encrypt packets using non-standard methods that are difficult to recognize even after installing a trust certificate. Requests are often encoded using protobuf or base64, hiding the structure of the fields, and the coordinates themselves can be transmitted in the form of mathematical expressions or JSON fragments. Because of this, part of the geodata remains unnoticed even after filtering, and for their recognition it is necessary to combine automatic algorithms with manual verification.
In addition, mobile operating systems are constantly updating security mechanisms, which makes it difficult to intercept requests. New versions of Android and iOS more actively block the installation of proxy certificates or restrict access to system network libraries. This creates technical barriers, but at the same time confirms the importance of such studies – only through practical experience can one understand what data is actually transmitted in real time.
Despite the limitations, the project continues to develop. The next steps include creating a system for automatic data export to CSV format so that anyone can quickly get a summary table with query parameters. It is also planned to develop modules for decoding protobuf responses, which will allow you to see the content even in cases where the data is transmitted in binary form. Another direction is to build a common database of typical mobile SDK queries, in which you can search for matches by domain or keyword.
In the future, the study may turn into a full-fledged monitoring platform with open APIs, where users from all over the world will exchange observation results. Such initiatives form a new culture of digital responsibility, when transparency becomes not an option, but a basic condition for the operation of any mobile service.
Today, a smartphone is not only a convenient tool, but also a constant source of data that is transmitted to dozens of servers without the user’s knowledge. The research shows that even basic knowledge and open tools allow you to see the hidden side of this system. Anyone can join the collaborative monitoring, install mitmproxy, intercept their own traffic and see exactly how their applications interact with the network.
This is not just an experiment, but a way to regain control over personal data. The more users participate in the collaborative analysis, the more transparent the digital ecosystem in which billions of devices live every day will become. And it is with such actions that real digital security begins – not declarative, but based on practice, community and knowledge.
