Replaying any minute of a day of ADS-B from a plain log file

Stargazer has saved 198,782 position fixes across the two antenna profiles on its project page, and it can play back any moment of any day. There is no database behind that. The storage is one text file for each day, and the replay is a binary search.

One line for each position

Each day gets a file named for its date, YYYY-MM-DD.jsonl, in a positions folder. Every decoded position is appended as one line holding eight values:

[ms, hex, lat, lon, alt, gs, track, local]

ms

is the time in milliseconds.

hex

is the aircraft's six character address.

lat

and lon are the position.

alt

is altitude.

gs

is ground speed, and track is the heading.

local

says whether this receiver heard the position itself or whether it was filled in from public data.

Every line is a small array and not an object with named keys, which keeps the file compact. A day holds a position every few seconds from every aircraft in range, so it is a lot of lines, and repeating eight key names on each of them would add up.

Why a plain file is enough

Replay needs one operation: given a time, start reading from there. A log written in time order makes that cheap. Because lines are appended as they arrive, the first number on each line only ever increases down the file.

Stargazer's replay reads by byte offset bisection. It looks at the byte in the middle of the file, finds the start of the next line, and reads that line's time. If the time is too early it moves into the upper half of the file, and if too late into the lower half. Each step halves the range, so even a file of hundreds of megabytes takes only a few dozen steps to arrive at the byte offset of the first line at or after the requested time. It then reads forward from there and plays the lines in order.

Scrubbing a slider to any moment of the day takes the same few reads, however long the day was. The file never has to be loaded whole.

There are three side benefits to keeping the log as text:

It is easy to inspect

Any line can be read in an editor, and grep finds an aircraft by its address.

It cannot be corrupted in the middle

Appends only ever add to the end, and a crash can at worst leave one partial last line.

It does not need migrating

The format is the whole of the schema.

Own data and filled data

The local value matters for how a position is shown. An aircraft Stargazer's own receiver hears is drawn in green. Where the receiver misses a position that the public network has, the gap is filled from public data and the aircraft is drawn in orange and pulses. The distinction is carried through to replay, so a recording of a day shows what the antenna heard and what it only learned about.

This is the same information the hit rate is built on. Each time an aircraft from the public network is beyond 50 km and above the radio horizon, either the receiver heard it too or it did not, and the log holds both.

The rest of what is kept

Alongside the daily files are three small ones in the data folder: statistics with the range and reach per degree and the totals of hits and misses against adsb.fi, a file with the callsign, registration and type of every aircraft seen, and a configuration file with the feed address and the receiver position. Aircraft cards show a photo from Planespotters with the photographer credited.

Stargazer needs Node 22 and no other dependencies. The Stargazer page has the measured antenna pattern and a radio horizon calculator, and the quadrant comparison shows what the data says about two antennas.

Written by Caisson AI. Questions or corrections go to hello@caissonai.co.