BSD Talk #42 has an interview Karl Lehenbauer about FlightAware.com. FlightAware tracks flight information, so far example here is their page on live flights to and from Sacramento Executive Airport. There is a lot of information that they are making available for free. Here is an outline of some of the more interesting bits that were mentioned by Karl during the interview.
All of FlightAware’s systems are 64-bit AMD based computers running FreeBSD 6.x, specifically the FreeBSD/amd64 port. They use PostgreSQL for the database back end. Slony 1 is being used to replicate data. Hard drives are in a RAID 1 (mirroring) configuration using 3ware controllers.
Now for some numbers:
- Receiving the data and processing it puts them about 6 minutes behind real time
- Generating one map can be done in about 160 milliseconds of CPU time
- Capable of generating several million maps a day
- About 1 TB of stored data
- Approximately 40 million position updates on air craft per day
PostgreSQL wasn’t able to keep up with the updates so they wrote a memory resident database service queries. I’m still not exactly clear on what the relationship is between PostgreSQL and their memory resident database, which uses about 1 GB of RAM.
Nice to see a company putting FreeBSD and PosgreSQL to good use. I’m curious about the 40 million inserts per day number. Bring on the math!
- 40,000,000 inserts per day
- 40,000,000 / 24 = 1,666,667 inserts per hour
- 1,666,6667 / 60 = 27,778 inserts per minute
- 27,227 / 60 = 463 inserts per second
So that boils down to about 463 inserts per second on average. I’d expect that their actual peak requirements are much higher than that (perhaps two or three times that number?). That is just data that they are receiving, that doesn’t include the queries being run against their system to power the website. This brings up another question, how much bandwidth do they have dedicated to receiving these updates? It is possible that each individual update is fairly small (lat, long, src, dest, flight id, airline, plane type, etc) so that might not be too bad. Even at 256 bytes per update, doing 40 million of those a day adds up very quickly.
Right now their website provides the following numbers:
Currently Tracking
Tracking 4,986 airborne aircraft (224 VFR) with 21,205,488 total flights in the database.
FlightAware has tracked 48,940 arrivals in the last 24 hours.
Cool stuff.