Comment by jonasdegendt

Comment by jonasdegendt 10 months ago

2 replies

> You often still need to negotiate individual agreements with major airlines in order to be able to publish and sell their fares.

Out of curiosity, what's stopping you from just scraping/crawling this on a daily basis? I'm at a scale-up that offers a BI solution for the hospitality sector, and we scrape well into the tens of millions of data points per day using web crawlers.

They obviously don't want you to, so it's an ongoing cat and mouse game of integrations breaking, but once we we're established and had a customer base, the ability to negotiate data API agreements actually opened up. Especially if you have data to offer in return.

tomhoward 10 months ago

We built the company from the start by scraping low cost airlines, and always did a lot of it.

It’s just a very big undertaking - building all the scrapers, maintaining them (every time any detail of an airline website changed), getting around anti-scraping measures, running servers, and even then your data is out of date as soon as a flight’s price or availability changes. And it’s an exponentially growing undertaking the more comprehensive you want your index to be - I.e, to include round trip and multi-stop tickets.