SkyTrax Web Scraping

Creating a Project and Installation :

Fetching Airline Names

Part 1 - Scraping Reviews From Airline

Part 2 - Scraping Reviews From A Airline

Part 3 - Scraping Reviews

Fetching Table Data

Creating Recorded Date Column

Rating Problem

SQL Injection

Basic Data Cleaning

Table Creation

Create Table Query

create_table_query = """
CREATE TABLE airline (
    id INT PRIMARY KEY IDENTITY(1,1),
    recorded_date DATE NULL,
    review_date TEXT NULL,
    airline VARCHAR(255) NULL,
    title TEXT NULL,
    review TEXT NULL,
    over_all_rating TEXT NULL,
    name VARCHAR(255) NULL,
    date VARCHAR(255) NULL,
    text TEXT NULL,
    type_of_traveller VARCHAR(255) NULL,
    seat_type VARCHAR(255) NULL,
    route VARCHAR(255) NULL,
    date_flown VARCHAR(255) NULL,
    seat_comfort VARCHAR(255) NULL,
    cabin_staff_service VARCHAR(255) NULL,
    food__beverages VARCHAR(255) NULL,
    ground_service VARCHAR(255) NULL,
    value_for_money VARCHAR(255) NULL,
    recommended VARCHAR(255) NULL,
    aircraft VARCHAR(255) NULL,
    inflight_entertainment VARCHAR(255) NULL,
    wifi__connectivity VARCHAR(255) NULL,
    airline_id VARCHAR(255) NULL,

);

"""

Part 2 - Table Creation

Data Insertion

Automation