ConsoleFlare
  • Python
    • Python Installation
    • Pandas and SQL
  • Projects
    • Data Analytics Project
      • Courier Analytics Challenge
      • Solution
    • Skytrax Airline Review Analysis Pipeline
      • Setting up Azure SQL Database
      • SkyTrax Web Scraping
  • Reporting
    • Power BI
      • Installation
      • Data Sources
      • Important Links
  • PySpark & Databricks
    • Spark vs Hadoop
    • Cluster Computing
    • PySpark
    • Databricks Introduction
    • PySpark in Databricks
    • Reading Data with PySpark
    • PySpark Transformation Methods
    • Handling Duplicate Data
    • PySpark Action Methods
    • PySpark Native Functions
    • Partitioning
    • Bucketing
    • Partitioning vs Bucketing
  • Live Data Streaming
    • Spark Streaming
      • Installation Issues
      • Jupyter Notebook Setup
  • Data Pipeline
    • Azure Data Factory
  • Blockchain
    • Smart Contract Guide
      • Setting up a Node project
      • Developing smart contracts
  • Interview Questions
    • SQL Interview Questions
    • Power BI Interview Questions
  • T-SQL Exercises
    • Exercise 0
    • Exercise 1
    • Exercise 2
    • Exercise 3
  • CHEAT SHEET
    • Ultimate SQL Server Cheat Sheet
Powered by GitBook
On this page
  • Creating a Project and Installation :
  • Fetching Airline Names
  • Part 1 - Scraping Reviews From Airline
  • Part 2 - Scraping Reviews From A Airline
  • Part 3 - Scraping Reviews
  • Fetching Table Data
  • Creating Recorded Date Column
  • Rating Problem
  • SQL Injection
  • Basic Data Cleaning
  • Table Creation
  • Create Table Query
  • Part 2 - Table Creation
  • Data Insertion
  • Automation

Was this helpful?

  1. Projects
  2. Skytrax Airline Review Analysis Pipeline

SkyTrax Web Scraping

PreviousSetting up Azure SQL DatabaseNextPower BI

Last updated 1 year ago

Was this helpful?

Creating a Project and Installation :

Fetching Airline Names

Part 1 - Scraping Reviews From Airline

Part 2 - Scraping Reviews From A Airline

Part 3 - Scraping Reviews

Fetching Table Data

Creating Recorded Date Column

Rating Problem

SQL Injection

Basic Data Cleaning

Table Creation

Create Table Query

create_table_query = """
CREATE TABLE airline (
    id INT PRIMARY KEY IDENTITY(1,1),
    recorded_date DATE NULL,
    review_date TEXT NULL,
    airline VARCHAR(255) NULL,
    title TEXT NULL,
    review TEXT NULL,
    over_all_rating TEXT NULL,
    name VARCHAR(255) NULL,
    date VARCHAR(255) NULL,
    text TEXT NULL,
    type_of_traveller VARCHAR(255) NULL,
    seat_type VARCHAR(255) NULL,
    route VARCHAR(255) NULL,
    date_flown VARCHAR(255) NULL,
    seat_comfort VARCHAR(255) NULL,
    cabin_staff_service VARCHAR(255) NULL,
    food__beverages VARCHAR(255) NULL,
    ground_service VARCHAR(255) NULL,
    value_for_money VARCHAR(255) NULL,
    recommended VARCHAR(255) NULL,
    aircraft VARCHAR(255) NULL,
    inflight_entertainment VARCHAR(255) NULL,
    wifi__connectivity VARCHAR(255) NULL,
    airline_id VARCHAR(255) NULL,

);

"""

Part 2 - Table Creation

Data Insertion

Automation

https://youtu.be/2BN2NDQBgL4youtu.be