Project Description


17 Mar 2020
Description:
The Qlik bot app (described here) connected to a simple app.
The app collected data from wikipedia through a webconnector and displayed the information visually.
Due to the fact that Qlik doesnt allow one to publish and share an app, where Tableau does,
I thought I would try and recreate the application using Python and Tableau.
Below is the code for Python to extract the data from wikipedia.
The data simply get's extracted, transformed in Python and then the final dataframe gets
saved to a csv file, where Tableau connects to and then visualizes it.

The python portion below uses Beautifulsoup package to connect to the site and send through the details.
Once the correct table is found a loop is used to find each tablerow for data.

The data visualization app can be seen here:
Tableau Viz: GDP Visualization

Python Code:

# -*- coding: utf-8 -*-

import pandas as pd
import requests
from bs4 import BeautifulSoup

quote_page = 'https://en.wikipedia.org/wiki/List_of_countries_by_public_debt'
page = requests.get(quote_page)
soup = BeautifulSoup(page.text, 'html.parser')

fndclass="wikitable sortable nowrap mw-datatable"

table = soup.find('table', attrs={'class':fndclass})
table_rows = table.find_all('tr')

res = []
for tr in table_rows:
td = tr.find_all('td')
row = [tr.text.strip() for tr in td if tr.text.strip()]
if row:
res.append(row)

tmpdf = pd.DataFrame(res, columns=["Country", "Debt%_CIA", "Year", "Debt%_IMF", "NetDebt", "Date2", "Region"])
#print(tmpdf)

df = tmpdf[['Country','Debt%_IMF']]

#print(df)

file_name='gdpdebt.csv'
df.to_csv(file_name, encoding='utf-8')



Created by RH [2020]