COVID19 Spread Visualization and Comparison using Flask and Plotly

Question

---

---

GOEDUHUB's Online Courses @ Udemy

---

---

Summer Training-Internship program-2021

Analysis, Visualization and Comparison of COVID19

Image Credits : Scientific Animations under a CC BY-SA 4.0 license

Coronavirus is a family of viruses that can cause illness, which can vary from common cold and cough to sometimes more severe disease. Middle East Respiratory Syndrome (MERS-CoV) and Severe Acute Respiratory Syndrome (SARS-CoV) were such severe cases with the world already has faced. SARS-CoV-2 (n-coronavirus) is the new virus of the coronavirus family, which first discovered in 2019, which has not been identified in humans before. It is a contiguous virus which started from Wuhan in December 2019. Which later declared as Pandemic by WHO due to high rate spreads throughout the world. Currently (on the date 10 June 2020), this leads to a total of 411K+ Deaths across the globe, including 180K+ deaths alone in Europe. Pandemic is spreading all over the world; it becomes more important to understand about this spread. This NoteBook is an effort to analyze the cumulative data of confirmed, deaths, and recovered cases over time. In this notebook, the main focus is to analyze the spread trend of this virus all over the world. 

This curve shows why we need to follow social distancing and other precautions to flatten this curve.

Till today when I'm writing this post there more than five hundred thousand (500K) only due to this cursed decease in this passed 6 months.

Purpose of the Project

In this project we will Analyze, Visualize the spread of Noval Corona Virus all over the world. And also we will make comparisons of spread in different countries. We will be using Plotly, Folium, Pandas, Matplotlib etc. to perform analysis and visualization of data. Then we will show it on a web-page using Flask Framework.

Requirements and Prerequisites

For writing our codes I've used Pycharm

For this project you must have prior knowledge of basic 

• HTML 
• CSS 
• JavaScript 
• Bootstrap. 

Here mainly we will be using Python and Flask. So prior knowledge of Flask would be beneficial. At last using Plotly's JavaScript library would be useful.

Answer 1

File Structure

In this project we will be using basic flask project file structure.

|-- Root

     |-- assets

          |-- 'datasets and images as required'

     |-- templates

          |-- 'html base templates and pages'

     |-- source codes

Here is my project directory structure

Resources:

JHU is currently using a GitHub repository to store its data instead of a web API. JHU already updates their data-set daily on github. visit

Other data resources can be found at this link: https://github.com/CSSEGISandData/COVID-19

If you are interested in knowing country level data, please refer to the following Kaggle datasets:
India - https://www.kaggle.com/sudalairajkumar/covid19-in-india
South Korea - https://www.kaggle.com/kimjihoo/coronavirusdataset
Italy - https://www.kaggle.com/sudalairajkumar/covid19-in-italy
Brazil - https://www.kaggle.com/unanimad/corona-virus-brazil
USA - https://www.kaggle.com/sudalairajkumar/covid19-in-usa
Switzerland - https://www.kaggle.com/daenuprobst/covid19-cases-switzerland
Indonesia - https://www.kaggle.com/ardisragen/indonesia-coronavirus-cases

Implementation:

We will divide the whole project into steps:

1. Creation of a basic Flask App Structure.
2. Creation of a python file named models.py which will contain all the models or function which will be used to generate the HTML visualizations for Flask app.
3. Creation of base.html template for defining page layout of the web app.
4. Integration of all aforementioned steps.

Step 1: Basic Flask App Structure

In this step we will create a file names app.py which will be root source file of the project. In this file initially we will make three routes named '/' or 'index.html' which will lead to Home page of the web app. Second route will '/worldmeter' which will lead to the World Map page of the web app. And the last route is '/about' which will lead to the about page of the web page in which we will write something about our self.

from flask import Flask, render_template

app = Flask(__name__, static_url_path='/assets',

            static_folder='./covidtrack/assets',

            template_folder='./templates')

@app.route('/')

@app.route('/index.html')

def root():

    return render_template('index.html', title='Home')

@app.route('/worldometer')

def worldMap():

    return render_template('worldMap.html', title = 'World Map')

@app.route('/about')

@app.route('/about_me.html')

def about_me():

    return render_template('about_me.html', title = 'About Me')

if __name__ == '__main__':

    app.run(debug=True)

But still our web app will not work because we have not created templates till now. So now we will create a new directory name templates which will hold our templates. for this we will create four html files:

• index.html
• worldMap.html
• about,html
• base.html

Note: Do not get confused for base.html  we will talk about this one int third step. 

Now in all html files except base.html we will write some initial html contents for testing. Such as:

<!DOCTYPE html>

<html>

    <head>

        <title>Home</title>

    </head>

    <body>

        <h1>this is a test</h1>

    </body>

</html>

At last we will create one more directory name assets which will hold our resources such as data sets and image files.

On executing app.py we should see a screen like this in the web browser in:

Note: Pycharm generate a localhost URL which should be same as http://127.0.0.1:5000/index.html.

---

Summer Training-Internship program-2021

Answer 2

Step 2: Creating Models for Data Analysis and Visualization 

Now we will create a new file models.py which we will hold all the models used by app.py. We will need to import several libraries for different purposes such as:

import json

import pandas as pd

import plotly.graph_objects as go

import plotly

import warnings

warnings.filterwarnings('ignore')

Now we define different colors for different conditions:

# color pallette

cnf, dth, rec, act = '#ffbf47', '#6f0000', '#0f9b0f', '#1cb5e0'

First model is plot_daywise_bar which will generate Bar plots for our web app: 

def plot_daywise_bar(url):

    df = pd.read_csv(url)

    date = pd.to_datetime(df['Date'])

    fig = [go.Bar(x=date, y=df['Confirmed'], name='Confirmed'),

           go.Bar(x=date, y=df['New cases'], name='New Cases'),

           go.Bar(x=date, y=df['Deaths'], name='Deaths'),

           go.Bar(x=date, y=df['Active'], name='Active'),

           go.Bar(x=date, y=df['Recovered'], name='Recovered'),

           go.Bar(x=date, y=df['New recovered'], name='New recovered'),

           go.Figure().update_layout(barmode='stack', title='Day Wise COVID Spread')]

    graphJSON = json.dumps(fig, cls=plotly.utils.PlotlyJSONEncoder)

    return graphJSON

Note: In each model we are passing url to the data set.

Another model is plot_daywise_line to generate line plots:

def plot_daywise_line(url):

    df = pd.read_csv(url)

    date = pd.to_datetime(df['Date'])

    fig = [go.Line(x=date, y=df['New cases'], mode='lines', name='New cases'),

           go.Line(x=date, y=df['Confirmed'], mode='lines', name='Confirmed'),

           go.Line(x=date, y=df['Deaths'], name='Deaths'),

           go.Line(x=date, y=df['Active'], name='Active'),

           go.Line(x=date, y=df['Recovered'], name='Recovered'),

           go.Line(x=date, y=df['New recovered'], name='New recovered'),

           go.Figure().update_layout(title='Day Wise COVID Spread', xaxis_title="", yaxis_title="")]

    graphJSON = json.dumps(fig, cls=plotly.utils.PlotlyJSONEncoder)

    return graphJSON 

Another model is top_twenty_hbar for plotting horizontal bar, by using this model we will compare the effects of COVID19 in top twenty Corona virus effected countries:

def top_twenty_hbar(url, col, n, hover_data=[]):

    df = pd.read_csv(url)

    df = df.sort_values(col).tail(n)

    fig = [go.Bar(x=df[col],

                  y=df['Country/Region'],

                  hovertext=hover_data,

                  orientation='h',

                  marker_color=df[col] )]

    graphJson = json.dumps(fig, cls=plotly.utils.PlotlyJSONEncoder)

    return graphJson

In plot_hbar_wm model we will see the effects of COVID19 on every One Million population:

def plot_hbar_wm(url, col, n, min_pop=1000000, sort='descending'):

    df = pd.read_csv(url)

    df = df[df['Population'] > min_pop]

    df = df.sort_values(col, ascending=True).tail(n)

    fig = [go.Bar(

        x=df[col],

        y=df['Country/Region'],

        marker_color=df[col],

        orientation='h',

        text=col + '(Only Countries with > 1M Population)' )]

    graphJson = json.dumps(fig, cls=plotly.utils.PlotlyJSONEncoder)

    return graphJson

And case_over_time_map model will be used to create a Choropleth map, which will show the spread of COVID19 on map using color gradients:

def case_over_time_map(url, col, title):

    df = pd.read_csv(url)

    fig = [go.Choropleth(locations=df['Country/Region'],

                         z=df[col].astype(float),

                         locationmode='country names',

                         colorbar_title=title,

                         text=df['Country/Region'],

                         reversescale=True,

                         marker_line_width=0.5).update(autocolorscale=True,

                                                       hovertemplate=df['Deaths'])]

    graphJSON = json.dumps(fig, cls=plotly.utils.PlotlyJSONEncoder)

    return graphJSON

Like wise we can add many more models to generate different kinds of very effective and interesting maps.

---

Summer Training-Internship program-2021

Answer 3

Step 3: Creation of base.html template

Now we can talk about base.html file. This file will be used to generate the main layout of the webspp. As we know that for using Python inside any html file we need to use Jinja2, hence here also we will use Jinja2 to taking data from python files and then showing on the page. Here we will take some help from bootstrap. You can copy any of the main page template from boot strap. But here I will create my own page structure, it is not very complex and easy to understand. You can visit this link for understanding more about bootstrap components.

<!DOCTYPE html>

<html lang="en">

<head>

    <meta charset="utf-8"/>

    <title>{% block title %} base {% endblock %}</title>

    <meta name="description" content=""/>

    <meta name="author" content="Daniyal Khan"/>

    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css">

    <script src="https://cdn.plot.ly/plotly-latest.min.js"></script>

    <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.6/d3.min.js"></script>

    <script src="https://code.jquery.com/jquery-3.3.1.slim.min.js"></script>

    <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js"></script>

    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js"></script>

</head>

<body>

<div>

    <div id="content">

        <!--Navbar -->

        <nav class="navbar navbar-expand-lg navbar-dark" style="background-color:#237a57">

            <a class="navbar-brand" href="#">CovidTrack</a>

            <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarNav"

                    aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation">

                <span class="navbar-toggler-icon"></span>

            </button>

            <div class="collapse navbar-collapse" id="navbarNav">

                <ul class="navbar-nav">

                    <li class="nav-item active">

                        <a class="nav-link" href={{ url_for('root') }}>Home <span class="sr-only">(current)</span></a>

                    </li>

                    <li class="nav-item">

                        <a class="nav-link" href="{{ url_for('worldMap') }}">Worldometer</a>

                    </li>

                    <li class="nav-item">

                        <a class="nav-link" href="{{ url_for('about_me') }}">About Me</a>

                    </li>

                </ul>

            </div>

        </nav>

        <!--/.Navbar -->

        {% block content %}

        {% endblock %}

    </div>

    <footer class="footer fixed-bottom navbar-expand-sm">

        <div class="container text-muted " style="text-align: center;">

            &copy; Copyright & powered by <a href="https://www.daniyalk.com"><b>Daniyal K.</b></a>

        </div>

    </footer>

</div>

</body>

<script>

        {{ script_txt }}

</script>

</html>

Now we will make changes in index.html file. Now it should look like:

{% extends 'base.html' %}

{% block title %}

        Home | TRACK COVID19

{% endblock title%}

{% block content %}

<!--Main Body-->

<div class="container">

       <h1>This is a test</h1>

</div>

<!--End of Main Body-->

{% endblock %}

worldMap.html should look like:

{% extends 'base.html' %}

{% block title %}

Worldometer | TRACK COVID19

{% endblock title%}

{% block content %}

<div class="container">

    <h1>This is a test</h1>

</div>

{% endblock %}

At last about.html should look like:

{% extends 'base.html' %}

{% block title %}

About Me | TRACK COVID19

{% endblock title%}

{% block content %}

<div class="container">

    <h1>About Me</h1>

</div>

<div>

</div>

{% endblock %}

After making these changes in all four files you should see some thing link this:

Summer Training-Internship program-2021

Answer 4

Step 4: Integration

This last but most important step in this project. Till now we have not seen any visualization on our app. 

Now we will import all the models from models.py into app.py like this:

from flask import Flask, render_template, request, session, redirect, url_for, flash

from models import plot_daywise_bar, ratio_dayWise, \

    plot_daywise_line, case_over_time_map, top_twenty_hbar, \

    plot_hbar_wm, plot_stacked, plot_line, scatter

Now we will define the paths of the resources which we have stored in assets folder into variables as string.

country_wise_latest = 'covidtrack/assets/country_wise_latest.csv'

covid_19_clean_complete = 'covidtrack/assets/covid_19_clean_complete.csv'

day_wise = 'covidtrack/assets/day_wise.csv'

full_grouped = 'covidtrack/assets/full_grouped.csv'

usa_county_wise = 'covidtrack/assets/usa_county_wise.csv'

worldometer_data = 'covidtrack/assets/worldometer_data.csv'

Now we will modify the root route to call data from models.py and we will store this data into variable. And at last we will send these data to index.html so that it could display it.

@app.route('/')

@app.route('/index.html')

def root():

    bar = plot_daywise_bar(day_wise)

    line = plot_daywise_line(day_wise)

    topTwentyCases = top_twenty_hbar(country_wise_latest, 'Confirmed', 15)

    topTwentyDeaths = top_twenty_hbar(country_wise_latest, 'Deaths', 15)

    topTwentyRecovered = top_twenty_hbar(country_wise_latest, 'Recovered', 15)

    topTwentyActive = top_twenty_hbar(country_wise_latest, 'Active', 15)

    topTwentyNewCases = top_twenty_hbar(country_wise_latest, 'New cases', 15)

    topTwentyNewDeaths = top_twenty_hbar(country_wise_latest, 'New deaths', 15)

    topTwentyDeaths_100_cases = top_twenty_hbar(country_wise_latest, 'Deaths / 100 Cases', 15)

    topTwentyNewRecovered = top_twenty_hbar(country_wise_latest, 'New recovered', 15)

    topTwentyRecovered_100_cases = top_twenty_hbar(country_wise_latest, 'Recovered / 100 Cases', 15)

    topTwenty1_week_change = top_twenty_hbar(country_wise_latest, '1 week change', 15)

    topTwenty1_week_increase = top_twenty_hbar(country_wise_latest, '1 week % increase', 15)

    perMDeaths = plot_hbar_wm(worldometer_data, 'Deaths/1M pop', 15)

    totalTests = plot_hbar_wm(worldometer_data, 'TotalTests', 15)

    perMTests = plot_hbar_wm(worldometer_data, 'Tests/1M pop', 15)

    return render_template('index.html', plot=bar, line=line, topTwentyCases=topTwentyCases, topTwentyDeaths=topTwentyDeaths,

                           topTwentyRecovered=topTwentyRecovered, topTwentyActive=topTwentyActive, topTwentyNewCases=topTwentyNewCases,

                           topTwentyNewDeaths=topTwentyNewDeaths, topTwentyDeaths_100_cases=topTwentyDeaths_100_cases,

                           topTwentyNewRecovered=topTwentyNewRecovered, topTwentyRecovered_100_cases=topTwentyRecovered_100_cases,

                           topTwenty1_week_change=topTwenty1_week_change, topTwenty1_week_increase=topTwenty1_week_increase,

                           perMDeaths=perMDeaths, perMTests=perMTests, totalTests=totalTests

                           )

Likewise we will change the worldMap function.

@app.route('/worldometer')

def worldMap():

    casesMap = case_over_time_map(country_wise_latest, 'Confirmed', 'Confirmed Cases')

    deathsMap = case_over_time_map(country_wise_latest, 'Deaths', 'Deaths')

    deaths_100_Map = case_over_time_map(country_wise_latest, 'Deaths / 100 Cases', 'Deaths / 100 Cases')

    return render_template('worldMap.html',

                           casesMap=casesMap, deathsMap=deathsMap, deaths_100_Map=deaths_100_Map)

To display the data we need to use some JavaScript. We will use Plotly.js for performing this thing. We will use the cloudflare cdn for linking ploly.js

src="https://cdn.plot.ly/plotly-latest.min.js"

We do not need to change anything for it, we already have included it into base.html, so it will work in all of our templets.

Now we will use following lines of code to display any graph:

<div class='chart' id='variable-name'>

       <script>

                var graphs = {{ 'variable-name' |safe }};

                var layout = {width: 900, height: 400, margin: {t: 50, b: 50, l:100, r:20}, coloraxis: {showscale: false, colorscale: "Viridis"}};

                Plotly.plot('variable-name', graphs, layout);

      </script>

</div>

For making different graphs we will change the variable-name and the graph will be displayed on the browser. For if we want to plot topTwentyCases we will make a <div> like this:

<div class="chart" id="topTwentyCases">

                  <script>

                                var graphs = {{ topTwentyCases|safe }};

                                var layout = {width: 900, height: 400, margin: {t: 50, b: 50, l:100, r:20}, coloraxis: {showscale: false, colorscale: "Viridis"}};

                                Plotly.plot('topTwentyCases', graphs, layout);

                 </script>

</div>

Now we will make changes in whole index.html like wise. I can not include this code because of it size. Hence you can find this code at https://github.com/Daniyalatjit/CoronaWebApp/blob/master/templates/index.html

After making these changes now we will show our map on the worldMap.html. After making changes in worldMap.html, it will look like as shown in next answer.

---

Summer Training-Internship program-2021

Answer 5

{% extends 'base.html' %}

{% block title %}

Worldometer | TRACK COVID19

{% endblock title%}

{% block content %}

<ul class="nav nav-pills mb-3 justify-content-center" id="pills-tab" role="tablist">

    <li class="nav-item">

        <a class="nav-link active btn btn-light" id="pills-casesMap-tab" data-toggle="pill" href="#pills-casesMap" role="tab"

           aria-controls="pills-casesMap" aria-selected="true">Confirmed Cases</a>

    </li>

    <li class="nav-item">

        <a class="nav-link btn btn-light" id="pills-deathsMap-tab" data-toggle="pill" href="#pills-deathsMap" role="tab"

           aria-controls="pills-deathsMap" aria-selected="false">Deaths</a>

    </li>

    <li class="nav-item">

        <a class="nav-link btn btn-light" id="pills-deaths_100_Map-tab" data-toggle="pill" href="#pills-deaths_100_Map" role="tab"

           aria-controls="pills-deaths_100_Map" aria-selected="false">Deaths / 100 Cases</a>

    </li>

</ul>

<div class="container justify-content-center">

    <div class="tab-content" id="pills-tabContent">

        <div class="tab-pane fade show active" id="pills-casesMap" role="tabpanel" aria-labelledby="pills-casesMap-tab">

            <div class="chart" id="casesMap">

                <script>

                                var graphs = {{ casesMap|safe }};

                                var layout = {mapbox: {style: "dark", center: {lon: -110, lat: 50}, zoom: 0.8},

                                width: 1200, height: 600, margin: {t: 50, b: 50}, coloraxis: {showscale: false, colorscale: "Viridis"}};

                                var config = {mapboxAccessToken: ""};

                                Plotly.newPlot('casesMap', graphs, layout, config);

                </script>

            </div>

        </div>

        <div class="tab-pane fade" id="pills-deathsMap" role="tabpanel" aria-labelledby="pills-deathsMap-tab">

            <div class="chart" id="deathsMap">

                <script>

                                var graphs = {{ deathsMap|safe }};

                                var layout = {mapbox: {style: "dark", center: {lon: -110, lat: 50}, zoom: 0.8},

                                width: 1200, height: 600, margin: {t: 50, b: 50}, coloraxis: {showscale: false, colorscale: "Viridis"}};

                                var config = {mapboxAccessToken: ""};

                                Plotly.newPlot('deathsMap', graphs, layout, config);

                </script>

            </div>

        </div>

        <div class="tab-pane fade" id="pills-deaths_100_Map" role="tabpanel" aria-labelledby="pills-deaths_100_Map-tab">

            <div class="chart" id="deaths_100_Map">

                <script>

                                var graphs = {{ deaths_100_Map|safe }};

                                var layout = {mapbox: {style: "dark", center: {lon: -110, lat: 50}, zoom: 0.8},

                                width: 1200, height: 600, margin: {t: 50, b: 50}, coloraxis: {showscale: false, colorscale: "Viridis"}};

                                var config = {mapboxAccessToken: ""};

                                Plotly.newPlot('deaths_100_Map', graphs, layout, config);

                </script>

            </div>

        </div>

    </div>

</div>

{% endblock %}

And Finally we will change our About me page accordingly. You can write your own bio. Here I'm writing mine:

{% extends 'base.html' %}

{% block title %}

About Me | TRACK COVID19

{% endblock title%}

{% block content %}

<div class="container">

    <h1 class="mt-5">About Me</h1>

    <div class="card mb-3 mt-5">

        <div class="row no-gutters">

            <div class="col-md-4">

                <img src="{{url_for('static', filename='me.jpg')}}" class="bd-placeholder-img" width="350px"

                     height="100%" alt="Daniyal">

            </div>

            <div class="col-md-8">

                <div class="card-body"></div>

                    <blockquote class="blockquote text-right card-text mr-3">

                        <h5 class="card-title display-4">Daniyal K.</h5>

                        <p class="mb-0">I am an enthusiastic student of Computer Science who has been striving to learn

                        and work

                        on several Artificial Intelligence Projects and researches. I have been doing several

                        projects and researches in the domain of Web Development,

                        Computer Networking, IoT, Algorithms and Machine Learning, Java Software development and

                        Maintenance.</p><br/>

                        <footer class="blockquote-footer card-text"><cite title="Source Title"><a href="http://www.daniyalk.com">Daniyal

                        K.</a></cite><br/> Student and Researcher </footer>

                    </blockquote>

                </div>

            </div>

        </div>

    </div>

    <div>

    </div>

    {% endblock %}

Note: You can clone this project from git using following url https://github.com/Daniyalatjit/CoronaWebApp.git

Answer 6

Final Review

Now if we execute our project then we should see some thing like this:

Index page

World Map Page

About Me Page

Final Words:

Thank You for following a long way till now. I appreciate your enthusiasm to learn this.

I hope this project would help everyone to take one more step in Machine Learning and Web Development.