Flask RSS Feeder

Hi all,

Today i am uploading one of the new fun project i was working on.
As i am big fan of Manga, and its hard to keep track of them, when new ones are available.
So, i decided to create a RSS feeder.

What You'll learn,
- How to create a simple web app using FLASK.
- Using multi-threading for parallel processing.
- DB operations  on MY-SQL DB, it's what i have used.

Code Snippet :-

Below is the snippet of the project file structure.

HTML :-
1) layout.html :- This is the backbone of the application design.
2) home.html  :- Home page of the application.
3) article.html :- this one the list of the rss feeds.
4) includes folder consist app navigation bar and messages(to send flash notification).

Python :-
1) add_rss.py is the one where mostly everything is happening.


Importing functions from FLASK and SQLAlchemy.

#for flask and its functionality
from flask import Flask, render_template, flash, redirect, url_for, session, logging, request

#for mysql connection
from flask_sqlalchemy import SQLAlchemy

#for reading rss feeds
import feedparser

#for parallel processing

import threading

#DB configuration

app.config['SQLALCHEMY_DATABASE_URI'] = "mysql://<UserName>:<Password>@localhost/RSS_python"
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
app.config['SQLALCHEMY_ECHO'] = True

#do set your My-SQL username and password, also make sure to create the DB as given in DB.png.

db = SQLAlchemy(app)

url_data = []

#Function i am using to get the data from feed.

def get_source(feed):
    return {
        'title'     : feed['title'],
        'link'      : feed['link'],
        'published' : feed['published']

    }

#function to get feed from URL.

def get_feed(db_data):
    rss = db_data.url
    feed = feedparser.parse(rss)
    feed = feed["entries"][0]
    new_feed = get_source(feed)
    new_feed['name']    = db_data.name
    if db_data.title != new_feed.get('title'):
        db_data.title       = new_feed['title']
        db_data.link        = new_feed['link']
        db_data.published   = new_feed['published']
        new_feed['is_read']     = 1
        #update the row
        db.session.commit()
    url_data.append(new_feed)

    return

#DB model of my DB
class rss_model(db.Model):
    __tablename__ = "rss"
    id          = db.Column(db.Integer, primary_key=True)
    url         = db.Column(db.Unicode)
    name        = db.Column(db.Unicode)
    title       = db.Column(db.Unicode)
    link        = db.Column(db.Unicode)
    published   = db.Column(db.Unicode)

    def __init__(self,url,name, is_read=0):
      self.url          = url
      self.name         = name

      self.is_read      = is_read

#Route of home page. this is how to tell application, which page to load.
@app.route('/')
def show_all():
    return render_template('home.html')


#Function to get the feed, read the db and run the threads, so mostly everything is happening.
@app.route('/article')
def articles():
    global url_data
    url_data = []

          #Query to get all data from DB.
    data = rss_model.query.all()

    #empty list of same size as the number of rows fetched from DB.
    t = [None] * len(data)
    for i in range(len(data)):
                   # starting the same number of thread as the rows, so the processing will be faster.
        t[i] = threading.Thread(target=get_feed, args=(data[i],))
        print ("\n\n", "data:",data[i], "\n\n", file=sys.stdout)
        try:
            t[i] = threading.Thread(target=get_feed, args=(data[i],))
            t[i].start()
        except:
           print ("Error: unable to start thread", file=sys.stdout)

    for i in range(len(data)):
        #here we are waiting for all the threads to complete.
        t[i].join()
    return render_template('article.html', chapters = url_data)

#function to add new RSS feed.
@app.route('/add_rss', methods=['GET','POST'])
def add_rss():
    if request.method == 'POST':
        name = request.form['name']
        url = request.form['url']
        new_rss = rss_model(name = name, url = url)
        db.session.add(new_rss)
        db.session.commit()
        db.session.flush()
        msg="New Manga Added"
        return render_template('home.html',msg=msg)
    return render_template('add_rss.html')

#Setting debug to true so, the changes reflect ASAP in application, and manually setting the port to 5876.

if __name__ == '__main__':
    app.run(debug=True, port=5876)

#end of the rss_app.py


rss_python.sh

Just a script to start the application. i am using this in my bashrc to start it with alias.
ps_out=`ps -ef | grep $1 | grep -v 'grep' | grep -v $0`
result=$(echo $ps_out | grep "$1")
if [ "$result" != "" ];then
    echo "Server already Running"
else
    ~/<path to virtualenv python>/python3 ~/<path to file>/rss_app.py &>/dev/null &
fi


FYI :- you can clone the repo from 
https://github.com/Shapboyz13/rssFeeder

Hope you'll enjoy it,

Comments, review, questions and suggestion are always welcome, 

you can also connect me here. shashank17nov@gmail.com

And at last, as always thanks to all the stack overflow sources for the help.