Social News in 1000 Steps – Step 13

This entry is part 13 of 14 in the series Social News

CSV files are great, but of course even a basic database such as SQLite has many added benefits. It’s efficient to start with the most appropriate setup, however even on a developer workstation we usually want to have a lightweight database with minimal data for practical reasons. I made a script (hopefully one off) to migrate CSV file content to the database:

import os
import core
import csv
db = core.connect()
tables = set(db.tables)
for f in os.listdir('.'):
    if f.endswith('.csv'):
        name = f.replace('.csv', '')
        if name not in tables:
            with open(f, encoding='cp1252') as csv_file:
                reader = csv.DictReader(csv_file)
                for row in reader:
                    db[name].insert({k: v for k, v in row.items()
                                     if k is not None and len(k) > 0})

The views from step 12 needed tweaks, for instance default sorting and HTML rendering:

class CseSearchAdmin(sqla.ModelView):
    def _html_formatter(view, context, model, name):
        return Markup('<a href={0}>{0}</a>'.format(
    def _snippet_formatter(view, context, model, name):
        return Markup(model.html_snippet)
    column_formatters = {'link': _html_formatter,
                         'html_snippet': _snippet_formatter}
    column_default_sort = ('search_date', True)
admin.add_view(CseSearchAdmin(CseSearch, db.session))
Series NavigationSocial News in 1000 Steps – Step 12Social News in 1000 Steps – Step 14
By the author of NumPy Beginner's Guide, NumPy Cookbook and Instant Pygame. If you enjoyed this post, please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.
This entry was posted in programming and tagged . Bookmark the permalink.