feedi is a web feed reader with a minimal interface akin to a Mastodon or Twitter feed.
Features:
- Easy local and self-hosted environment setup.
- Mastodon home and notification streams support.
- Mobile-friendly UI.
- Installable as a Progressive Web App.
- Custom parsers for Reddit, Github and Goodreads.
- Hackable RSS parsers and ad hoc scrapers.
- Smart feed sorting options (highlight infrequent sources, auto mark as read).
- Local article reading and preview using Mozilla's reader mode.
- Send to Kindle device support.
For background about the development of this project, see this blog post.
feedi requires Python >= 3.9. If you don't have it installed already consider using pyenv or asdf.
To install feedi on a local environment:
git clone https://github.com/facundoolano/feedi.git
cd feedi
make
Then, to run the app:
make run
The application will be available at http://localhost:9988/.
Alternatively, see the instructions for running with Docker or in a non-local setup.
By default, your feed will be empty. You can load content in a number of ways:
- Manually adding a feed by clicking the
+ Add Feedbutton or navigating to/feeds/new. - Discovering a website RSS feed by pasting or dragging its url to the search input.
- Loading a set of default websites included in this repo by running:
make feed-load feed-sync. - Importing a collection of feeds from an OPML or CSV file.
- Logging into a mastodon account.
When you first add a feed, the app will fetch its most recent articles, then it will check periodically for new content (every 30 minutes by default).
- Sources can be put into folders, accesible on the left sidebar (desktop) or on the navbar menu (mobile).
- The feed behavior can be tweaked with the controls on the right sidebar (desktop) or on the navbar menu:
- Entries are auto-marked as viewed as the user scrolls down the feed. By default, already seen entries are skipped next time the app is open.
- The default sorting puts least frequent sources at the top. Alternatively, there's are are strict chronological sorting.
There are different ways to interact with a feed entry:
- If you click on the article title the original website will be open on a new browser tab.
- In some cases, as with link agreggators like reddit, hacker news or lobste.rs, there will be a separate link for the article discussion.
- If you click on the content or press Enter when focusing on the entry, the article content will be fetch and displayed on the local reader. This will be a stripped-down version of the article (removing some site links, ads and paywalls) powered by the mozilla/readability library. Note that for this to work you need node >= 20 installed when setting up the project.
- The reader can also be used to preview arbitrary articles by dragging their url to the searchbox.
Entries get deleted automatically some days after their publication (defaulting to 7). Pinned and favorited entries are never deleted. Additionally, a minimum of entries (defaulting to 5) is kept for all sources, regardless of their publication date.
make feed-load will load feeds from a local feeds.csv file. A sample file is included in the repo
in case you want to see some content right away.
There's also a make feed-load-opml to import a list of RSS feeds from a feeds.opml file in the OPML format.
There are analogous make feed-dump and make feed-dump-opml targets to export feed data from the app.
The Mastodon integration allows ingesting both the user home feed and the notification inbox as feeds.
feedi first needs to be authorized to access the Mastodon account by navigating to
Manage feeds > Mastodon login or to the url /auth/mastodon. After filling the instance name
and grating access, feedi will redirect to the add feed form, where either mastodon or mastodon notifications feeds (or both) can be added.
You can ingest the notifications from GitHub into feedi. To do so, navigate to your home feed at https://github.com/, open the page HTML source and search for an atom feed link. It should look something like:
<link rel="alternate" type="application/atom+xml" title="ATOM" href="/facundoolano.private.atom?token=<TOKEN>" />Copy the href url and use it to add a new RSS feed in feedi.
You can ingest the notifications from Goodreads.com into feedi. To do so, navigate to your home feed at https://www.goodreads.com/, open the page HTML source and search for an atom feed link. It should look something like:
<link href='https://www.goodreads.com/home/index_rss/<ID>?key=<KEY>' rel='alternate' title='Goodreads' type='application/atom+xml'>Copy the href url and use it to add a new RSS feed in feedi.
Reddit exposes public pages as RSS feeds by appending .rss to the URL, for example https://www.reddit.com/r/selfhosted.rss or www.reddit.com/user/someuser.rss.
Additionally, authenticated users have access to RSS feeds for private account pages: the front page, saved links, message inbox, etc. Links can be found here. feedi uses special purpose feed parsers both for reddit listing messages and links, and for the user inbox.
The app allows to register a kindle email to send the cleaned up article HTML to it, yielding better results than the default Amazon Send to Kindle Chrome extension. This requires setting up SMTP credentials for sending email.
Steps to make it work with a basic gmail account:
- Register an google email account.
- Turn on 2-factor authentication for the account
- navigate to the app passwords settings and generate a password for the app (requires 2FA activated to work).
- Add SMTP settings to the app config:
FEEDI_EMAIL = "YOUR.EMAIL@gmail.com"
FEEDI_EMAIL_PASSWORD = "GENERATED APP PASSWORD"
FEEDI_EMAIL_SERVER = "smtp.gmail.com"
FEEDI_EMAIL_PORT = 587- navigate to
/auth/kindleor type 'kindle' in the searchbox, and enter the @kindle.com email for your device. - Go to your Amazon Personal Document Settings and add your feedi email address to the approved document email list.
After this setup the "Send to Kindle" command will be available when browsing articles.
The app works by periodically fetching items from different feed sources (RSS/Atom, Mastodon toots and notifications, custom scrapers) and adjusting them to an Entry db model which more or less matches what we expect to display in the front end.
Most RSS feeds should be processed correctly by the default parser, but sometimes it's desirable to add customizations that cleanup or extend the data for a better look and feel. This can be done by subclassing feedi.parsers.rss.BaseParser. The is_compatible static method determines whether a given feed should be parsed with that specific class; the parse_* methods overrides the default logic for each field expected in the front end.
As an example, this parser for the lobste.rs link aggregator is adjusted to inline a summary of external link submissions and distinguish between the source article url and the lobste.rs discussion url:
class LobstersParser(BaseParser):
def is_compatible(_feed_url, feed_data):
return 'lobste.rs' in feed_data['feed'].get('link', '')
def parse_content_short(self, entry):
# A 'Comments' link is only present on external link submissions
if 'Comments' in entry['summary']:
url = self.parse_content_url(entry)
return (self.fetch_meta(url, 'og:description') or
self.fetch_meta(url, 'description'))
return entry['summary']
def parse_entry_url(self, entry):
# return the discussion url, which is different from entry['link']
# for external links
if 'Comments' in entry['summary']:
soup = BeautifulSoup(entry['summary'], 'lxml')
return soup.find("a", string="Comments")['href']
return entry['link']You can see several custom RSS parsers in this module.
Other than RSS and Mastodon feeds, the app can ingest arbitrary sources with custom parsers. This is useful for scraping websites that don't provide feeds or consuming JSON APIs directly.
To add a custom parser, subclass feedi.parsers.custom.CustomParser. The is_compatible method determines wheter a given url should be parsed with that parser. The fetch method does the actual fetching and parsing of entries. See the feedi.parsers.custom module for some examples.
Once the parser is implemented, it will be used when a new feed of type "Custom" is added in the webapp with the expected url.
| shortcut | when | action |
|---|---|---|
| Cmd+k | focus search input | |
| Enter | search focused | submit first suggestion |
| Escape | search or suggestion focused | hide suggestions |
| Down Arrow, Ctrl+n | search or suggestion focused | next suggestion |
| Up Arrow, Ctrl+n | suggestion focused | previous suggestion |
| Enter | entry focused | open entry content |
| Cmd+Enter, Cmd+Left Click | entry focused | open entry content on new tab |
| Cmd+Shift+Enter, Cmd+Shift+Left Click | entry focused | open entry discussion on new window |
| Down Arrow, Tab | entry focused | focus next entry |
| Up Arrow, Shift+Tab | entry focused | focus previous entry |
| p | entry focused | pin entry |
| f | entry focused | favorite entry |
| Escape | viewing entry content | go back |
The default app configuration assumes a single-user unauthenticated setup, but authentication can be enabled in case security is necessary, for example to deploy the app on the internet or to support multiple accounts.
To enable user authentication:
- Remove the
DEFAULT_AUTH_USERsetting from the configuration. - If you already have a DB created, reset it with
make db-reset. Or, alternatively, remove the default user withmake user-del EMAIL=admin@admin.com. Note that this will also remove feeds and entries associated to it in the DB. - You can create new users by running
make user-add EMAIL=some@email.address. The command will prompt for a password.
Note that there's no open user registration functionality exposed to the front end, but it should be straightforward to add it if you need it. Check the auth module and the flask-login documentation for details.
Get the image from github packages:
docker pull ghcr.io/facundoolano/feedi:latest
Create a volume for persisting the db data:
docker volume create feedidb
Load the default feeds into the default admin user:
docker run -v feedidb:/app/instance ghcr.io/facundoolano/feedi flask --app feedi/app.py feed load feeds.csv admin@admin.com
docker run -v feedidb:/app/instance ghcr.io/facundoolano/feedi flask --app feedi/app.py feed sync
Run in development mode:
docker run -p 9988:9988 -v feedidb:/app/instance ghcr.io/facundoolano/feedi
To run in production mode, a config file with at least a secret key is expected:
echo "SECRET_KEY = '$(python -c 'import secrets; print(secrets.token_hex())')'" >> production.py
docker run -p 9988:9988 -e FLASK_ENV=production -v feedidb:/app/instance -v $(pwd)/production.py:/app/feedi/config/production.py ghcr.io/facundoolano/feedi
To enable authentication, add DEFAULT_AUTH_USER=None to that production config file.
You can refer to the Flask documentation for a instructions on how to deploy feedi to a non-local environment. The setup script included in the repository shows an example setup for a Debian server. You can run it remotely with ssh like make prod-install SSH=user@server.