Skip to content

Add Google news and website crawler as Source #108

@lalitpagaria

Description

@lalitpagaria

Idea to add Google News as Source.
Google News provide RSS feed and query support hence it is easy to crawl it.
RSS link -

https://news.google.com/rss/search?q=[INPUT]

For now just add GoogleNews as source later we can add few other news sources.

Google RSS feed give title, headlight, date and url. So inorder to fetch full article we need to use another library like https://github.com/adbar/trafilatura

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions