Skip to content

Problem: We can scale and improve harvesting performance by designing a new harvesting system #7492

@thegostev

Description

@thegostev

Problem statement

The harvesting mechanism was not designed to work at the scale that some major sites like data.gov operate. If CKAN harvesting is up to their expectations, everybody won as then the whole community will benefit from the update.

Hypothesis

  1. We can create a new harvesting standard that addresses data.gov needs.
  2. We can enable new harvesting standard in CKAN.
  3. We can make it possible to harvest geospatial data.
  4. It's possible to engage GSA Taskforce into this job.
  5. It's possible to engage data.gov team to participate in delivery.

Discovery work
Log here everything you've found during the discovery

Validation
Why the solution is trustworthy? What makes it strong?

Questions to consider:

Is this change going to break current installations?

Can we provide a backwards compatibility?

How easy is gonna be for current implementations to migrate to this new release?

Do current versions of CKAN have the adequate resources?

Does/support to migrate to this new version?

Are we going to change the database schema?

Are we going to change the API?

Are we going to deprecate Interfaces?

Image

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

Problem Research

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions