Skip to content

feat(apps): plugin source in its own model, part 2 #9854

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 35 commits into from
May 19, 2022

Conversation

mariusandra
Copy link
Collaborator

@mariusandra mariusandra commented May 19, 2022

Problem

Here's part 2 of #9825 to be merged after #9853

Changes

  • Simplifies the loadPlugin code to treat each source (local file, zip via plugin.archive or PluginSourceFile) as exactly the same.
  • Allows querying for other local files, yet caches the source from pluginsourcefile for index.ts in the plugin if it exists, to avoid one extra query.

How did you test this code?

I'm not fully convinced in the caching system is needed or makes sense. Probably I should use a new field in this case, instead of regurgitating source.

Does this approach make sense?

Copy link
Contributor

@macobo macobo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's stagger this deploy from the main PR but code-wise lgtm! 💪

Base automatically changed from plugin-source-db-simple to master May 19, 2022 12:04
@mariusandra
Copy link
Collaborator Author

Thanks! Had to add a quick fix based on what I saw on app. after step 1 went live.

The migration in step 1 didn't create a plugin.json source file if the plugin didn't have a config_schema. Turns out also plugins with config_schema [] didn't get a plugin.json, as not [] == True.

This bug doesn't change how the plugins work or load, but just editing. Right now, for a few plugins, if you click "edit source", you see an error and must fill in { name: 'name' } to be allowed to save.

So I added a migration to create plugin.json plugin source files that don't exist.

@mariusandra mariusandra enabled auto-merge (squash) May 19, 2022 12:55
@mariusandra
Copy link
Collaborator Author

mariusandra commented May 19, 2022

Another failing test in the "postgres parity" folder that hopefully gets yeeted:

image

(plugin server kafka 1)

So that's another 30min to the review cycle.

@mariusandra mariusandra merged commit 8d68d45 into master May 19, 2022
@mariusandra mariusandra deleted the plugin-source-db-step2 branch May 19, 2022 13:51
fuziontech added a commit that referenced this pull request May 19, 2022
* master:
  refactor(ingestion): Make `KAFKA_ENABLED` true by default and set `KAFKA_HOSTS` default (#9844)
  feat(apps): transpile frontend.tsx (#9828)
  feat: show api call status when adding insights to dashboards (#9817)
  feat: track metrics on zapier hook firings (#9866)
  fix(onboarding): instrumentation (#9845)
  feat(whitelabel-shared-dashboard): Hide branding on shared dashboards paid (#9849)
  fix(apps): plugin source quickfix (#9862)
  refactor(plugin-server): Remove `ts-jest`, use `jest.mocked` (#9861)
  refactor(plugin-server): Trigger public jobs with graphile insted of celery queue (#9811)
  chore: upgrade pip tools (#9859)
  feat(apps): plugin source in its own model, part 2 (#9854)
  chore: Update sprint_planning_retro.md (#9791)
  feat(apps): plugin source in its own model, part 1 (#9853)
  feat(matrix): Add option to save `simulate_matrix` like `setup_dev` (#9836)
  fix(cohort): add mapping from event to person (#9841)
  feat(person-on-events): Enable CI to run using both old and new queries (#9814)
alexkim205 pushed a commit that referenced this pull request May 23, 2022
* create plugin source model

* edit source via plugin_source model

* deprecate "source"

* test plugin source updates

* add support for index.ts and index.js

* refactor plugin loading, support plugin sources from db

* fix source code in tests

* remove transpilation code

* reload plugin after saving

* store defaults in the db instead of persisting in form

* remove fields that don't exist

* remove unused fields

* commit suggestion

* rename to PluginSourceFile

* fix code feedback

* add comments

* make it safer to call

* convert to upsert

* convert to upsert

* comment on the null

* revert plugin server changes

* get source from new model behind the scenes

* simplify

* revert accidental changes

* revert accidental changes

* remove defaults

* sync the old source field

* use the source model in the plugin server

* cache the source from pluginsourcefile

* test that getPluginSource gets data from the database

* safer null check that doesn't override `{}` in the db with `null`

* fix error from migration 0233 ([] is falsy in python!)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants