-
Notifications
You must be signed in to change notification settings - Fork 10.4k
feat: implement calendar cache denormalized SQL system #22708
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Add CalendarSubscription and CalendarEvent database tables with proper indexes - Implement repository pattern with dependency injection following selectedSlots pattern - Add calendar-cache-sql-read and calendar-cache-sql-write feature flags for manual team rollout - Create Google Calendar webhook endpoint based on existing webhook implementation - Add cron job for subscription management and watch setup - Include comprehensive unit and integration tests (12 tests passing) - Follow existing Cal.com patterns for error handling, logging, and DI - Support etag-based webhook reliability and race condition handling - Enable gradual rollout via TeamFeature table entries This implements the complete calendar cache SQL system as described in the PRD, replacing the JSON-based CalendarCache with proper SQL tables for better performance and queryability. Co-Authored-By: zomars@cal.com <zomars@me.com>
Co-Authored-By: zomars@cal.com <zomars@me.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
WalkthroughThis PR adds a SQL-backed calendar cache system with Prisma models (CalendarSubscription, CalendarEvent), repositories, and services for availability, webhook processing, subscription management, and cleanup. It introduces API routes for webhook handling and cron jobs, Vercel cron schedules, and a cron tester update. Feature flags (read/write/cleanup) are added and wired into handlers. A getCalendarService selector enables conditional use of the SQL cache. UI and types are updated to surface event source and SQL cache metadata. Legacy JSONB calendar-cache code is marked deprecated. Multiple unit tests and migrations are included. Possibly related PRs
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. 📜 Recent review detailsConfiguration used: .coderabbit.yaml 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 180000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
Status, Documentation and Community
|
…ionship - Replace inefficient per-user feature flag checking with database-level team filtering - Update CalendarSubscription to have one-to-one relationship with SelectedCalendar - Fix webhook to use CalendarCacheService instead of old fetchAvailabilityAndSetCache - Add new getNextBatchForSqlCache method for efficient team-based filtering - Update repository interfaces and implementations for new schema - Update tests to reflect new schema relationship Co-Authored-By: zomars@cal.com <zomars@me.com>
The latest updates on your projects. Learn more about Vercel for GitHub. |
- Add calendar-cache-sql-read feature for reading from SQL cache - Add calendar-cache-sql-write feature for writing to SQL cache - Both features disabled by default for manual team enablement - Use OPERATIONAL type following existing pattern Co-Authored-By: zomars@cal.com <zomars@me.com>
Thanks for the feedback! I'm a bit confused about the query issue you mentioned. My current query has:
Could you clarify what specific part of the query logic is wrong? The condition seems to match what you're asking for - finding calendars that don't have entries in the new CalendarSubscriptions table yet. Is there additional logic I should be including from the existing method (like error handling, expiration checks, etc.)? |
.../20250723210325_update_calendar_subscription_to_selected_calendar_relationship/migration.sql
Outdated
Show resolved
Hide resolved
DevinAI, you're right I got confused reviewing. |
…lendar interface - Remove destructive migration file that would cause data loss - Update webhook to use fetchAvailabilityAndSetCache instead of invalid listEvents method - Fix TypeScript errors in webhook implementation Co-Authored-By: zomars@cal.com <zomars@me.com>
- Add type assertion for watchCalendar response to handle Calendar interface returning unknown - Ensure cron job properly calls Google Calendar watch API and saves webhook fields - Addresses final GitHub feedback about cron job not populating googleChannelId, googleChannelToken, googleChannelExpiration Co-Authored-By: zomars@cal.com <zomars@me.com>
…acheService - Remove destructive migration file that would cause data loss - Update webhook to use CalendarCacheService.processWebhookEvents instead of fetchAvailabilityAndSetCache - Fix CalendarSubscriptionRepository interface to include required selectedCalendar properties - All type checks and tests passing Co-Authored-By: zomars@cal.com <zomars@me.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🧹 Nitpick comments (4)
packages/prisma/migrations/20250819013055_add_calendar_subscription_system/migration.sql (4)
54-55
: Consider query patterns: add standalone index on backoffUntilThe composite index (channelExpiration, syncErrors, backoffUntil) won’t help queries that filter primarily on backoffUntil (the leftmost column rule applies). If your scheduler picks “all subs where backoffUntil <= now” regardless of channelExpiration, you’ll get a seq scan.
Add:
+-- CreateIndex +CREATE INDEX "CalendarSubscription_backoffUntil_idx" ON "CalendarSubscription"("backoffUntil");
13-15
: Add CHECK constraints to keep error counters saneDefensive constraints prevent accidental negatives and make invariants explicit.
You can add:
+ALTER TABLE "CalendarSubscription" + ADD CONSTRAINT "CalendarSubscription_syncErrors_nonneg_chk" CHECK ("syncErrors" >= 0), + ADD CONSTRAINT "CalendarSubscription_maxSyncErrors_nonneg_chk" CHECK ("maxSyncErrors" >= 0);
69-70
: Reassess global start/end/status index; consider a partial indexA global index on (start, end, status) may be low-selectivity and large. If your common scans target confirmed events, a partial index can reduce bloat and improve performance.
Keep existing indexes for now, but consider adding:
+-- CreateIndex (Postgres partial index example) +CREATE INDEX "CalendarEvent_start_end_confirmed_idx" + ON "CalendarEvent"("start", "end") + WHERE "status" = 'confirmed';You can then measure usage and decide whether to drop "CalendarEvent_start_end_status_idx".
40-41
: Optional: add recurrence-friendly indexIf you query instances by series, an index helps. You already have (calendarSubscriptionId, iCalUID). If queries use recurringEventId or pair iCalUID+originalStartTime, add one accordingly.
For series lookups by recurringEventId:
+-- CreateIndex +CREATE INDEX "CalendarEvent_calendarSubscriptionId_recurringEventId_idx" + ON "CalendarEvent"("calendarSubscriptionId", "recurringEventId");Also applies to: 62-67
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
packages/prisma/migrations/20250819013055_add_calendar_subscription_system/migration.sql
(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-07-18T08:47:01.264Z
Learnt from: vijayraghav-io
PR: calcom/cal.com#21072
File: packages/prisma/schema.prisma:891-891
Timestamp: 2025-07-18T08:47:01.264Z
Learning: The Outlook Calendar integration in Cal.com intentionally reuses subscription IDs across multiple event types for efficiency. The `upsertSelectedCalendarsForEventTypeIds` method creates separate SelectedCalendar records for each eventTypeId, all sharing the same outlookSubscriptionId. This subscription sharing pattern means that unique constraints like `@unique([outlookSubscriptionId, eventTypeId])` should not be applied as they would prevent this intended functionality.
Applied to files:
packages/prisma/migrations/20250819013055_add_calendar_subscription_system/migration.sql
📚 Learning: 2025-07-18T08:47:01.264Z
Learnt from: vijayraghav-io
PR: calcom/cal.com#21072
File: packages/prisma/schema.prisma:891-891
Timestamp: 2025-07-18T08:47:01.264Z
Learning: In Cal.com's calendar integration, both Google Calendar and Outlook Calendar are designed to allow multiple eventTypeIds to share the same subscription ID (googleChannelId or outlookSubscriptionId). This is an intentional design pattern to reuse existing subscriptions for efficiency rather than creating separate subscriptions for each event type. Therefore, unique constraints like `@unique([outlookSubscriptionId, eventTypeId])` should not be added as they would break this subscription sharing functionality.
Applied to files:
packages/prisma/migrations/20250819013055_add_calendar_subscription_system/migration.sql
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Atoms E2E Tests
🔇 Additional comments (4)
packages/prisma/migrations/20250819013055_add_calendar_subscription_system/migration.sql (4)
51-52
: Unique per selectedCalendarId is correct and aligns with shared-subscription patternOne subscription row per SelectedCalendar prevents duplicates while still allowing multiple SelectedCalendars to point to the same channelId, which preserves the “shared subscription ID across event types” design (per our prior learning). LGTM.
33-37
: Confirm intent to use timestamp without time zone for event start/endYou’re storing start/end as TIMESTAMP(3) (no tz) and carrying timeZone separately. That’s fine if all reads/writes normalize accordingly. If you plan cross-timezone queries at SQL level, timestamptz may be safer.
Please confirm the service layer consistently normalizes to UTC and uses timeZone to reconstruct local times where needed. If not, consider switching to timestamptz for start/end.
75-76
: Event uniqueness per subscription looks rightUniqueness on (calendarSubscriptionId, externalEventId) prevents duplicates while allowing the same externalEventId across different subscriptions. LGTM.
16-17
: Ensure Prisma schemas use @updatedat on updatedAt fieldsThe migration adds an updatedAt column as
NOT NULL
without a default, so inserts will fail unless Prisma auto-manages this field. Both schema files are missing the @updatedat attribute on these models:• packages/platform/examples/base/prisma/schema.prisma
• packages/prisma/schema.prismaPlease update each model’s updatedAt field from:
- updatedAt DateTime + updatedAt DateTime @updatedAtThis ensures Prisma sets a default timestamp on create and automatically updates it on save.
packages/prisma/migrations/20250819013055_add_calendar_subscription_system/migration.sql
Show resolved
Hide resolved
packages/prisma/migrations/20250819013055_add_calendar_subscription_system/migration.sql
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🧹 Nitpick comments (5)
packages/features/calendar-cache-sql/CalendarCacheService.ts (5)
92-100
: Early return when no valid selected calendar IDsIf
selectedCalendars
is non-empty but none have anid
,selectedCalendarIds
becomes empty and the repo call might inadvertently query broadly. Short-circuit to avoid unnecessary/expensive queries.const selectedCalendarIds = CalendarCachePresenter.presentCalendarIds(selectedCalendars); + if (selectedCalendarIds.length === 0) { + log.debug("No valid selectedCalendar IDs, returning empty results"); + return []; + } + const subscriptions = await this.subscriptionRepo.findBySelectedCalendarIds(selectedCalendarIds);
83-109
: Validate and reuse parsed dates to avoid invalid ranges and double parsingCurrently
new Date()
is called twice and invalid inputs aren’t handled. Validate once, bail early on invalid ranges, and reuse the parsed dates.private async loadEventsForSelectedCalendars( dateFrom: string, dateTo: string, selectedCalendars: IntegrationCalendar[] ): Promise<CalendarEventForAvailability[]> { if (selectedCalendars.length === 0) { return []; } + const from = new Date(dateFrom); + const to = new Date(dateTo); + if (isNaN(from.getTime()) || isNaN(to.getTime())) { + log.warn( + "Invalid date range provided to loadEventsForSelectedCalendars", + safeStringify({ dateFrom, dateTo }) + ); + return []; + } + const selectedCalendarIds = CalendarCachePresenter.presentCalendarIds(selectedCalendars); const subscriptions = await this.subscriptionRepo.findBySelectedCalendarIds(selectedCalendarIds); const subscriptionIds = CalendarCachePresenter.presentSubscriptionIds(subscriptions); if (subscriptionIds.length === 0) { log.debug("No subscriptions found, returning empty results"); return []; } const allEvents = await this.eventRepo.getEventsForAvailabilityBatch( subscriptionIds, - new Date(dateFrom), - new Date(dateTo) + from, + to ); return allEvents; }
118-119
: Trim debug logs to avoid logging potentially large objectsLogging full
selectedCalendars
can be verbose. Log counts and identifiers only.- log.debug("Getting availability from cache", safeStringify({ dateFrom, dateTo, selectedCalendars })); + log.debug( + "Getting availability from cache", + safeStringify({ dateFrom, dateTo, selectedCount: selectedCalendars.length }) + );- log.debug( - "Getting availability with timezones from cache", - safeStringify({ dateFrom, dateTo, selectedCalendars }) - ); + log.debug( + "Getting availability with timezones from cache", + safeStringify({ dateFrom, dateTo, selectedCount: selectedCalendars.length }) + );Also applies to: 136-139
43-50
: Type guard for calendar IDsIf
IntegrationCalendar["id"]
can bestring | undefined | null
in some call sites, prefer a type-guard to ensurestring[]
at compile-time.- static presentCalendarIds(selectedCalendars: IntegrationCalendar[]): string[] { - return selectedCalendars.reduce<string[]>((acc, sc) => { - if (sc.id !== undefined) { - acc.push(sc.id); - } - return acc; - }, []); - } + static presentCalendarIds(selectedCalendars: IntegrationCalendar[]): string[] { + return selectedCalendars + .map((sc) => sc.id) + .filter((id): id is string => typeof id === "string" && id.length > 0); + }Please confirm
IntegrationCalendar.id
is alwaysstring
. If it can be number, we should adjust the return type and consumers accordingly.
20-20
: Minor: extract source string into a constantAvoid string duplication and typos by centralizing the source value.
-const log = logger.getSubLogger({ prefix: ["CalendarCacheService"] }); +const log = logger.getSubLogger({ prefix: ["CalendarCacheService"] }); +const SOURCE = "calendar-cache-sql";Then use
SOURCE
in presenters:- source: "calendar-cache-sql", + source: SOURCE,
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
packages/features/calendar-cache-sql/CalendarCacheService.ts
(1 hunks)packages/types/Calendar.d.ts
(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/types/Calendar.d.ts
🧰 Additional context used
📓 Path-based instructions (3)
**/*Service.ts
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
Service files must include
Service
suffix, use PascalCase matching exported class, and avoid generic names (e.g.,MembershipService.ts
)
Files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
**/*.ts
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
**/*.ts
: For Prisma queries, only select data you need; never useinclude
, always useselect
Ensure thecredential.key
field is never returned from tRPC endpoints or APIs
Files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
**/*.{ts,tsx}
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
Flag excessive Day.js use in performance-critical code; prefer native Date or Day.js
.utc()
in hot paths like loops
Files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
🧠 Learnings (2)
📚 Learning: 2025-08-05T12:04:29.037Z
Learnt from: din-prajapati
PR: calcom/cal.com#21854
File: packages/app-store/office365calendar/__tests__/unit_tests/SubscriptionManager.test.ts:0-0
Timestamp: 2025-08-05T12:04:29.037Z
Learning: In packages/app-store/office365calendar/lib/CalendarService.ts, the fetcher method in Office365CalendarService class is public, not private. It was specifically changed from private to public in this PR to support proper testing and external access patterns.
Applied to files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
📚 Learning: 2025-07-18T17:57:16.395Z
Learnt from: vijayraghav-io
PR: calcom/cal.com#21072
File: packages/app-store/office365calendar/api/webhook.ts:120-123
Timestamp: 2025-07-18T17:57:16.395Z
Learning: The office365calendar webhook handler in packages/app-store/office365calendar/api/webhook.ts is specifically designed for Office365 calendar integration, not as a generic webhook handler. Therefore, it's safe to assume that fetchAvailabilityAndSetCache method will be implemented in the Office365CalendarService, making explicit validation checks unnecessary.
Applied to files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Atoms E2E Tests
🔇 Additional comments (1)
packages/features/calendar-cache-sql/CalendarCacheService.ts (1)
72-79
: Good separation of concerns and adherence to Service conventionsClass name/file suffix match the guidelines, repositories are injected (DI-friendly), and a Presenter isolates response shaping. This will be easy to swap/extend during rollout.
Bump @calcom/platform-libraries dependency from version 0.0.314 to 0.0.316 in package.json and yarn.lock to use the latest features and fixes.
…itory.ts Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
…itory.ts Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Refactors channelExpiration from string to Date type across repository interfaces, implementations, and service logic. Updates tests to use Date objects for channelExpiration, and adds parsing logic to handle both timestamp and ISO string formats for expiration values.
Throws an error if the watch response is missing channel id or expiration, improving robustness and error reporting in CalendarSubscriptionService.
Replaces thrown error with setting a watch error on the subscription when the watch response is missing channel id or expiration. Increments error count and continues processing, improving error handling and robustness.
Signed-off-by: Omar López <zomars@me.com> # Conflicts: # apps/api/v2/package.json # yarn.lock
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
♻️ Duplicate comments (3)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts (1)
162-171
: Type mismatch: channelExpiration should be Date (matches schema/interface)The interface and Prisma model use DateTime. Using string here risks invalid values and breaks comparisons.
Apply:
async updateWatchDetails( id: string, details: { channelId: string; channelKind?: string; channelResourceId?: string; channelResourceUri?: string; - channelExpiration: string; + channelExpiration: Date; } ) {packages/features/calendar-cache-sql/CalendarCacheService.ts (2)
31-38
: Mask event titles to avoid PII leakage in availability responsesDo not expose event.summary in public/free-busy outputs. Always return “Busy”.
Apply:
static presentAvailabilityData(events: CalendarEventForAvailability[]): EventBusyDate[] { return events.map((event) => ({ start: event.start.toISOString(), end: event.end.toISOString(), source: "calendar-cache-sql", - title: event.summary || "Busy", + title: "Busy", })); }
59-69
: Also mask titles in timezone variantSame rationale as above; avoid leaking summaries.
Apply:
static presentAvailabilityDataWithTimezones( events: CalendarEventForAvailability[] ): Ensure<EventBusyDate, "timeZone">[] { return events.map((event) => ({ start: event.start.toISOString(), end: event.end.toISOString(), timeZone: event.timeZone || "", source: "calendar-cache-sql", - title: event.summary || "Busy", + title: "Busy", })); }
🧹 Nitpick comments (4)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts (3)
61-69
: Use explicitselect
to comply with Prisma “select-only” guidelineThis read currently returns all columns. Add a
select
projection consistent with other methods to keep the contract tight.Apply:
async findByCredentialId(credentialId: number) { return await this.prismaClient.calendarSubscription.findFirst({ where: { selectedCalendar: { credentialId, }, }, + select: { + id: true, + selectedCalendarId: true, + channelId: true, + channelKind: true, + channelResourceId: true, + channelResourceUri: true, + channelResource: true, + clientState: true, + channelExpiration: true, + syncCursor: true, + syncErrors: true, + maxSyncErrors: true, + backoffUntil: true, + createdAt: true, + updatedAt: true, + }, }); }
136-150
: Batch upsert: consider chunking to avoid large concurrent burstsUsing Promise.all is fine for small/medium sets. If this ever handles hundreds/thousands of IDs, chunk requests to keep DB and connection pool healthy.
I can provide a chunking helper if you expect large batches.
241-265
: Optional: Use the error parameter for observability and progressive backoff contextThe
_error
arg is ignored. Consider logging it at debug/warn level to aid troubleshooting, without storing sensitive payloads.- async setWatchError(id: string, _error: string) { + async setWatchError(id: string, error: string) { await this.prismaClient.$transaction(async (tx) => { + // Optionally log error context (avoid logging tokens/PII) + // log.warn({ id, error }, "Set watch error");packages/features/calendar-cache-sql/CalendarCacheService.ts (1)
118-121
: Reduce PII in logs: avoid logging full selectedCalendars payloadLogging the entire selectedCalendars can include emails and other PII. Log counts and IDs instead.
Apply:
- log.debug("Getting availability from cache", safeStringify({ dateFrom, dateTo, selectedCalendars })); + const selectedCalendarIds = CalendarCachePresenter.presentCalendarIds(selectedCalendars); + log.debug( + "Getting availability from cache", + safeStringify({ dateFrom, dateTo, selectedCalendarCount: selectedCalendars.length, selectedCalendarIds }) + );
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
yarn.lock
is excluded by!**/yarn.lock
,!**/*.lock
📒 Files selected for processing (6)
apps/api/v2/package.json
(1 hunks)packages/features/calendar-cache-sql/CalendarCacheService.test.ts
(1 hunks)packages/features/calendar-cache-sql/CalendarCacheService.ts
(1 hunks)packages/features/calendar-cache-sql/CalendarSubscriptionRepository.interface.ts
(1 hunks)packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
(1 hunks)packages/features/calendar-cache-sql/CalendarSubscriptionService.ts
(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/features/calendar-cache-sql/CalendarCacheService.test.ts
- packages/features/calendar-cache-sql/CalendarSubscriptionService.ts
🧰 Additional context used
📓 Path-based instructions (4)
**/*Service.ts
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
Service files must include
Service
suffix, use PascalCase matching exported class, and avoid generic names (e.g.,MembershipService.ts
)
Files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
**/*.ts
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
**/*.ts
: For Prisma queries, only select data you need; never useinclude
, always useselect
Ensure thecredential.key
field is never returned from tRPC endpoints or APIs
Files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.interface.ts
**/*.{ts,tsx}
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
Flag excessive Day.js use in performance-critical code; prefer native Date or Day.js
.utc()
in hot paths like loops
Files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.interface.ts
**/*Repository.ts
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
Repository files must include
Repository
suffix, prefix with technology if applicable (e.g.,PrismaAppRepository.ts
), and use PascalCase matching the exported class
Files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
🧠 Learnings (7)
📚 Learning: 2025-08-05T12:04:29.037Z
Learnt from: din-prajapati
PR: calcom/cal.com#21854
File: packages/app-store/office365calendar/__tests__/unit_tests/SubscriptionManager.test.ts:0-0
Timestamp: 2025-08-05T12:04:29.037Z
Learning: In packages/app-store/office365calendar/lib/CalendarService.ts, the fetcher method in Office365CalendarService class is public, not private. It was specifically changed from private to public in this PR to support proper testing and external access patterns.
Applied to files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
📚 Learning: 2025-07-18T17:57:16.395Z
Learnt from: vijayraghav-io
PR: calcom/cal.com#21072
File: packages/app-store/office365calendar/api/webhook.ts:120-123
Timestamp: 2025-07-18T17:57:16.395Z
Learning: The office365calendar webhook handler in packages/app-store/office365calendar/api/webhook.ts is specifically designed for Office365 calendar integration, not as a generic webhook handler. Therefore, it's safe to assume that fetchAvailabilityAndSetCache method will be implemented in the Office365CalendarService, making explicit validation checks unnecessary.
Applied to files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
📚 Learning: 2025-08-08T09:29:11.681Z
Learnt from: Udit-takkar
PR: calcom/cal.com#22919
File: packages/features/calAIPhone/interfaces/AIPhoneService.interface.ts:118-143
Timestamp: 2025-08-08T09:29:11.681Z
Learning: In calcom/cal.com PR #22919, packages/features/calAIPhone/interfaces/AIPhoneService.interface.ts (TypeScript), the AIPhoneServiceAgentListItem is required to include user.email in listAgents responses (per maintainer Udit-takkar). Future reviews should not flag this as unnecessary PII unless requirements change.
Applied to files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
📚 Learning: 2025-08-08T10:26:13.362Z
Learnt from: Udit-takkar
PR: calcom/cal.com#22919
File: packages/features/calAIPhone/providers/retellAI/services/PhoneNumberService.ts:212-220
Timestamp: 2025-08-08T10:26:13.362Z
Learning: In calcom/cal.com PR #22919, packages/features/calAIPhone/providers/retellAI/services/PhoneNumberService.ts should include the phone number in client-facing HttpError messages (e.g., in updatePhoneNumber/getPhoneNumber catch blocks). Do not suggest redacting the phone number from these errors unless requirements change (per maintainer: Udit-takkar).
Applied to files:
packages/features/calendar-cache-sql/CalendarCacheService.ts
📚 Learning: 2025-07-28T11:50:23.946Z
Learnt from: CR
PR: calcom/cal.com#0
File: .cursor/rules/review.mdc:0-0
Timestamp: 2025-07-28T11:50:23.946Z
Learning: Applies to **/*.ts : For Prisma queries, only select data you need; never use `include`, always use `select`
Applied to files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
📚 Learning: 2025-07-18T08:47:01.264Z
Learnt from: vijayraghav-io
PR: calcom/cal.com#21072
File: packages/prisma/schema.prisma:891-891
Timestamp: 2025-07-18T08:47:01.264Z
Learning: The Outlook Calendar integration in Cal.com intentionally reuses subscription IDs across multiple event types for efficiency. The `upsertSelectedCalendarsForEventTypeIds` method creates separate SelectedCalendar records for each eventTypeId, all sharing the same outlookSubscriptionId. This subscription sharing pattern means that unique constraints like `@unique([outlookSubscriptionId, eventTypeId])` should not be applied as they would prevent this intended functionality.
Applied to files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
📚 Learning: 2025-08-07T18:42:34.081Z
Learnt from: Udit-takkar
PR: calcom/cal.com#22919
File: packages/lib/server/repository/PrismaPhoneNumberRepository.ts:412-417
Timestamp: 2025-08-07T18:42:34.081Z
Learning: In Cal.com codebase, the coding guideline requiring explicit `select` clauses instead of `include` for Prisma queries applies to read operations but not to update operations. Update operations don't need explicit select clauses.
Applied to files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
🧬 Code Graph Analysis (2)
packages/features/calendar-cache-sql/CalendarCacheService.ts (3)
packages/features/calendar-cache-sql/CalendarEventRepository.interface.ts (2)
CalendarEventForAvailability
(4-7)ICalendarEventRepository
(15-47)packages/types/Calendar.d.ts (6)
EventBusyDate
(56-62)IntegrationCalendar
(247-259)Calendar
(270-316)CalendarServiceEvent
(266-268)NewCalendarEventType
(73-86)CalendarEvent
(165-228)packages/features/calendar-cache-sql/CalendarSubscriptionRepository.interface.ts (1)
ICalendarSubscriptionRepository
(16-49)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts (2)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.interface.ts (1)
ICalendarSubscriptionRepository
(16-49)packages/prisma/index.ts (1)
PrismaClient
(83-83)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Install dependencies / Yarn install & cache
- GitHub Check: Atoms E2E Tests
🔇 Additional comments (6)
apps/api/v2/package.json (1)
41-41
: Version bump LGTM; confirm no breaking changes in 0.0.316 for API v2 runtimeThe scoped bump to @calcom/platform-libraries looks fine. If this package affects DI tokens or shared types used by the new SQL cache routes, just sanity-check locally for regressions.
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts (3)
10-18
: Good: safe credential projection excludes sensitive fieldsExplicit select with a narrow projection is aligned with the “select-only” guideline and prevents returning credential.key.
122-134
: Upsert by selectedCalendarId: concise and correctTransactionally safe and minimal. Works with a unique index on selectedCalendarId.
267-276
: Clear watch error: straightforward resetResets counters and backoff as expected.
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.interface.ts (1)
1-14
: SafeCredential and SubscriptionWithSelectedCalendar types look solidGood explicit narrowing of credential fields and helpful composite type for subscription + selectedCalendar.
packages/features/calendar-cache-sql/CalendarCacheService.ts (1)
171-178
: getCredentialId: simple and correctMatches the Calendar interface’s optional method and returns the id from the injected credential.
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.interface.ts
Show resolved
Hide resolved
Updated the channelExpiration field from string to Date in CalendarSubscriptionRepository to ensure proper handling of expiration values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
♻️ Duplicate comments (1)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts (1)
181-239
: Fix invalid field-to-field comparison in Prisma filter; filter in application layer and add deterministic ordering.
this.prismaClient.calendarSubscription.fields.maxSyncErrors
is not a valid Prisma construct. Prisma does not support column-to-column comparisons inwhere
. Fetch a superset, filtersyncErrors < maxSyncErrors
in memory, then slice to the limit. Also add anorderBy
for determinism.Apply:
async getSubscriptionsToWatch(limit = 100) { const oneDayInMS = 24 * 60 * 60 * 1000; const tomorrow = new Date(Date.now() + oneDayInMS); - return await this.prismaClient.calendarSubscription.findMany({ - take: limit, + const rows = await this.prismaClient.calendarSubscription.findMany({ + // Over-fetch to compensate for post-filtering (syncErrors < maxSyncErrors) + take: limit * 3, where: { selectedCalendar: { user: { teams: { some: { team: { features: { some: { featureId: CALENDAR_CACHE_SQL_WRITE_FEATURE, }, }, }, }, }, }, integration: "google_calendar", }, - syncErrors: { - lt: this.prismaClient.calendarSubscription.fields.maxSyncErrors, - }, OR: [{ channelExpiration: null }, { channelExpiration: { lt: tomorrow } }], AND: [{ OR: [{ backoffUntil: null }, { backoffUntil: { lte: new Date() } }] }], }, + orderBy: [{ channelExpiration: "asc" }, { updatedAt: "asc" }], select: { id: true, selectedCalendarId: true, channelId: true, channelKind: true, channelResourceId: true, channelResourceUri: true, channelResource: true, clientState: true, channelExpiration: true, syncCursor: true, syncErrors: true, maxSyncErrors: true, backoffUntil: true, createdAt: true, updatedAt: true, selectedCalendar: { select: { id: true, externalId: true, integration: true, userId: true, credential: { select: safeCredentialSelectForCalendarCache, }, }, }, }, }); + return rows.filter((s) => s.syncErrors < s.maxSyncErrors).slice(0, limit); }
🧹 Nitpick comments (5)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts (5)
71-84
: Early-return on empty array to skip unnecessary DB call.Minor optimization: if
selectedCalendarIds
is empty, return[]
immediately.Apply:
async findBySelectedCalendarIds(selectedCalendarIds: string[]) { - return await this.prismaClient.calendarSubscription.findMany({ + if (selectedCalendarIds.length === 0) return []; + return await this.prismaClient.calendarSubscription.findMany({ where: { selectedCalendarId: { in: selectedCalendarIds, }, }, select: { id: true, selectedCalendarId: true, updatedAt: true, }, }); }
136-150
: Avoid unbounded Promise.all inside a transaction; chunk for scalability.Large arrays can create many concurrent operations within a single transaction. Chunking reduces DB pressure and memory spikes.
Apply:
async upsertManyBySelectedCalendarId(selectedCalendarIds: string[]) { // Use transactional callback for better compatibility with Prismock tests return await this.prismaClient.$transaction(async (tx) => { - const results = await Promise.all( - selectedCalendarIds.map((selectedCalendarId) => - tx.calendarSubscription.upsert({ - where: { selectedCalendarId }, - create: { selectedCalendar: { connect: { id: selectedCalendarId } } }, - update: { updatedAt: new Date() }, - }) - ) - ); - return results; + const results: Prisma.CalendarSubscription[] = []; + const BATCH_SIZE = 50; + for (let i = 0; i < selectedCalendarIds.length; i += BATCH_SIZE) { + const slice = selectedCalendarIds.slice(i, i + BATCH_SIZE); + const batch = await Promise.all( + slice.map((selectedCalendarId) => + tx.calendarSubscription.upsert({ + where: { selectedCalendarId }, + create: { selectedCalendar: { connect: { id: selectedCalendarId } } }, + update: { updatedAt: new Date() }, + }) + ) + ); + results.push(...batch); + } + return results; }); }
152-160
: Nit: Method name vs field name mismatch (token vs cursor).
updateSyncToken
writessyncCursor
. Consider renaming toupdateSyncCursor
(or documenting the mapping) to avoid confusion.
241-265
: setWatchError: Progressive backoff is good; consider using the error param or documenting why it’s ignored.If the schema has a place to store the last error (message/timestamp), consider persisting it. Otherwise, add a short comment indicating the error string is handled upstream to avoid confusion.
Where is the
error
parameter consumed or logged today (service layer? monitoring)? If not used, should we remove it from the interface or store it for troubleshooting?
61-69
: Consider narrowing selected fields since only existence is checkedIn the only call site (getCalendar.ts), the returned subscription is used solely to test for existence (truthiness), not to access any of its properties. You can reduce query payload and improve clarity by:
- Adding a
select: { id: true }
(or minimal required fields) tofindByCredentialId
- Alternatively, renaming it to something like
existsByCredentialId
and returning a booleanIf you ever need full subscription data downstream, update this method’s
select
accordingly./// packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts async findByCredentialId(credentialId: number) { - return await this.prismaClient.calendarSubscription.findFirst({ - where: { selectedCalendar: { credentialId } }, - }); + return await this.prismaClient.calendarSubscription.findFirst({ + where: { selectedCalendar: { credentialId } }, + select: { id: true }, // only fetch the primary key since we only check existence + }); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.test.ts
(1 hunks)packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/features/calendar-cache-sql/CalendarSubscriptionRepository.test.ts
🧰 Additional context used
📓 Path-based instructions (3)
**/*Repository.ts
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
Repository files must include
Repository
suffix, prefix with technology if applicable (e.g.,PrismaAppRepository.ts
), and use PascalCase matching the exported class
Files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
**/*.ts
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
**/*.ts
: For Prisma queries, only select data you need; never useinclude
, always useselect
Ensure thecredential.key
field is never returned from tRPC endpoints or APIs
Files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
**/*.{ts,tsx}
📄 CodeRabbit Inference Engine (.cursor/rules/review.mdc)
Flag excessive Day.js use in performance-critical code; prefer native Date or Day.js
.utc()
in hot paths like loops
Files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
🧠 Learnings (3)
📚 Learning: 2025-07-28T11:50:23.946Z
Learnt from: CR
PR: calcom/cal.com#0
File: .cursor/rules/review.mdc:0-0
Timestamp: 2025-07-28T11:50:23.946Z
Learning: Applies to **/*.ts : For Prisma queries, only select data you need; never use `include`, always use `select`
Applied to files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
📚 Learning: 2025-07-18T08:47:01.264Z
Learnt from: vijayraghav-io
PR: calcom/cal.com#21072
File: packages/prisma/schema.prisma:891-891
Timestamp: 2025-07-18T08:47:01.264Z
Learning: The Outlook Calendar integration in Cal.com intentionally reuses subscription IDs across multiple event types for efficiency. The `upsertSelectedCalendarsForEventTypeIds` method creates separate SelectedCalendar records for each eventTypeId, all sharing the same outlookSubscriptionId. This subscription sharing pattern means that unique constraints like `@unique([outlookSubscriptionId, eventTypeId])` should not be applied as they would prevent this intended functionality.
Applied to files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
📚 Learning: 2025-08-07T18:42:34.081Z
Learnt from: Udit-takkar
PR: calcom/cal.com#22919
File: packages/lib/server/repository/PrismaPhoneNumberRepository.ts:412-417
Timestamp: 2025-08-07T18:42:34.081Z
Learning: In Cal.com codebase, the coding guideline requiring explicit `select` clauses instead of `include` for Prisma queries applies to read operations but not to update operations. Update operations don't need explicit select clauses.
Applied to files:
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts
🧬 Code Graph Analysis (1)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts (2)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.interface.ts (1)
ICalendarSubscriptionRepository
(16-49)packages/prisma/index.ts (1)
PrismaClient
(83-83)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Install dependencies / Yarn install & cache
- GitHub Check: Atoms E2E Tests
🔇 Additional comments (5)
packages/features/calendar-cache-sql/CalendarSubscriptionRepository.ts (5)
7-18
: Safe credential projection looks solid (no key exposure).Good use of
satisfies Prisma.CredentialSelect
and explicit omission of sensitive fields likecredential.key
.
25-59
: findBySelectedCalendar: adheres to select-only rule and safe credential projection.The query is precise, returns only required fields, and keeps credentials safe.
86-120
: findByChannelId: correct switch to select-only and safe credential projection.This addresses prior feedback (include→select) and prevents credential leaks.
162-179
: updateWatchDetails: correct Date typing for channelExpiration.Matches schema and enables native Date comparisons.
267-276
: clearWatchError: straightforward and correct.Resets counters and backoff atomically.
Graphite Automations"Add community label" took an action on this PR • (08/20/25)1 label was added to this PR based on Keith Williams's automation. |
🚀 Calendar Cache SQL Implementation
calendar-cache-sql-demo - Watch Video
Summary
This PR implements a complete calendar cache SQL system to replace the current JSON-based CalendarCache with proper SQL tables for better performance and queryability. The implementation includes:
CalendarSubscription
andCalendarEvent
tables with proper indexes for performancecalendar-cache-sql-read
andcalendar-cache-sql-write
for manual team-based rolloutThe system is designed for gradual rollout using manual team-based feature flags via the
TeamFeature
table, with comprehensive error handling and monitoring capabilities.Overview
This PR introduces a comprehensive calendar caching system for Google Calendar integration, significantly reducing API calls to third-party providers while maintaining real-time synchronization through webhook subscriptions.
🎯 Key Features
Feature Flag System
calendar-cache-sql-read
,calendar-cache-sql-write
,calendar-cache-sql-cleanup
Subscription Management
Intelligent Cache Reading
🏗️ Architecture
Database Schema
CalendarSubscription
: Manages webhook subscriptions and sync stateCalendarEvent
: Stores cached calendar eventsServices
CalendarCacheService
: Main caching service implementing Calendar interfaceCalendarSubscriptionService
: Handles subscription lifecycleGoogleCalendarWebhookService
: Processes webhook notificationsCalendarCacheSqlCleanupService
: Manages old event cleanupRepository Pattern
CalendarEventRepository
: Event data access layerCalendarSubscriptionRepository
: Subscription data access layer✅ Comprehensive Testing Results
Subscription System
Cache Reading
🔧 Implementation Highlights
Smart Service Selection
Feature Flag Integration
Webhook Processing
🎯 Performance Impact
🔮 Future Enhancements (Wishlist)
🧪 Testing
All core functionality has been thoroughly tested including:
Breaking Changes: None - feature is opt-in via feature flags
Migration: Requires database migration for new CalendarSubscription and CalendarEvent tables
Review & Testing Checklist for Human
/api/webhook/google-calendar-sql
) with valid/invalid tokens to ensure proper authentication. VerifyGOOGLE_WEBHOOK_TOKEN
validation works correctly.CalendarEvent
table with correctetag
handling.TeamFeature
entries forcalendar-cache-sql-write
and verify the cron job only processes calendars for enabled teams.Recommended Test Plan:
Diagram
Notes
CalendarCache
data is requiredetag
tracking and deduplication to handle out-of-order Google webhooksCalendarEvent
for availability queries andCalendarSubscription
for webhook managementGOOGLE_WEBHOOK_TOKEN
and includes comprehensive error handlingSession Info: Implemented by @zomars via Devin session: https://app.devin.ai/sessions/2b24599eb86543928cf058a0ce35477e
Link to Devin run: https://app.devin.ai/sessions/2b24599eb86543928cf058a0ce35477e