Versioned entities

#3457 introduced the concept of versioned entities for validating incoming data corresponding to public data structures against maintained Zod schemas and migrating them to the latest version before further consumption. This is achieved via verzod.

The existing versioned entities are:

• HoppCollection
• Environment
• HoppGQLRequest
• HoppRESTRequest

Generally, data that is persisted in localStorage and can evolve are potential candidates to be versioned entities. For instance, in the case of collections HoppCollection, they are persisted in localStorage under the personal workspace and can evolve with new field additions. #3505 added support for specifying authorization/headers at the collection level which introduced the same as a versioned entity. For entities like collections, environments, etc which can be imported/exported, importing a file conforming to an older version of the versioned entity is migrated to the latest version before consumption.

Data persisted under localStorage is migrated at the persistence service. Few methods are exposed on the versioned entity, safeParse being the most commonly used for validation/migration.

  const result = HoppRESTRequest.safeParse(data)
  
  if (result.type === "ok") {
    // Incoming data migrated to the latest version
    return result.value
  }
  // Validation failed, handle error.

The schemas conforming to different versions for an entity can be found under the v directory. For instance, different schemas corresponding to HoppRESTRequest can be found here.

Versioned entity definitions are kept in dedicated directories under the @hoppscotch/data package. An entity is created via the createVersionedEntitiy function from verzod, supplying the latest version latestVersion, a map of the various schemas versionMap and a definition for the getVersion() method.

The corresponding schema to validate against the incoming data is determined via the definition supplied for the getVersion method above. The v field is taken into account in such a case.

An example corresponding to HoppRESTRequest is given below.

Please take a look at the verzod documentation for more context regarding the usage

Proposal

The following are potential candidates to be made versioned entities.

• Global environment - Done in refactor: make global environment a versioned entity #4216.

#3779 added support for secret environment variables with a new secret field. Given below is the current schema for data maintained under localStorage.

Below is a sample versionMap definition with the expected schema variants.

const V0_SCHEMA = z.array(
  z.union([
    z.object({
      key: z.string(),
      value: z.string(),
      secret: z.literal(false),
    }),
    z.object({
      key: z.string(),
      secret: z.literal(true),
    }),
    z.object({
      key: z.string(),
      value: z.string(),
    }),
  ])
)

const V1_SCHEMA = z.object({
  v: z.literal(1),
  variables: z.array(
    z.union([
      z.object({
        key: z.string(),
        secret: z.literal(true),
      }),
      z.object({
        key: z.string(),
        value: z.string(),
        secret: z.literal(false),
      }),
    ])
  ),
})

• Settings

Given below is the expected schema. The v0 schema shouldn't have a v field similar to global-environment above to conform with the existing shape. There isn't a need for a v1 schema entry and can be added down the line with new field additions, although could optionally add it in with the v field addition alongside relevant migrations being made a versioned entity.

Consumed in the persistence service where the data read from localStorage is validated and necessary migrations are to be applied via the versioned entity.

• History (REST/GQL) variants

Given below are the expected schemas (v1). It can have different variants REST/GQL similar to the case of requests (HoppRESTRequest / HoppGQLRequest).

Consumed in the persistence service where the data read from localStorage is validated and necessary migrations are to be applied via the versioned entity.

Notes

The setupLocalPersistence method from the persistence service is invoked while the app boots up where data persisted in localStorage is validated and necessary migrations are applied.

Please ensure to follow the below convention:

• Definitions for global environment should be created under a global-environment directory within packages/hoppscotch-data/src.
• Definitions for settings should be created under a settings directory within packages/hoppscotch-data/src.
• For history, there should be a history directory under packages/hoppscotch-data/src with nested directories for rest & gql.