Skip to content

Conversation

aSemy
Copy link
Contributor

@aSemy aSemy commented Jul 20, 2022

Update:

any() and slot() now support value classes!

Summary

  • Added new 'expect' functions for 'unboxing' a value class to get its internal type, and to 'wrap' an object instance within a value class.
  • refactored RecordingState matcher to account for value classes
  • refactored the SignatureValueGenerator jvm impl to recursively generate values for value classes

resolves #152
resolves #791
resolves #683
resolves #847

Fix (maybe?) #778?

Notes:


This PR will hopefully provide some support for value classes #152

WIP - just some failing tests at the moment (credit to @qoomon)

This PR is blocked at the moment because the Kotlin language level is set to 1.4 - see #850 MockK is now on language level 1.5 🎉

@Raibaz
Copy link
Collaborator

Raibaz commented Jul 20, 2022

I'm fine with upgrading Kotlin to 1.7.0 by default, just make sure we maintain backwards compatibility at least to a couple versions back, i.e. we can drop support to 1.4.* if needed but I'd keep at least support to 1.5.*.

@aSemy
Copy link
Contributor Author

aSemy commented Jul 22, 2022

Okay, I think I've cracked the code.

A quick summary:

To fix slots, TypedMatcher needs to account for value classes

/**
 * Checks if argument is of specific type
 */
interface TypedMatcher {
    val argumentType: KClass<*>

    fun checkType(arg: Any?): Boolean {
        return when {
            argumentType.simpleName === null -> true
            argumentType.isValue             -> {
                val unboxedClass = InternalPlatformDsl.unboxClass(argumentType)
                return unboxedClass.isInstance(arg)
            }
            else                             -> argumentType.isInstance(arg)
        }
    }
}

And to handle matching stubs to invocations, JvmSignatureValueGenerator also needs to account for value classes.

class JvmSignatureValueGenerator(val rnd: Random) : SignatureValueGenerator {
    override fun <T : Any> signatureValue(
        cls: KClass<T>,
        anyValueGeneratorProvider: () -> AnyValueGenerator,
        instantiator: AbstractInstantiator,
    ): T {

        if (cls.isValue) {
            val valueCls = InternalPlatformDsl.unboxClass(cls)
            val valueSig = signatureValue(valueCls, anyValueGeneratorProvider, instantiator)

            val constructor = cls.primaryConstructor!!.apply { isAccessible = true }
            return constructor.call(valueSig)
        }

        return cls.cast(
            when (cls) {
                java.lang.Boolean::class -> rnd.nextBoolean()
                java.lang.Byte::class -> rnd.nextInt().toByte()
                java.lang.Short::class -> rnd.nextInt().toShort()
                java.lang.Character::class -> rnd.nextInt().toChar()
                java.lang.Integer::class -> rnd.nextInt()
                java.lang.Long::class -> rnd.nextLong()
                java.lang.Float::class -> rnd.nextFloat()
                java.lang.Double::class -> rnd.nextDouble()
                java.lang.String::class -> rnd.nextLong().toString(16)

                else ->
                    @Suppress("UNCHECKED_CAST")
                    anyValueGeneratorProvider().anyValue(cls, isNullable = false) {
                        instantiator.instantiate(cls)
                    } as T
            }
        )
    }
}

In order to do this I created a common function on InternalPlatformDsl, and in JVM implemented it using the code from ValueClassSupport.kt

actual object InternalPlatformDsl {

    // ...

    actual fun unboxClass(cls: KClass<*>): KClass<*> {
        if (!cls.isValue) return cls

        // get backing field
        val backingField = cls.valueField()

        // get boxed value
        return backingField.returnType.classifier as KClass<*>
    }
}

I'll commit what I have. I think there are some edgecases if a value class' value is also a value class.

The test coverage for value classes is very low - basically every MockK API operation should verify it is compatible with value classes.

@aSemy aSemy marked this pull request as ready for review July 22, 2022 17:11
Comment on lines 261 to 282
// TODO this is copy-pasted from ValueClassSupport
// I will try to move that class so it's available here

private val valueClassFieldCache = mutableMapOf<KClass<out Any>, KProperty1<out Any, *>>()

private fun <T : Any> KClass<T>.valueField(): KProperty1<out T, *> {
@Suppress("UNCHECKED_CAST")
return valueClassFieldCache.getOrPut(this) {
require(isValue) { "$this is not a value class" }

// value classes always have a primary constructor...
val constructor = primaryConstructor!!.apply { isAccessible = true }
// ...and exactly one constructor parameter
val constructorParameter = constructor.parameters.first()
// ...with a backing field
val backingField = declaredMemberProperties
.first { it.name == constructorParameter.name }
.apply { isAccessible = true }

backingField
} as KProperty1<out T, *>
}
Copy link
Contributor Author

@aSemy aSemy Jul 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how to resolve this

TODO this is from ValueClassSupport.kt - try and refactor to avoid copy-pasting

  • ValueClassSupport.kt is in project mockk-agent-jvm
  • InternalPlatformDsl is in project mockk-dsl-jvm

I'm looking for suggestions. Maybe it's okay just to have it copy and pasted?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep i think if they are in those two separate modules there's no better way and we can live with it being copypasted.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a comment where the same functionality is implemented would help, i case that code ever needs to be updated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, and I've made an issue to try and de-duplicated it later #857

@Raibaz
Copy link
Collaborator

Raibaz commented Jul 22, 2022

Thanks a lot for putting this together!

I'll take a look early next week.

@aSemy
Copy link
Contributor Author

aSemy commented Jul 23, 2022

There's a bug. I tried adding a test case for #729, but it causes MockK to hang indefinitely!

    /** https://github.com/mockk/mockk/issues/729 */
    @Test
    fun `verify extension function with UInt return can be stubbed`() {

        val fn = mockk<String.() -> UInt>()

        every { "string".fn() } returns 777u

        val result = "string".fn()

        assertEquals(777u, result)
    }

update: it gets stuck in a loop somewhere

2022-07-23 12:38:52 [Test worker] TRACE io.mockk.impl.JvmMockKGateway - Starting JVM MockK implementation. Java version = 1.8.0_332. 
2022-07-23 12:38:52 [Test worker] TRACE i.m.proxy.jvm.JvmMockKAgentFactory - Byte buddy agent installed
2022-07-23 12:38:52 [Test worker] TRACE i.m.p.jvm.dispatcher.BootJarLoader - Bootstrap class loaded io.mockk.proxy.jvm.dispatcher.JvmMockKDispatcher
2022-07-23 12:38:52 [Test worker] TRACE i.m.p.jvm.dispatcher.BootJarLoader - Bootstrap class loaded io.mockk.proxy.jvm.dispatcher.JvmMockKWeakMap
2022-07-23 12:38:52 [Test worker] TRACE i.m.p.jvm.dispatcher.BootJarLoader - Bootstrap class loaded io.mockk.proxy.jvm.dispatcher.JvmMockKWeakMap$StrongKey
2022-07-23 12:38:52 [Test worker] TRACE i.m.p.jvm.dispatcher.BootJarLoader - Bootstrap class loaded io.mockk.proxy.jvm.dispatcher.JvmMockKWeakMap$WeakKey
2022-07-23 12:39:19 [Test worker] DEBUG i.m.i.i.AbstractMockFactory - Creating mockk for Function1 name=#1
2022-07-23 12:39:25 [Test worker] TRACE i.m.i.i.AbstractMockFactory - Building proxy for Function1 hashcode=37efd131
2022-07-23 12:39:33 [Test worker] TRACE i.m.p.j.t.JvmInlineInstrumentation - Retransforming [Ljava.lang.Class;@5cbe877d
2022-07-23 12:39:33 [Test worker] TRACE io.mockk.proxy.jvm.ProxyMaker - Building subclass proxy for interface kotlin.jvm.functions.Function1 with additional interfaces []
2022-07-23 12:39:40 [Test worker] TRACE io.mockk.proxy.jvm.ProxyMaker - Instantiating proxy for interface kotlin.jvm.functions.Function1 via instantiator
2022-07-23 12:39:40 [Test worker] TRACE i.m.proxy.jvm.ObjenesisInstantiator - Skipping instantiation subclassing class kotlin.jvm.functions.Function1$Subclass0 because class is not abstract.
2022-07-23 12:39:40 [Test worker] TRACE i.m.proxy.jvm.ObjenesisInstantiator - Creating new empty instance of class kotlin.jvm.functions.Function1$Subclass0
2022-07-23 12:39:40 [Test worker] TRACE i.m.i.recording.CommonCallRecorder - Starting stubbing
2022-07-23 12:39:51 [Test worker] TRACE i.m.i.instantiation.JvmInstantiator - Building empty instance void
2022-07-23 12:39:51 [Test worker] TRACE i.m.proxy.jvm.ObjenesisInstantiator - Creating new empty instance of class java.lang.Void
2022-07-23 12:39:53 [Test worker] TRACE i.m.i.i.AbstractMockFactory - Building proxy for Any hashcode=2a3a299
2022-07-23 12:39:57 [Test worker] TRACE i.m.p.j.t.JvmInlineInstrumentation - Retransforming [Ljava.lang.Class;@219f4597
2022-07-23 12:39:57 [Test worker] TRACE io.mockk.proxy.jvm.ProxyMaker - Taking instance of class java.lang.Object itself because it is not abstract and no additional interfaces specified.
2022-07-23 12:39:59 [Test worker] TRACE io.mockk.proxy.jvm.ProxyMaker - Instantiating proxy for class java.lang.Object via instantiator
2022-07-23 12:40:01 [Test worker] TRACE i.mockk.impl.recording.JvmAutoHinter - Auto hint for 2-th call: class kotlin.UInt
2022-07-23 12:40:06 [Test worker] TRACE i.m.i.i.AbstractMockFactory - Building proxy for UInt hashcode=99a65d3
2022-07-23 12:40:13 [Test worker] TRACE i.m.p.j.t.JvmInlineInstrumentation - Retransforming [Ljava.lang.Class;@32fdec40
2022-07-23 12:40:17 [Test worker] TRACE io.mockk.proxy.jvm.ProxyMaker - Taking instance of class kotlin.UInt itself because it is final.
2022-07-23 12:40:19 [Test worker] TRACE io.mockk.proxy.jvm.ProxyMaker - Instantiating proxy for class kotlin.UInt via instantiator
2022-07-23 12:40:19 [Test worker] TRACE i.m.proxy.jvm.ObjenesisInstantiator - Creating new empty instance of class kotlin.UInt
2022-07-23 12:40:25 [Test worker] TRACE i.mockk.impl.recording.JvmAutoHinter - Auto hint for 4-th call: class kotlin.UInt
2022-07-23 12:40:28 [Test worker] TRACE i.m.i.i.AbstractMockFactory - Building proxy for UInt hashcode=99a65d3
2022-07-23 12:40:35 [Test worker] TRACE io.mockk.proxy.jvm.ProxyMaker - Taking instance of class kotlin.UInt itself because it is final.
2022-07-23 12:40:37 [Test worker] TRACE io.mockk.proxy.jvm.ProxyMaker - Instantiating proxy for class kotlin.UInt via instantiator
2022-07-23 12:40:37 [Test worker] TRACE i.m.proxy.jvm.ObjenesisInstantiator - Creating new empty instance of class kotlin.UInt
2022-07-23 12:40:41 [Test worker] TRACE i.m.i.i.AbstractMockFactory - Building proxy for UInt hashcode=99a65d3

I guess this issue is related: #656

@qoomon
Copy link
Contributor

qoomon commented Jul 23, 2022

I think we can reduce the code of ValueClassSupport, have look at this. This looks so much cleaner to me.

package me.qoomon.enhancements.kotlin

import kotlin.reflect.KClass
import kotlin.reflect.KProperty1
import kotlin.reflect.full.declaredMemberProperties
import kotlin.reflect.jvm.isAccessible

/**
 * Underlying property value of a **`value class`** or self
 */
val <T : Any> T.boxedValue: Any?
    @Suppress("UNCHECKED_CAST")
    get() = if (!this::class.isValue) this
    else (this::class as KClass<T>).boxedProperty.get(this)

/**
 * Underlying property class of a **`value class`** or self
 */
val KClass<*>.boxedClass: KClass<*>
    get() = if (!this.isValue) this
    else this.boxedProperty.returnType.classifier as KClass<*>

/**
 * Underlying property of a **`value class`**
 */
private val <T : Any> KClass<T>.boxedProperty: KProperty1<T, *>
    get() = if (!this.isValue) throw UnsupportedOperationException("$this is not a value class")
    // value classes always have exactly one property
    else this.declaredMemberProperties.first().apply { isAccessible = true }

@aSemy
Copy link
Contributor Author

aSemy commented Jul 23, 2022

I think we can reduce the code of ValueClassSupport

Agreed - but fun <T : Any> KClass<T>.isValueClass() is necessary because isValue throws an exception when used from fun <T : Any> T.boxedValue(): Any?, I think when T is a KFunction

@qoomon
Copy link
Contributor

qoomon commented Jul 23, 2022

Is it the native isValue property that's throwing an exception or my polyfill implemention?

@qoomon
Copy link
Contributor

qoomon commented Jul 23, 2022

@aSemy I wasn't able to reproduce an exception by calling .isValue on a KFunction instance

@qoomon
Copy link
Contributor

qoomon commented Jul 23, 2022

@aSemy there was a bug in my reduced ValueClassSupport code in the previous comment. I fixed it already within the comment. .boxedClassneeds to check forif (this.isValue)instead ofif (this::class.isValue)`

@aSemy
Copy link
Contributor Author

aSemy commented Jul 24, 2022

The issue with tests like this

https://github.com/aSemy/mockk/blob/592bdaf710e1be674742afa61147dfad584d6072/mockk/common/src/test/kotlin/io/mockk/it/ValueClassTest.kt#L385-L410

failing is that value classes are handled inconsistently in MockK.

For example, if I comment out this line (or another one like it, I can't remember exactly)

https://github.com/aSemy/mockk/blob/592bdaf710e1be674742afa61147dfad584d6072/agent/android/src/main/kotlin/io/mockk/proxy/android/advice/Advice.kt#L84

then the extension function tests work

It's hard to keep track of all of the entry and exit points of MockK's stubbing and mocking and such.


I think a proper fix to this is to create a new internal KClassData type.

Whenever MockK has to handle a KClass, it needs to determine if it's a value class or not. If it's not, then no problem, continue as normal. Else, create a wrapper for the value class that can recursively unwrap the actual type (the one the compiler sees) and also wrap instances.

Creating a new type for this would ensure consistent behaviour across MockK.

sealed interface KClassData<T> {
  val cls: KClass<T>

  /** A regular [KClass] - treat as normal */
  @JvmInline
  value class Cls<T>(
    override val cls: KClass<T>
  ) : KClassData<T>
  
  /** A value class - must be wrapped/unwrapped as required */
  data class VCls<T, E>(
    override val cls: KClass<T>,
  ): KClassData<T> {
    val unwrappedType: KClass<E> = ...
    
    fun wrap(any: Any?): T = // recursively wrap

    fun unwrap(instance: T): E = // recursively unwrap
  }
}

I'm not going to do it in this ticket - just throwing out ideas. It would be a big refactor.

@aSemy
Copy link
Contributor Author

aSemy commented Jul 24, 2022

@qoomon @Raibaz can you cancel the checks? I think they're stuck. I don't want to burn through all your GitHub Action hours :)

build.gradle Outdated
Comment on lines 50 to 52
tasks.withType(Test).configureEach {
timeout.set(Duration.ofMinutes(2))
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this timeout for all test tasks - I think 2 minutes should be fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😎👍

build.gradle Outdated
Comment on lines 50 to 52
tasks.withType(Test).configureEach {
timeout.set(Duration.ofMinutes(2))
}
Copy link
Contributor Author

@aSemy aSemy Jul 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this timeout to prevent the tests from hanging. I think 2 5 minutes should be fine.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

build.gradle Outdated
Comment on lines 50 to 52
tasks.withType(Test).configureEach {
timeout.set(Duration.ofMinutes(2))
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😎👍

@Raibaz
Copy link
Collaborator

Raibaz commented Jul 24, 2022

Thanks a lot to both of you for looking so deep into this!

I'll review the code over the next days.

@qoomon
Copy link
Contributor

qoomon commented Jul 25, 2022

Thank you for this great project.

@Raibaz
Copy link
Collaborator

Raibaz commented Jul 26, 2022

Ok, this is really cool, thanks a lot!

Merging this as soon as the build finishes.

@Raibaz Raibaz merged commit 02a6d45 into mockk:master Jul 26, 2022
@danieldisu
Copy link

Thank you!

@dybarsky
Copy link

Thank you @Raibaz

@qoomon
Copy link
Contributor

qoomon commented Jul 29, 2022

@Raibaz I would have liked to be mentioned in the release changelog as well 😅.

@Raibaz
Copy link
Collaborator

Raibaz commented Jul 29, 2022

Damn you're right, sorry!

I edited the changelog.

@maciejtulaza
Copy link

maciejtulaza commented Jul 30, 2022

hey, I still notice issues with any() and value classes.

I described two problems I found here: #847 (comment)

Not sure if this is related to this issue, but I encountered it because I first had the issue described here, then upgraded the version of MockK, error was gone but another two came up to life 😅 (you can see it in the issue I linked)

@qoomon
Copy link
Contributor

qoomon commented Jul 30, 2022

@aSemy @Raibaz I found the bug causing Duration and Result errors.
Because declaredMemberProperties includes properties without backing fields as well just .first() is not sufficient . We need to add following filter .first { it.javaField != null }

Fixed code. WDYT?

private val <T : Any> KClass<T>.boxedProperty: KProperty1<T, *>
        get() = if (!this.isValue_safe) {
            throw UnsupportedOperationException("$this is not a value class")
        } else {
            // value classes always have exactly one property with a backing field
            this.declaredMemberProperties.first { it.javaField != null }.apply { isAccessible = true }
        }

@qoomon
Copy link
Contributor

qoomon commented Jul 30, 2022

I'll create a PR this evening, including some tests.

@qoomon
Copy link
Contributor

qoomon commented Jul 30, 2022

Have a look at #872

@milgner
Copy link
Contributor

milgner commented Dec 21, 2023

Just ran into this issue today. Method definition:

@JvmInline
value class ConnectorType(val name: String) {
    override fun toString() = name
}

@JvmInline
@Serializable
value class DataSourceId(@Serializable(with = UUIDSerializer::class) val id: UUID) {
    override fun toString() = id.toString()
}

class Supervisor() {
    suspend fun createSource(connectorInfo: ConnectorType, dataSourceId: DataSourceId) = either {
        //...
    }
}

These are both very straightforward and look like they should be matched with the current implementation.
However, the following test code

        val supervisor = mockk<Supervisor>()
        val dataSourceId = slot<DataSourceId>()
        val response = createSourceResponse { status = SupervisorOuterClass.StatusCode.OK }.right()
        
        coEvery {
            supervisor.createSource(TestConnector.TYPE, capture(dataSourceId))
        } returns response

produces

io.mockk.MockKException: Failed matching mocking signature for
SignedCall(retValue=, isRetValueMock=true, retType=class arrow.core.Either, self=Supervisor(#2), method=createSource-vYlXMxg(String, UUID, Continuation), args=[test, 00000000-0000-0000-0000-000000000000, Continuation at io.redigital.spectre.data_source.AddDataSourceTest$1$1$1.invokeSuspend(AddDataSourceTest.kt:34)], invocationStr=Supervisor(#2).createSource-vYlXMxg(test, 00000000-0000-0000-0000-000000000000, continuation {}))
left matchers: [slotCapture<DataSourceId>()]

Changing the parameter types to non-value-classes makes things work as expected.

mockk version: 1.13.8
Kotlin Version 1.9.21

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants