Skip to content
This repository was archived by the owner on Sep 8, 2024. It is now read-only.

Add VK test - specific skill should not reply #2618

Merged
merged 1 commit into from
Jun 22, 2020
Merged

Conversation

krisgesling
Copy link
Contributor

Description

Adds a test tool opposite to then_wait. Instead of the test succeeding when the criteria is met, the test fails if the given criteria is detected.

This is then used to test if a specific Skill responds to an utterance.

Improvements needed

It feels like there is a better way to do this, but it's working for me right now, so thought I'd put it out in the world for critique. Suggestions welcome.

Would also prefer to utilize MSM to search for the name of the skill in the test which would add more flexibility to what test authors enter. Currently it requires that you enter the class name of the Skill.

How to test

With the following feature file for the News Skill:

Feature: mycroft-news

  Scenario Outline: play music with names similar to news channels
    Given an english speaking user
     When the user says "<play some music>"
     Then "NewsSkill" should not reply

    Examples:
      | play some music |
      | what time is it |
      | what's the weather |
      | play the bbc news |
      | what's news |

First two should pass, last two should fail.

Contributor license agreement signed?

@devops-mycroft devops-mycroft added the CLA: Yes Contributor License Agreement exists (see https://github.com/MycroftAI/contributors) label Jun 16, 2020
return False, debug
context.bus.new_message_available.wait(0.5)
# Timed out return debug from test
return True, debug
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test_failed variable doesn't sem to be used.

The answer to this is probably "No" but I'll ask anyway,

could this function be

    def then_wait_fail(msg_type, criteria_func, context, timeout=TIMEOUT):
    """Wait for a specified time failing if criteria is fulfilled.
    [...]
    """
    res = then_wait(msg_type, criteria_func, context, timeout)
    return (not res[0], res[1])

@devops-mycroft
Copy link

Voight Kampff Integration Test Succeeded (Results)

@forslund
Copy link
Collaborator

forslund commented Jun 16, 2020

Regarding using msm to lookup the skill as you suggest

The context contains msm so a check could be done if self.skill_id is added to metahere

    skill = context.msm.local_skills.get(skill_name) or context.msm.find(skill_name)
    if basename(skill.path) == skill_id_from_speak_message:
        do_stuff()

Or something similar perhaps?

Edit: Just to clarify, I think this is a good first pass that could be merged as is and then built upon to allow more variants of skill name.

@krisgesling krisgesling force-pushed the feature/vk-not-reply branch from d825875 to b60c038 Compare June 18, 2020 06:42
@pep8speaks
Copy link

pep8speaks commented Jun 18, 2020

Hello @krisgesling! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-06-18 13:03:38 UTC

@krisgesling krisgesling force-pushed the feature/vk-not-reply branch from b60c038 to 028adb1 Compare June 18, 2020 06:43
@krisgesling
Copy link
Contributor Author

Thanks Ake, negating the existing function does work. I went through a lot of variations trying to use the existing functions rather than adding another. By the time I resigned myself to adding a new function to tools.py I'd lost sight of the easy answer!

Agree that we should add this, and I'll revisit the Skill searching and adding skill_id to the meta dict later.

@devops-mycroft
Copy link

Voight Kampff Integration Test Succeeded (Results)

1 similar comment
@devops-mycroft
Copy link

Voight Kampff Integration Test Succeeded (Results)

@devops-mycroft
Copy link

Voight Kampff Integration Test Failed (Results)

@forslund forslund self-requested a review June 18, 2020 14:31
@devops-mycroft
Copy link

Voight Kampff Integration Test Succeeded (Results)

@krisgesling krisgesling merged commit 200606d into dev Jun 22, 2020
@krisgesling krisgesling deleted the feature/vk-not-reply branch September 2, 2020 21:02
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
CLA: Yes Contributor License Agreement exists (see https://github.com/MycroftAI/contributors)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants