Skip to content

Add skill to replace words in text with corresponding emoji #733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 5, 2024

Conversation

Mohit-Gaur
Copy link
Contributor

Re-submitting PR #303

Describe the contribution to the taxonomy

Added a new skill to replace words in text with their corresponding emoji

Input given at the prompt

Replace words with emoji in this text: I had a pizza for dinner last night

Response from the original model

I 🍕 had 🍴 a 🍕 pizza 🍕 for 🍴 dinner 🍴 last night 🌙

Response from the fine-tuned model

I had a 🍕 for dinner last night 🌃

Prompt 2 - Testing the fine-tuned model with a new example

Replace words with emoji in this text: I'm going to the movies tonight, I can't wait to see the new superhero film!

Response that was received

I'm 🎬 going to the 🍿movies tonight, I can't 😱 wait to see the new superhero 🎥film!

Response that is now received instead

I'm going to the 🎥 tonight, I can't wait to see the new 🦸‍♀️🦸‍♂️ film!

Contribution checklist

  • The contribution was tested with ilab generate
  • No errors or warnings were produced by ilab generate
  • All commits are signed off (DCO)
  • The qna.yaml file contains at least 5 seed_examples
  • The qna.yaml file was linted and prettified (yaml-validator can do both)
  • An attribution.txt file in the same folder as the qna.yaml file.

@Mohit-Gaur Mohit-Gaur requested a review from a team as a code owner April 20, 2024 10:19
@github-actions github-actions bot added triage-needed (Auto labeled) skill is ready to be triaged skill (Auto labeled) labels Apr 20, 2024
Copy link

Beep, boop 🤖, Hi, I'm @instructlab-bot and I'm going to help you with your pull request. Thanks for you contribution! 🎉

I support the following commands:

  • @instructlab-bot precheck -- Check existing model behavior using the questions in this proposed change.
  • @instructlab-bot generate -- Generate a sample of synthetic data using the synthetic data generation backend infrastructure.
  • @instructlab-bot generate-local -- Generate a sample of synthetic data using a local model.
  • @instructlab-bot help -- Print this help message again.

Note

Results or Errors of these commands will be posted as a pull request check in the Checks section below

Note

Currently only maintainers belongs to [[taxonomy-triagers taxonomy-approvers taxonomy-maintainers labrador-org-maintainers instruct-lab-bot-maintainers]] teams are allowed to run these commands.

@jjasghar
Copy link
Member

@instructlab-bot precheck

Copy link

Beep, boop 🤖, Generating test data for your PR with the job type: precheck. Your Job ID is 188. The results will be presented below in the pull request status box. This may take several minutes...

Copy link

Results for job ID: 188 using the model merlinite-7b!

Results can be found here.

Signed-off-by: Mohit Gaur <56885276+Mohit-Gaur@users.noreply.github.com>
Signed-off-by: BJ Hargrave <hargrave@us.ibm.com>
@instructlab instructlab deleted a comment from RobotSail Apr 24, 2024
@jjasghar jjasghar added community-build-ready Triage Team has signed off for synthetic data generation and removed triage-needed (Auto labeled) skill is ready to be triaged labels Apr 24, 2024
Copy link

Beep, boop 🤖, Hi, I'm @instructlab-bot and I'm going to help you with your pull request. Thanks for you contribution! 🎉

I support the following commands:

  • @instructlab-bot precheck -- Check existing model behavior using the questions in this proposed change.
  • @instructlab-bot generate -- Generate a sample of synthetic data using the synthetic data generation backend infrastructure.
  • @instructlab-bot generate-local -- Generate a sample of synthetic data using a local model.
  • @instructlab-bot help -- Print this help message again.

Note

Results or Errors of these commands will be posted as a pull request check in the Checks section below

Note

Currently only maintainers belongs to [[taxonomy-triagers taxonomy-approvers taxonomy-maintainers labrador-org-maintainers instruct-lab-bot-maintainers]] teams are allowed to run these commands.

@jjasghar jjasghar merged commit 4872bf9 into instructlab:main Jun 5, 2024
@alinaryan
Copy link
Member

alinaryan commented Jun 5, 2024

Thank you for your submission. Here are your results:
Your PR decreased the granite-7b-lab model's performance by 0.27499999999999947 points.
The model's new score is: 4.508333333333334
The old score was: 4.783333333333333
||||||||||||||||||||||||||||||||||||||||||||| 45.083333333333336%

@alinaryan
Copy link
Member

alinaryan commented Jun 6, 2024

Thank you for your submission. Here are your results:
Your PR improved the merlinite-7b-lab model's performance by 0.18333333333333357 points.
The model's new score is: 6.483333333333333
The old score was: 6.3
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 64.83333333333333%

jjasghar pushed a commit to luke-inglis/taxonomy that referenced this pull request Jun 7, 2024
…lab#733)

Re-submitting PR instructlab#303 

**Describe the contribution to the taxonomy**

<!-- A concise description of what the contribution brings, replace
"..." in the bullet list -->

Added a new skill to replace words in text with their corresponding
emoji

**Input given at the prompt**

<!-- What you entered, replace "..." -->

```
Replace words with emoji in this text: I had a pizza for dinner last night
```

**Response from the original model**

<!-- What you received from the original model in response to your
input,
replace "..." -->

```
I 🍕 had 🍴 a 🍕 pizza 🍕 for 🍴 dinner 🍴 last night 🌙
```

**Response from the fine-tuned model**

<!-- Generate a synthetic dataset based on your newly added seed data;
train the model
with the synthetic data and now re-test the model's response with the
same prompt.
Replace "..." with what you receive with the finetuned model. -->

```
I had a 🍕 for dinner last night 🌃
```
---
**Prompt 2 - Testing the fine-tuned model with a new example**

<!-- What you entered, replace "..." -->

```
Replace words with emoji in this text: I'm going to the movies tonight, I can't wait to see the new superhero film!
```

**Response that was received**

<!-- What you received in response to your input, replace "..." -->

```
I'm 🎬 going to the 🍿movies tonight, I can't 😱 wait to see the new superhero 🎥film!
```

**Response that is now received instead**

<!-- What you receive with your contribution, replace "..." -->

```
I'm going to the 🎥 tonight, I can't wait to see the new 🦸‍♀️🦸‍♂️ film!
```

**Contribution checklist**

<!-- Insert an x between the empty brackets: [ ] >> [x] -->

- [x] The contribution was tested with `ilab generate`
- [x] No errors or warnings were produced by `ilab generate`
- [x] All [commits are signed
off](https://github.com/instructlab/taxonomy/blob/main/CONTRIBUTING.md#legal)
(DCO)
- [x] The `qna.yaml` file contains at least 5 `seed_examples`
- [x] The `qna.yaml` file was [linted](https://yamllint.com) and
[prettified](https://onlineyamltools.com/prettify-yaml)
([yaml-validator](https://jsonformatter.org/yaml-validator) can do both)
- [x] An `attribution.txt` file in the same folder as the `qna.yaml`
file.

Signed-off-by: Mohit Gaur <56885276+Mohit-Gaur@users.noreply.github.com>
Signed-off-by: BJ Hargrave <hargrave@us.ibm.com>
jjasghar pushed a commit to acidonper/taxonomy that referenced this pull request Jun 10, 2024
…lab#733)

Re-submitting PR instructlab#303 

**Describe the contribution to the taxonomy**

<!-- A concise description of what the contribution brings, replace
"..." in the bullet list -->

Added a new skill to replace words in text with their corresponding
emoji

**Input given at the prompt**

<!-- What you entered, replace "..." -->

```
Replace words with emoji in this text: I had a pizza for dinner last night
```

**Response from the original model**

<!-- What you received from the original model in response to your
input,
replace "..." -->

```
I 🍕 had 🍴 a 🍕 pizza 🍕 for 🍴 dinner 🍴 last night 🌙
```

**Response from the fine-tuned model**

<!-- Generate a synthetic dataset based on your newly added seed data;
train the model
with the synthetic data and now re-test the model's response with the
same prompt.
Replace "..." with what you receive with the finetuned model. -->

```
I had a 🍕 for dinner last night 🌃
```
---
**Prompt 2 - Testing the fine-tuned model with a new example**

<!-- What you entered, replace "..." -->

```
Replace words with emoji in this text: I'm going to the movies tonight, I can't wait to see the new superhero film!
```

**Response that was received**

<!-- What you received in response to your input, replace "..." -->

```
I'm 🎬 going to the 🍿movies tonight, I can't 😱 wait to see the new superhero 🎥film!
```

**Response that is now received instead**

<!-- What you receive with your contribution, replace "..." -->

```
I'm going to the 🎥 tonight, I can't wait to see the new 🦸‍♀️🦸‍♂️ film!
```

**Contribution checklist**

<!-- Insert an x between the empty brackets: [ ] >> [x] -->

- [x] The contribution was tested with `ilab generate`
- [x] No errors or warnings were produced by `ilab generate`
- [x] All [commits are signed
off](https://github.com/instructlab/taxonomy/blob/main/CONTRIBUTING.md#legal)
(DCO)
- [x] The `qna.yaml` file contains at least 5 `seed_examples`
- [x] The `qna.yaml` file was [linted](https://yamllint.com) and
[prettified](https://onlineyamltools.com/prettify-yaml)
([yaml-validator](https://jsonformatter.org/yaml-validator) can do both)
- [x] An `attribution.txt` file in the same folder as the `qna.yaml`
file.

Signed-off-by: Mohit Gaur <56885276+Mohit-Gaur@users.noreply.github.com>
Signed-off-by: BJ Hargrave <hargrave@us.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community-build-ready Triage Team has signed off for synthetic data generation skill (Auto labeled)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants