useWhisper Native

• Dependencies

• Expo version

  • expo-av - speech recording
  • expo-file-system - convert speech binary data to base64
  • react-native-voice/voice - detect speaking event

• React Native version

  • react-native-audio-recorder-player: speech recording
  • react-native-audio-toolkit - speech recording
  • react-native-fs - convert speech binary data to base64
  • react-native-voice/voice - detect speaking event

• Tasks

  • speech recording

    • able to start recording
    • able to stop recording
    • able to pause recording
    • able to get recorded data in chunk

  • speech detection

    • able to detect speaking start event
    • able to detect speaking stop event

  • speech transcription

    • able to convert recorded data to base64 string
    • send base64 data to Whisper via multipart/form-data
    • able to transcribe in real-time via recorded chunks

  • silence removal (optional)

    • use ffmpeg to remove silence part from recorded audio