Making rosie listen Rosie and overlay stuff

For a while, I wanted to turn Rosie, our Chatbot, into a voice assistant. Today we worked on that. We used the Web Speech API to do speech recognition. We started to connect to wit.ai for the intents.

Segments

Timestamp Topic
0:00:00 Hello World
0:09:43 Giving a tour around Rosie and the layout system
0:35:14 Working the speech recognition
2:16:00 Connect to Wit.ai
2:41:25 The end

Announcements

  • Improved the MIDI Extension, Great feedback!
  • Tried to create visual AI, face tracking and hand tracking. Unsuccessfull :(
  • !discord
  • !git

Project working on today

  • Want to make Rosie ‘listen’ and respond to spoken commands

Next Steps

  • Add more intents
  • Connect intents to Chatbot code
  • Combine layouts and bot into 1 app. Possibly Electron.

Things to Review & Notes

  • [21:45] +wingysam: for (const module of fs.readdirSync('modules')) { const moduleExports = modules[moduleExports.name] = moduleExports.run }

  • [22:19] +wingysam: Btw VSCode understands JSDoc natively, just type /** above a function and it will spit out a jsdoc template with your function's params

  • [22:25] +wingysam: https://github.com/sorskoot/RosieBot/issues/1

  • https://whatwebcando.today


Todays stream brought to you by

All the wonderful people that hung out with us in chat!

Rosie and Midi

This stream we fixed a couple of issues in the Stream Deck MIDI plugin we're building.

Until the MIDI plugin is in the official library, you can find it on the GitHub.

We also added 2 new sound effects to Rosie.

Segments

Timestamp Topic
0:00:00 Hello World
0:10:29 Fixing the MIDI plugin
2:29:00 Adding SFX to rosie
2:41:25 The END

Announcements

  • Midi plugin is working
  • !discord
  • !git

Project working on today

  • Improving the MIDI Extension
  • added 2 new sfx to Rosie.

Zombie Massacre VR Game - Part 2

This live stream is HAUNTED! I was planning on working on the Zombie Massacre WebVR game, but ended the stream early. There seems to be a ghost haunting today…

Apparitions and glitches start around 1:42:00.

Segments

Timestamp Topic

Announcements

  • Happy Halloween
  • Midi plugin is working
  • Question of the day, What's your favorite horror movie?
  • Building a small VR Game
  • !discord
  • !git

Project working on today

  • Throwing heads
  • Trying to hit them

Next Steps


Things to Review & Notes


Todays stream brought to you by

Subscribers


Raids

Name Raiders

Zombie Massacre VR Game - Part 1

Today we start a horror VR game for Halloween. Zombie Massacre


Announcements

  • Building a small VR Game
  • !discord
  • !git

Project working on today

  • A new WebVR game

Building Stream Deck Action for doing Rest calls

Building Stream Deck Action for Triggering Rosie. To be able for Rosie to handle Text To Speech, we needed something to make REST calls from our stream deck. We created a special plugin to do exactly that.


Announcements

  • Short Stream - Bad voice
  • Building Stream Deck Action for Triggering Rosie
  • !discord
  • !git

Project working on today

  • Stream Deck Action