Hello Coders! 👾

Time for a little update on the progress I’ve been making on Roseie. Or should I say ‘we’ since Rosie is helping a lot :)

Conversion to TypeScript

The first big change we made was converting Rosie to TypeScript. This transition has improved the structure and readability of the codebase, making it easier to manage and maintain. A lot of small issues immidately came to light and needed to be fixed. Other parts have been disabled at first and are slowly re-implemented as we go.

Integrating Azure’s OpenAI SDK

We integrated Azure’s OpenAI SDK to handle the API calls. Initially, we used the ‘completions’ endpoint, but we have since moved to the assistant’s API, which offers more possibilities with thread-based conversations. Besides just having a conversation, integrating with the OpenAI SDK also allowed for the use of function calls. These little functions can be called by the AI. Through these Rosie can start interacting with the outside world.

Email and Calendar Access

First, we gave Rosie access to email and calendar data. This allows for more interactive and informative conversations about upcoming events, debugging, and viewing underlying data.

Microsoft To-Do Integration

Rosie can now interact with Microsoft To-Do. She can list, add, and update tasks, which is a key step toward having her help manage daily schedules and Pomodoros.

Date Interpretation

One of the challenges was getting Rosie to correctly interpret dates such as ’tomorrow’ or ‘Monday.’ We refined her instructions to improve accuracy, ensuring she can manage dates effectively.

GitHub Integration

The initial steps for GitHub integration are complete. Rosie can now interact with GitHub, making her more effective in development workflows.

Notion Integration

The next goal is to integrate Notion. This will allow Rosie to combine information from various sources, helping to create daily plans, work on tasks, and generate reports or devlogs.

Proactive Behavior

We are working on making Rosie more proactive. The goal is for her to start tasks on her own and handle specific responsibilities autonomously, such as managing emails and news. This is also one of the benefits of the OpenAI SDK integration.

Fixes and updates

The conversion to TypeScript and the switch to the OpenAI SDK broke a couple of features, but also brought the opportunity to update a couple parts.

Context-Specific Instructions

We enhanced the system for adding context-specific instructions. Rosie can now store and recall information based on the current context, which helps in managing tasks and projects more efficiently.

Debugging Event Issues

We spent time debugging an issue with events not firing due to a naming conflict with the EventEmitter from Node.js. Renaming the private property fixed the issue, and we learned valuable lessons about avoiding such conflicts.

Copy Code Feature

The copy code feature is now fixed, making it easier to transfer code snippets around.

Rosie’s Voice and Plugin System

We fixed Rosie’s voice and the old plugin system, transitioning everything to TypeScript. While these plugins were initially built for Twitch streams, they are now functional again.

Pomodoro Timer

One of the things I wanted to add to the Rosie suite of tools was a pomodoro timer. I’ve been using different apps for this for years, but having Rosie work with it would open up a lot of possibilities. The Pomodoro timer feature is complete and functional. Rosie can now help manage work sessions and breaks, improving productivity.

Multiple Assistants and Initiating Conversations

We experimented with using multiple assistants and found a way to have Rosie initiate conversations. This can be useful for notifying about incoming messages or other important updates.

Future Plans

The future looks promising for Rosie. We plan to refine her capabilities, making her more autonomous and useful in managing tasks and projects. The ultimate goal is to have her create plans, work on tasks together, and generate reports effortlessly.

Happy Coding! 🚀