VR Controller Haptics in WebXR

Intro

You might have noticed that when you are playing games on your VR headset, you can feel the controller vibrating. This haptic feedback is a great way to get a little bit of extra feeling when you are playing. Here’s how you can add it to your WebXR game. In this tutorial, I’m showing how to use haptics in a Wonderland component, but the part that is actually sending a pulse to the controller is not specific to Wonderland. If you’re looking for an example in A-Frame, you can find it in a game I build a while ago for the GameDevJS Game Jam.

Haptic Feedback API

In the WebXR Device API, there’s no support for haptic feedback, but instead,you can use the Gamepad API for it . The support is still experimental, but since it is implemented in the Meta browser it should be safe to use. The slightly difficult part is that you need to get access to the Gamepad API, luckily you can get that through the WebXR Device API.

Getting to the Gamepad API

As an example, I’m going to add a little vibration of the controller to a shooting component. To handle the input from the controller I’m using the default Input component of Wonderland. When the ‘gun’ component is added to the same object as the input component we can access it very easily.

I’d like to “shoot” my gun when I pull the trigger of the controller. There’s an event on the WebXR session we can subscribe to, the ‘select’ event . This event is not specific to any framework. We can get access to the WebXR session in Wonderland when the app switches to VR mode by using WL.onXRSessionStart. The ‘session’ object is passed as a parameter to the callback function. You can see this in context in lines 6 and 8 of the code example below .

To get access to the gamepad, we get use the parameter ‘inputSource’ of the ‘select’ event callback. The ‘inputSource’ contains a lot of information about the input, but we are only interested in the ‘gamepad’ property and the ‘handedness’. The ‘handedness’ is either the left or the right hand and we just use this property to check if we are about to vibrate the correct controller.

Once we have the gamepad object we pass this off to a function that will handle the haptic feedback, pulse in this case.

Pulse

The get the controller to vibrate we use the <code>pulse</code> function . This function takes two parameters, the first is the duration of the pulse and the second is the strength of the pulse. The strength is a value between 0 and 1. The duration is in milliseconds. The pulse function can be found on the ‘hapticActuators’ array on the gamepad object. There could be more than 1 haptic actuator, but we’re just going to assume we only need the first one. If there’s none we just return out of the pulse function. You can find the entire function in lines 20 to 26 in the example below.

Complete Example

The example below shows a component I’ve reused in pretty much all my Wonderland games in some shape or form. Extra to the description above, I’ve made sure the ‘select event’ is only added once. If the app switches to VR a second time, the initialized variable is already set thus the event is not added again.

Assets from Unreal in WebXR

Intro

Although it’s not possible to export from Unreal to WebXR, there are a lot of very nice assets available in the Unreal Marketplace that could be useful for your WebXR game. The license allows you to use these assets in your game, except for products that have been created by Epic Games themselves. But, since Unreal Engine uses its own format for assets it’s not straightforward.

Assets

There are a lot of great free assets in the Unreal Marketplace. But, they can even get better. Every first Tuesday of the month, there’s a new set of assets available for free that would normally cost money: Free for the month . Even if you don’t need the assets right now, they might become handy at a later moment. If you’ve bought them for free they stay yours after the month is over and they return back to their normal price.

Unreal Engine

I’m going to assume you never have really used Unreal Engine before, otherwise, you probably won’t have ended up here. It could be a little bit of work to get everything to work, especially if the textures are in TGA format. But, it’s worth it. My Christmas-themed game Santa&rsquo;s Workshop is built using an Unreal asset. I took the gamble to buy it and it worked out great.

First of all, you need to have the Unreal Engine installed on your computer. You can download it from here .

New Project

Now that you have the engine installed it’s time to launch it through the Epic Games Launcher and create a new project. As options, I just select Games, the Blank Template, and Blueprint. I don’t think it matters since we’re not using anything of Unreal except the exporter. I usually name the project after the asset I’m going to export.

Add Asset

To add the asset we want in a WebXR project, we need to go back to the Epic Games Launcher and look it up in the marketplace, or the library if you have already bought it. After you’ve found it you just click Add to Project and select the project you want to add the asset to. If you haven’t used the asset before it will be downloaded.

Export Models

Now that the assets are in the project you need to open it again and look for them in the content browser, usually at the bottom of the screen. Once you’ve found the folder containing the models or meshes, you need to select them all. This can be done by hitting CTRL+A. Then you Right Click and from the context menu select Asset Actions -> Bulk Export. You need to select a folder you want to export the assets to. Keep in mind that you can only export one folder at a time. You need to repeat the process for every folder. There are a few options you need to disable in the export. You don’t need Level of Detail, Collision and Preview Mesh as these will add geometry you probably don’t need.

Export Textures

The textures are not inside the models, therefore you need to export them separately. The process is the same as the models. The folder structure is the same as the folder structure in the asset pack. If you place the textures in the same place as you did with the models they end up following the correct structure.

Convert .tga to .png

There’s a big chance the textures you’ve exported in the previous step are in .tga (Targa) format. The tool you are going to use the models with is probably not going to support .tga files. If you’re going to build a level in Blender and bake your lighting it’s not necessary to convert the textures, because Blender supports TGA and in the export of the bake you can select to use .png.

For converting the textures I use a free tool called XnConvert, which can be downloaded here . In that tool, you can select a folder and it will recursively search for all the .tga files and convert them to .png.

After selecting the folders you can go to the output tab, change the format to .png

Use the assets

Now that you have the assets you can use them in your game. Keep in mind that the materials on the models are broken since they’re converted to .png. You need to fix them manually in the game engine you’re using.

Custom templates for Wonderland Editor

Intro

A lesser-known feature of Wonderland Engine is the ability to use custom templates. In this quick post, I’ll show you how I created my own.

TL;DR

If you’re just here for the download 👉 This is the Template .

Extract the zip file in %appdata%\Roaming\WonderlandEngine\Templates and you’re good to go.

The Template

By default, a template in Wonderland is not any different than a normal project. The easiest way to reuse something is to just copy a project to the %appdata%\Roaming\WonderlandEngine\Templates folder. You don’t need to copy the deploy folder since that is automatically created when building the project. Scripts and assets that you often use can be included in the template.

When instantiating a new project, Wonderland renames the .wlp file and renames it internally as well.

But that’s not enough for me.

Sorskoot Template

In my projects, I use a little bit more. I’m using npm to manage my dependencies and I’m using RxJS to manage state in my games. I have a global object that is named after my game and I’m not even using the default esbuild to build the project but have some custom Webpack set up. And then there are a couple of components and utilities I use all the time, for music, sound effects, and other things.

To get this to work I created a little Gulp script. The biggest challenge in this was to run it automatically. I’m pretty lazy when it comes to things like this. It’s the whole reason I wanted my own template in the first place. Luckily for me, it turned out there’s a default npm script that is executed when after an npm install, postinstall. If that script is defined in your package.json it will be executed automatically.

My little Gulp script renames the main bundle file I’m using. It reads the name of the project from the .wlp file (this is just a nice json file 😊). After that, it scans all .js files and a few .json files for the SorskootBasicTemplate, the name of the template. It then replaces this string with the project name found in the .wlp file. It checks if the bundle is there with the original name, to prevent the script from running twice.

Current limitations

Wonderland does not run npm install automatically when a new project is created. This means you’ll have to exit the editor after creating the project and run it. After the installation can restart the editor and load your project.

WebXR at Global XR Conference

I posted this to the WebXR discord ( https://discord.gg/Jt5tfaM) , but I thought it might be helpful to anyone interested outside of the Discord as well (but you should join 😉).

This week is the Global XR Conference (Dec 1 - 3) and there are a couple of WebXR related talks and workshops.

But I thought I’d make a list of all sessions and workshops that have the tag WebXR… Turned out be more than I thought 🙂 , but here they are:

December 1

December 2

December 3

They have a discord as well if you like to reach out to one of the speakers ().

Creating an APK from a WebXR app

Intro

Since recently it is possible to package progressive web apps into an APK using the tools provided by Oculus. This is a great way to get your app into the hands of users who are not able to use the web. And, it is just as easy as it sounds.

This tutorial is not a detailed walkthrough on how to create a WebXR app or how to create a PWA. It’s a quick overview of the steps you need to take to get your app packaged into an APK and deploy it to the Oculus Quest.

WebXR app

Let’s start with a simple example. I created a pretty empty A-Frame example that just shows a cube on a black background. To make everything work offline and to prevent cross-origin issues, I downloaded the A-Frame library and added it to the project.

Web manifest

The first step in converting the example is to add a webmanifest . Most of the file is pretty much the same for all Progressive Web Apps, with a few exceptions. The biggest and maybe the most important addition is "ovr_package_name":"TheCube.Sorskoot.com". This is the name of the package that will be used to install the app on the Oculus Quest. Another thing to mention is the display property. Right now there are two valid values, standalone and minimal-ui. I noticed no difference on the Quest. I will leave it as standalone for now. I hope that soon Oculus figures out a way of opening the app in full immersive mode, maybe by using fullscreen or something. For now, you have to show the UI to enter VR. In the example below I use a reference of my app running at localhost. If you release the app on a server or use a different address on your machine, you need to change this line. Lastly, there has to be at least one image in the icons array, a 512x512 image. This image has to have its purpose set to any maskable. More sizes are recommended.

Service Worker

For this example, I used a service worker from the service workers cookbook at serviceworke.rs . This service worker has a precache that loads the specified files as soon as possible and then updates the files in the background if possible. I’m not going into the details of how this works, because it is fully described at serviceworke.rs . The only downside of this service worker is that an update is always 1 behind. I would advise you to have a look at the cookbook and find a way that best suits your needs.

To see if it works you can look it up in the Chrome Dev Tools, under the Application tab. You should see the service worker in the list of installed scripts.

updating the app

To finalize the implementation of the progressive web app, we need to add a few lines to the index.html example. In the head section, I added a link tag referring to the web manifest. I also added a small piece of script to load the service worker.

Building the APK

Now that we have a complete PWA, we need to package it into an APK. This is done using the CLI tool provided by Oculus, you download it here . I believe this also needs Java 1.7 or later to be installed. The Android SDK or at least the Android SDK Build tools is also needed. I installed the tool on my C: drive in tools/ovr-platform-util-pwa. If you run the following from the command line it will create an APK file from your PWA. /tools/ovr-platform-util-pwa/ovr-platform-util.exe create-pwa -o TheCube.apk --android-sdk %localappdata%/Android/Sdk --manifest-content-file manifest.webmanifest The paths used in the command above are relative to the location of the tool, and probably will be completely different on your machine.

Deploy using adb

The fastest and easiest way to get your freshly built APK on your Oculus Quest is to use the Android debug CLI tool, adb . There’s only a short line needed to deploy the APK to the Quest. adb install TheCube.apk

npm scripts

To speed things up, I added a few npm scripts to the package.json file. You don’t really have to use npm or any packages in your project, but just having these scripts around saves a lot of time. I added the following to the scripts section of the package.json file:

scripts:{
    "build": "/tools/ovr-platform-util-pwa/ovr-platform-util.exe create-pwa -o TheCube.apk --android-sdk %localappdata%/Android/Sdk --manifest-content-file manifest.webmanifest",
    "deploy":"/tools/scrcpy/adb.exe install TheCube.apk",
}

Now I can just run npm run build and npm run deploy to build the APK and deploy the app to the Quest when it is connected.

Keep in mind that the folders might be different on your machine.

Closing words

At the moment of writing this, it is not possible to release the app in the Oculus store yet. It is possible to sign the APK with a keystore, but it is not possible to do anything more with that than what I’ve shown above. When Oculus starts allowing PWA/APK apps to be uploaded to the store by the general public, I will write part 2 of this tutorial showing what steps to take there.