1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
|
---
title: Using VR controllers with WebVR
slug: Web/API/WebVR_API/Using_VR_controllers_with_WebVR
translation_of: Web/API/WebVR_API/Using_VR_controllers_with_WebVR
---
<div>{{APIRef("WebVR API")}}</div>
<p class="summary">Many WebVR hardware setups feature controllers that go along with the headset. These can be used in WebVR apps via the <a href="https://developer.mozilla.org/en-US/docs/Web/API/Gamepad_API">Gamepad API</a>, and specifically the <a href="https://developer.mozilla.org/en-US/docs/Web/API/Gamepad_API#Experimental_Gamepad_extensions">Gamepad Extensions API</a> that adds API features for accessing <a href="https://developer.mozilla.org/en-US/docs/Web/API/GamepadPose">controller pose</a>, <a href="https://developer.mozilla.org/en-US/docs/Web/API/GamepadHapticActuator">haptic actuators</a>, and more. This article explains the basics.</p>
<p class="summary">许多WebVR硬件的功能设置控制器与头戴设备在一起,实际这些功能可以通过手柄控制器在WebVR软件中实现,尤其对于添加了姿态控制器,触觉驱动器,等拓展性API的手柄控制器。本篇文章介绍了一些基本的内容。</p>
<h2 id="The_WebVR_API">The WebVR API</h2>
<p>The <a href="/en-US/docs/Web/API/WebVR_API">WebVR API</a> is a nascent, but very interesting new feature of the web platform that allows developers to create web-based virtual reality experiences. It does this by providing access to VR headsets connected to your computer as {{domxref("VRDisplay")}} objects, which can be manipulated to start and stop presentation to the display, queried for movement data (e.g. orientation and position) that can be used to update the display on each frame of the animation loop, and more.</p>
<p>Before you read this article, you should really be familiar with the basics of the WebVR API already — go and read <a href="/en-US/docs/Web/API/WebVR_API/Using_the_WebVR_API">Using the WebVR API</a> first, if you haven't already done so, which also details browser support and required hardware setup.</p>
<h2 id="The_Gamepad_API">The Gamepad API</h2>
<p>The <a href="/en-US/docs/Web/API/Gamepad_API">Gamepad API</a> is a fairly well-supported API that allows developers to access gamepads/controllers connected to your computer and use them to control web apps. The basic Gamepad API provides access to connected controllers as {{domxref("Gamepad")}} objects, which can then be queried to find out what buttons are being pressed and thumbsticks (axes) are being moved at any point, etc.</p>
<p>You can find more about basic Gamepad API usage in <a href="/en-US/docs/Web/API/Gamepad_API/Using_the_Gamepad_API">Using the Gamepad API</a>, and <a href="/en-US/docs/Games/Techniques/Controls_Gamepad_API">Implementing controls using the Gamepad API</a>.</p>
<p>However, in this article we will mainly be concentrating on some of the new features provided by the {{specname("GamepadExtensions")}} API, which allows access to advanced controller information such as position and orientation data, control over haptic actuators (e.g. vibration hardware), and more. This API is very new, and currently is only supported and enabled by default in Firefox 55+ Beta/Nightly channels.</p>
<h2 id="Types_of_controller">Types of controller</h2>
<p>There are two types of controller you'll encounter with VR hardware:</p>
<ul>
<li>6DoF (six-degrees-of-freedom) controllers provide access to both positional and orientation data — they can manipulate a VR scene and the objects it contains with movement but also rotatation. A good example is the HTC VIVE controllers.</li>
<li>3DoF (three-degrees-of-freedom) controllers provide orientation but not positional data. A good example is the Google Daydream controller, which can be rotated to point to different things in 3D space like a laser pointer, but can't be moved inside a 3D scene.</li>
</ul>
<h2 id="Basic_controller_access">Basic controller access</h2>
<p>Now onto some code. Let's look first at the basics of how we get access to VR controllers with the Gamepad API. There are a few strange nuances to bear in mind here, so it is worth taking a look.</p>
<p>We've written up a simple example to demonstrate — see our <a href="https://github.com/mdn/webvr-tests/blob/master/vr-controller-basic-info/index.html">vr-controller-basic-info</a> source code (<a href="https://mdn.github.io/webvr-tests/vr-controller-basic-info/">see it running live here also</a>). This demo simply outputs information on the VR displays and gamepads connected to your computer.</p>
<h3 id="Getting_the_display_information">Getting the display information</h3>
<p>The first notable code is as follows:</p>
<pre class="brush: js">var initialRun = true;
if(navigator.getVRDisplays && navigator.getGamepads) {
info.textContent = 'WebVR API and Gamepad API supported.'
reportDisplays();
} else {
info.textContent = 'WebVR API and/or Gamepad API not supported by this browser.'
}</pre>
<p>Here we first use a tracking variable, <code>initialRun</code>, to note that this is the first time we have loaded the page. You'll find out more about this later on. Next, we detect to see if the WebVR and Gamepad APIs are supported by cheking for the existence of the {{domxref("Navigator.getVRDisplays()")}} and {{domxref("Navigator.getGamepads()")}} methods. If so, we run our <code>reportDisplays()</code> custom function to start the process off. This function looks like so:</p>
<pre class="brush: js">function reportDisplays() {
navigator.getVRDisplays().then(function(displays) {
console.log(displays.length + ' displays');
for(var i = 0; i < displays.length; i++) {
var cap = displays[i].capabilities;
// cap is a VRDisplayCapabilities object
var listItem = document.createElement('li');
listItem.innerHTML = '<strong>Display ' + (i+1) + '</strong>'
+ '<br>VR Display ID: ' + displays[i].displayId
+ '<br>VR Display Name: ' + displays[i].displayName
+ '<br>Display can present content: ' + cap.canPresent
+ '<br>Display is separate from the computer\'s main display: ' + cap.hasExternalDisplay
+ '<br>Display can return position info: ' + cap.hasPosition
+ '<br>Display can return orientation info: ' + cap.hasOrientation
+ '<br>Display max layers: ' + cap.maxLayers;
list.appendChild(listItem);
}
setTimeout(reportGamepads, 1000);
// For VR, controllers will only be active after their corresponding headset is active
});
}</pre>
<p>This function first uses the promise-based {{domxref("Navigator.getVRDisplays()")}} method, which resolves with an array containing {{domxref("VRDisplay")}} objects representing the connected displays. Next, it prints out each display's {{domxref("VRDisplay.displayId")}} and {{domxref("VRDisplay.displayName")}} values, and a number of useful values contained in the display's associated {{domxref("VRCapabilities")}} object. The most useful of these are {{domxref("VRCapabilities.hasOrientation","hasOrientation")}} and {{domxref("VRCapabilities.hasPosition","hasPosition")}}, which allow you to detect whether the device can return orientation and position data and set up your app accordingly.</p>
<p>The last line contained in this function is a {{domxref("WindowOrWorkerGlobalScope.setTimeout()")}} call, which runs the <code>reportGamepads()</code> function after a 1 second delay. Why do we need to do this? First of all, VR controllers will only be ready after their associated VR headset is active, so we need to invoke this after <code>getVRDisplays()</code> has been called and returned the display information. Second, the Gamepad API is much older than the WebVR API, and not promise-based. As you'll see later, the <code>getGamepads()</code> method is synchronous, and just returns the <code>Gamepad</code> objects immediately — it doesn't wait for the controller to be ready to report information. Unless you wait for a little while, returned information may not be accurate (at least, this is what we found in our tests).</p>
<h3 id="Getting_the_Gamepad_information">Getting the Gamepad information</h3>
<p>The <code>reportGamepads()</code> function looks like this:</p>
<pre class="brush: js">function reportGamepads() {
var gamepads = navigator.getGamepads();
console.log(gamepads.length + ' controllers');
for(var i = 0; i < gamepads.length; ++i) {
var gp = gamepads[i];
var listItem = document.createElement('li');
listItem.classList = 'gamepad';
listItem.innerHTML = '<strong>Gamepad ' + gp.index + '</strong> (' + gp.id + ')'
+ '<br>Associated with VR Display ID: ' + gp.displayId
+ '<br>Gamepad associated with which hand: ' + gp.hand
+ '<br>Available haptic actuators: ' + gp.hapticActuators.length
+ '<br>Gamepad can return position info: ' + gp.pose.hasPosition
+ '<br>Gamepad can return orientation info: ' + gp.pose.hasOrientation;
list.appendChild(listItem);
}
initialRun = false;
}</pre>
<p>This works in a similar manner to <code>reportDisplays()</code> — we get an array of {{domxref("Gamepad")}} objects using the non-promise-based <code>getGamepads()</code> method, then cycle through each one and print out information on each:</p>
<ul>
<li>The {{domxref("Gamepad.displayId")}} property is the same as the <code>displayId</code> of the headet the controller is associated with, and therefore useful for tying controller and headset information together.</li>
<li>The {{domxref("Gamepad.index")}} property is unique numerical index that identifies each connected controller.</li>
<li>{{domxref("Gamepad.hand")}} returns which hand the controller is expected to be held in.</li>
<li>{{domxref("Gamepad.hapticActuators")}} returns an array of the haptic actuators available in the controller. Here we are returning its length so we can see how many each has available.</li>
<li>Finally, we return {{domxref("GamepadPose.hasPosition")}} and {{domxref("GamepadPose.hasOrientation")}} to show whether the controller can return position and orientation data. This works just the same as for the displays, except that in the case of gamepads these values are available on the pose object, not the capabilities object.</li>
</ul>
<p>Note that we also gave each list item containing controller information a class name of <code>gamepad</code>. We'll explain what this is for later.</p>
<p>The last thing to do here is set the <code>initialRun</code> variable to <code>false</code>, as the initial run is now over.</p>
<h3 id="Gamepad_events">Gamepad events</h3>
<p>To finish off this section, we'll look at the gamepad-associated events. There are two we need concern ourselves with — {{event("gamepadconnected")}} and {{event("gamepaddisconnected")}} — and it is fairly obvious what they do.</p>
<p>At the end of our example we first include the <code>removeGamepads()</code> function:</p>
<pre class="brush: js">function removeGamepads() {
var gpLi = document.querySelectorAll('.gamepad');
for(var i = 0; i < gpLi.length; i++) {
list.removeChild(gpLi[i]);
}
reportGamepads();
}</pre>
<p>This function simply grabs references to all list items with a class name of <code>gamepad</code>, and removes them from the DOM. Then it re-runs <code>reportGamepads()</code> to populate the list with the updated list of connected controllers.</p>
<p><code>removeGamepads()</code> will be run each time a gamepad is connected or disconnected, via the following event handlers:</p>
<pre class="brush: js">window.addEventListener('gamepadconnected', function(e) {
info.textContent = 'Gamepad ' + e.gamepad.index + ' connected.';
if(!initialRun) {
setTimeout(removeGamepads, 1000);
}
});
window.addEventListener('gamepaddisconnected', function(e) {
info.textContent = 'Gamepad ' + e.gamepad.index + ' disconnected.';
setTimeout(removeGamepads, 1000);
});</pre>
<p>We have <code>setTimeout()</code> calls in place here — like we did with the initialization code at the top of the script — to make sure that the gamepads are ready to report their information when <code>reportGamepads()</code> is called in each case.</p>
<p>But there's one more thing to note — you'll see that inside the <code>gamepadconnected</code> handler, the timeout is only run if <code>initialRun</code> is <code>false</code>. This is because if your gamepads are connected when the document first loads, <code>gamepadconnected</code> is fired once for each gamepad, therefore <code>removeGamepads()</code>/<code>reportGamepads()</code> will be run several times. This could lead to inaccurate results, therefore we only want to run <code>removeGamepads()</code> inside the <code>gamepadconnected</code> handler after the initial run, not during it. This is what <code>initialRun</code> is for.</p>
<h2 id="Introducing_a_real_demo">Introducing a real demo</h2>
<p>Now let's look at the Gamepad API being used inside a real WebVR demo. You can find this demo at <a href="https://github.com/mdn/webvr-tests/tree/master/raw-webgl-controller-example">raw-webgl-controller-example</a> (<a href="https://mdn.github.io/webvr-tests/raw-webgl-controller-example/">see it live here also</a>).</p>
<p>In exactly the same way as our <a href="https://github.com/mdn/webvr-tests/tree/master/raw-webgl-example">raw-webgl-example</a> (see <a href="/en-US/docs/Web/API/WebVR_API/Using_the_WebVR_API">Using the WebVR API</a> for details), this renders a spinning 3D cube, which you can choose to present in a VR display. The only difference is that, while in VR presenting mode, this demo allows you to move the cube by moving a VR controller (the original demo moves the cube as you move your VR headset).</p>
<p>We'll explore the code differences in this version below — see <a href="https://github.com/mdn/webvr-tests/blob/master/raw-webgl-controller-example/webgl-demo.js">webgl-demo.js</a>.</p>
<h3 id="Accessing_the_gamepad_data">Accessing the gamepad data</h3>
<p>Inside the <code>drawVRScene()</code> function, you'll find this bit of code:</p>
<pre class="brush: js">var gamepads = navigator.getGamepads();
var gp = gamepads[0];
if(gp) {
var gpPose = gp.pose;
var curPos = gpPose.position;
var curOrient = gpPose.orientation;
if(poseStatsDisplayed) {
displayPoseStats(gpPose);
}
}</pre>
<p>Here we get the connected gamepads with {{domxref("Navigator.getGamepads")}}, then store the first gamepad detected in the <code>gp</code> variable. As we only need one gamepad for this demo, we'll just ignore the others.</p>
<p>The next thing we do is to get the {{domxref("GamepadPose")}} object for the controller stored in gpPose (by querying {{domxref("Gamepad.pose")}}), and also store the current gamepad position and orientation for this frame in variables so they are easuy to access later. We also display the post stats for this frame in the DOM using the <code>displayPoseStats()</code> function. All of this is only done if <code>gp</code> actually has a value (if a gamepad is connected), which stops the demo erroring if we don't have our gamepad connected.</p>
<p>Slightly later in the code, you can find this block:</p>
<pre class="brush: js">if(gp && gpPose.hasPosition) {
mvTranslate([
0.0 + (curPos[0] * 15) - (curOrient[1] * 15),
0.0 + (curPos[1] * 15) + (curOrient[0] * 15),
-15.0 + (curPos[2] * 25)
]);
} else if(gp) {
mvTranslate([
0.0 + (curOrient[1] * 15),
0.0 + (curOrient[0] * 15),
-15.0
]);
} else {
mvTranslate([
0.0,
0.0,
-15.0
]);
}</pre>
<p>Here we alter the position of the cube on the screen according to the {{domxref("GamepadPose.position","position")}} and {{domxref("GamepadPose.orientation","orientation")}} data received from the connected controller. These values (stored in <code>curPos</code> and <code>curOrient</code>) are {{domxref("Float32Array")}}s containing the X, Y, and Z values (here we are just using [0] which is X, and [1] which is Y).</p>
<p>If the <code>gp</code> variable has a <code>Gamepad</code> object inside it and it can return position values (<code>gpPose.hasPosition</code>), indicating a 6DoF controller, we modify the cube position using position and orientation values. If only the former is true, indicating a 3DoF controller, we modify the cube position using the orientation values only. If there is no gamepad connected, we don't modify the cube position at all.</p>
<h3 id="Displaying_the_gamepad_pose_data">Displaying the gamepad pose data</h3>
<p>In the <code>displayPoseStats()</code> function, we grab all of the data we want to display out of the {{domxref("GamepadPose")}} object passed into it, then print them into the UI panel that exists in the demo for displaying such data:</p>
<pre class="brush: js">function displayPoseStats(pose) {
var pos = pose.position;
var orient = pose.orientation;
var linVel = pose.linearVelocity;
var linAcc = pose.linearAcceleration;
var angVel = pose.angularVelocity;
var angAcc = pose.angularAcceleration;
if(pose.hasPosition) {
posStats.textContent = 'Position: x ' + pos[0].toFixed(3) + ', y ' + pos[1].toFixed(3) + ', z ' + pos[2].toFixed(3);
} else {
posStats.textContent = 'Position not reported';
}
if(pose.hasOrientation) {
orientStats.textContent = 'Orientation: x ' + orient[0].toFixed(3) + ', y ' + orient[1].toFixed(3) + ', z ' + orient[2].toFixed(3);
} else {
orientStats.textContent = 'Orientation not reported';
}
linVelStats.textContent = 'Linear velocity: x ' + linVel[0].toFixed(3) + ', y ' + linVel[1].toFixed(3) + ', z ' + linVel[2].toFixed(3);
angVelStats.textContent = 'Angular velocity: x ' + angVel[0].toFixed(3) + ', y ' + angVel[1].toFixed(3) + ', z ' + angVel[2].toFixed(3);
if(linAcc) {
linAccStats.textContent = 'Linear acceleration: x ' + linAcc[0].toFixed(3) + ', y ' + linAcc[1].toFixed(3) + ', z ' + linAcc[2].toFixed(3);
} else {
linAccStats.textContent = 'Linear acceleration not reported';
}
if(angAcc) {
angAccStats.textContent = 'Angular acceleration: x ' + angAcc[0].toFixed(3) + ', y ' + angAcc[1].toFixed(3) + ', z ' + angAcc[2].toFixed(3);
} else {
angAccStats.textContent = 'Angular acceleration not reported';
}
}</pre>
<h2 id="Summary">Summary</h2>
<p>This article has given you a very basic idea of how to use the Gamepad Extensions to use VR controllers inside WebVR apps. In a real app you'd probably have a much more complex control system in effect, with controls assigned to the buttons on the VR controllers, and the display being affected by both the display pose and the controller poses simultaneously. Here however, we just wanted to isolate the pure Gamepad Extensions parts of that.</p>
<h2 id="See_also">See also</h2>
<ul>
<li><a href="/en-US/docs/Web/API/WebVR_API">WebVR API</a></li>
<li><a href="/en-US/docs/Web/API/Gamepad_API">Gamepad API</a></li>
<li><a href="/en-US/docs/Web/API/WebVR_API/Using_the_WebVR_API">Using the WebVR API</a></li>
<li><a href="/en-US/docs/Games/Techniques/Controls_Gamepad_API">Implementing controls using the Gamepad API</a></li>
</ul>
|