25 Apr 2018

iPhone Panorama with Forge Viewer

> Scan this QRCode with your iPhone camera to try now <

Introduction:

I recently had a customer asking - 'how do I create an iPhone panorama with Forge Viewer'.  The idea is simple (see video)...

1. open iPhone camera
2. scan QR-Code
3. aim your iPhone up, down to look around
 

pano

Bonus points - add 3D markers everywhere using the MarkupExt.js.

 

How does it work ?

There's some quaternion math that ties the onDeviceOrientation events (in Euler degrees) from the iPhone compass/accelerometer, into the Forge Viewer's position/target camera object.

Anyone who's tried this, knows there's a bunch of ugly edge cases due to the portait/landscape modes and iPhone/Android differences.

Now, I know what you are thinking, "why don't you just use the webVR extension?"

True, but I need full-screen panorama, not the split-screen that webVR* offers. 

*And the webvr-polyfill, has been 'somewhat' unreliable between versions.

Thankfully, the clever folks at Three.js have already figured out a robust solution already - deviceOrientation.js

I simply adapted this code to the Three.R71 camera/target model, and 'it just works'TM

To minimize the code further, I re-use the 'gamepad' interface inside the 'first person' tool. I create a custom 'gamepad' component and override the camera-direction with the events from 'deviceOrientation' event. 

DEMO

 

How to use:

Steps:

  1. Add <script src="deviceOrientationExt.js"></script> to your index.html
  2. Activate 'first person' tool programmatically, like this...
viewer.toolController.activateTool('firstperson')

firstpersontool

That's it.  

 

Generate a QR-Code:

Open your index.html page via your iPhone - or better yet, host your webpage publicly, then generate a QR-code and scan it with your iPhone/Android.

You can find an example of how to use the extension on GitHub here: https://github.com/wallabyway/deviceOrientationExt

Feel free to add any issues you find to my Github issues repo.

 

Source Code:

(function() {
AutodeskNamespace('Autodesk.Viewing.Extensions.GamepadModule');

Autodesk.Viewing.Extensions.GamepadModule = function(viewer) {

    window.addEventListener('deviceorientation', e => this.deviceOrientation = e );

    this.update = function(camera) {
        if (!this.deviceOrientation) return camera;

        doe = this.deviceOrientation;
        euler.set(deg(doe.beta), deg(doe.alpha), -deg(doe.gamma), 'YXZ'); // 'ZXY' for the device, but 'YXZ' for us
        qe.setFromEuler(euler); // orient the device

        quaternion.set(Math.sqrt(0.5), 0, 0, Math.sqrt(0.5)); //  PI/2 around the x-axis
        quaternion.multiply(qe); // orient the device
        quaternion.multiply(q1); // camera looks out the back of the device, not the top
        quaternion.multiply(q0.setFromAxisAngle(zee, -this.orient)); // adjust for screen orientation
        m1.makeRotationFromQuaternion(quaternion);
        camera.setRotationFromMatrix(m1);

        // adjust camera target
        var lookAtDir = new THREE.Vector3(0, 0, -1);
        lookAtDir.applyQuaternion(camera.quaternion);
        camera.target = camera.position.clone().add(lookAtDir.clone().multiplyScalar(10));

        viewer.impl.invalidate(true, true, true);
        return camera;
    }

    const zee = new THREE.Vector3(0, 0, 1);
    const deg = THREE.Math.degToRad;
    let doe = null;
    let qe = new THREE.Quaternion();
    let q0 = new THREE.Quaternion();
    let q1 = new THREE.Quaternion(-Math.sqrt(0.5), 0, 0, Math.sqrt(0.5)); // - PI/2 around the x-axis
    let quaternion = new THREE.Quaternion();
    let euler = new THREE.Euler();
    var m1 = new THREE.Matrix4();
    this.orient = 0;
    window.addEventListener('orientationchange', e => this.orient = deg(window.orientation), false);

    this.activate = function(toolName) {};

    this.deactivate = function() {};
};
})();

 

Related Article

Posted By

Michael Beale

Michael Beale

Michael Beale is a senior software engineer at Autodesk, where he has been a globe-trotting technical advocate for Forge. Michael focuses on connecting Autodesk cloud data and Autodesk Forge APIs to the browser. He’s also contributed to Autodesk Homestyler, the Forge Large Model Viewer (LMV), Autodesk Prooflab, Stereo-Panorama service with webVR and recently 3D formats, such as glTF and 3D-Tiles-Next. Twitter:...