Image Recognition in webAR with AR.js 3.0

3 min readApr 8, 2020

Recently AR.js was moved from https://github.com/jeromeetienne/AR.js to https://github.com/AR-js-org/AR.js . They add new functionality which is image recognition without dark frame surrounding marker. It is possible by using NFT.

Look what we expect

Ok. At the beginning will be needed server. You can make it using Web Server for Chrome.

Caution: AR.js in not working on Chrome in iPhones. You need Android device

Next in Chrome open

chrome://inspect/#devices

Check Discover USB devices

And setup Port forwarding…

Connect you phone to pc in debugging mode and now you should able to see your device on the list. Type 127.0.0.1:5000 and click Open and Inspect.

You will see new blank window. Unlock your phone and open Chrome. Here we go. We can start with AR.

At first look at example and find a-nft block. There you can find reference to your image saved in specific format. This format is NFT, you need to generate this file from your *.jpg. Photo which i use. To generate you can use https://carnaux.github.io/NFT-Marker-Creator/ if you want to read how make good marker Creating good marker.

After generating you will have three files ( lake.fset , lake.fset3, lake.iset ), copy it to your server in folder ‘lake-example’. Next copy file from example to this same folder and edit a-nft url to:

<a-nft type='nft' url='lake-example/lake' 
smooth='true' smoothCount='10' smoothTolerance='0.01' smoothThreshold='5'>

After this let’s find animated model https://github.com/KhronosGroup/glTF-Sample-Models. I Choose CesiumMan. Get CesiumMan.glb and copy to ‘lake-example’. Edit *.html:

<script src="https://cdn.rawgit.com/donmccurdy/aframe-extras/cfe5f316/dist/aframe-extras.js"></script>...<a-entity
    gltf-model='CesiumMan.glb'
    scale="50 50 50"
    position="200 350 300"
    animation-mixer
>

And that’s it. You can download all project there.

<script src='https://cdn.jsdelivr.net/gh/aframevr/aframe@1c2407b26c61958baa93967b5412487cd94b290b/dist/aframe-master.min.js'></script><script src="https://cdn.rawgit.com/donmccurdy/aframe-extras/cfe5f316/dist/aframe-extras.js"></script><style>
    .arjs-loader { height: 100%; width: 100%; position: absolute; top: 0; left: 0; background-color: rgba(0, 0, 0, 0.8); z-index: 9999; display: flex; justify-content: center; align-items: center; }    .arjs-loader div { text-align: center; font-size: 1.25em; color: white; }</style>
<script src='https://raw.githack.com/AR-js-org/AR.js/master/aframe/build/aframe-ar-nft.js'></script><body style='margin : 0px; overflow: hidden;'>
    <div class="arjs-loader">
        <div>Loading, please wait...</div>
    </div>    <a-scene vr-mode-ui='enabled: false;' renderer="logarithmicDepthBuffer: true;" embedded arjs='trackingMethod: best; sourceType: webcam; debugUIEnabled: false;'>    <a-nft type='nft' url='lake-example/lake' smooth='true' smoothCount='10' smoothTolerance='0.01' smoothThreshold='5'>
        <a-entity
        gltf-model='CesiumMan.glb'
        scale="50 50 50"
        position="200 350 300"
        animation-mixer
        >
        </a-entity>
    </a-nft>
    <a-entity camera></a-entity>
    </a-scene>
</body>

Complete index.html

My english isn’t good but i hope everyone can understand every step.

Image Recognition in webAR with AR.js 3.0

Written by Marcin Kulwicki