How to Build Argumented Reality Apps In Android With Ar Core

(image Credit to google)

What is ARCore?

ARCore is a software development kit developed by Google that allows for augmented reality applications to be built.

ARCore uses three key technologies to integrate virtual content with the real environment:

Motion Tracking:

it allows the phone to understand its position relative to the world.

Environmental understanding:

It allows the phone to detect the size and location of all type of surfaces, vertical, horizontal and angled.

Light Estimation:

it allows the phone to estimate the environment’s current lighting conditions.

How it works

Ever seen why most increased reality applications expect you to more your camera around the space to identify surfaces? This is on the grounds that the application needs to manufacture its very own universe. ARCore is the same.

At the point when you move your telephone around, ARCore remembers the environmental factors and assembles its very own universe where it can put virtual articles. It utilizes movement following innovation to recognize certain highlights which permit it to monitor how these focuses move pair with your camera developments.

This is the reason when you place a virtual article in your room and go out and return, the item is still there!! ARCore has assembled a comprehension of the earth and fabricated a virtual existence where it recalls where each element is found.

At the point when you place an article, it figures it's situation corresponding to different items and when you return to a similar spot, the article is rendered once more.

AR Core Apps Can be Used in

Android
Android NDK
Unity for Android
Unity for iOS
iOS
Unreal

Getting Started

o get started with ARCore app development, you first need to enable ARCore in your project. This is simple as we will be using Android Studio and Sceneform SDK. There are two major operations Sceneform performs automatically:

Checking for availability of ARCore
Asking for camera permission

Step1.Create a new Android Studio project and select an empty activity.

Then Add the following dependency to your project level build.gradle file:

dependencies {    classpath 'com.google.ar.sceneform:plugin:1.5.0'}

Add the latest ARCore library as a dependency in your app's build.gradle file:

dependencies {
   
    implementation 'com.google.ar:core:1.16.0'
    implementation "com.google.ar.sceneform.ux:sceneform-ux:1.5.0"

}

Then sync Your project with Gradle files and wait for the build to finish. This will install the Sceneform SDK to the project and Sceneform plugin to AndroidStudio. It will help you to view the .sfb files. These files are the 3D models which are rendered in your camera. It also helps you in importing, viewing, and building 3D assets.

Presently with our Android Studio sync is complete and Sceneform SDK Installed, we can begin with creating our absolute first ARCore application.

In the first place, we have to add the Sceneform piece to our design record. This will be where we place all our 3D models. It deals with the camera instatement and authorization taking care of.

Head over to your fundamental design document. For my situation it is activity_main.xml and include the Sceneform part:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    tools:context=".MainActivity">

    <fragment
        android:id="@+id/ux_fragment"
        android:name="com.google.ar.sceneform.ux.ArFragment"
        android:layout_width="match_parent"
        android:layout_height="0dp"
        android:layout_weight="9" />

    <LinearLayout
        android:id="@+id/gallery_layout"
        android:layout_width="match_parent"
        android:layout_height="0dp"
        android:layout_weight="1"
        android:orientation="horizontal" />

</LinearLayout>

This is all that you need to do in the layout file. Now head over to Our MainActivity java file, in my case which is MainActivity.java. Add the method below in your class That will help with the Compactibility Check:

public static boolean checkIsSupportedDeviceOrFinish(final Activity activity) {  
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) {  
        Log.e(TAG, "Sceneform requires Android N or later");  
        Toast.makeText(activity, "Sceneform requires Android N or later", Toast.LENGTH_LONG).show(); 
        activity.finish();        return false;    } 
        String openGlVersionString =           
        ((ActivityManager) activity.getSystemService(Context.ACTIVITY_SERVICE)).getDeviceConfigurationInfo() 
        .getGlEsVersion();    if (Double.parseDouble(openGlVersionString) < MIN_OPENGL_VERSION) { 
            Log.e(TAG, "Sceneform requires OpenGL ES 3.0 later");
            Toast.makeText(activity, "Sceneform requires OpenGL ES 3.0 or later", Toast.LENGTH_LONG).show();
            activity.finish();
            return false;
            }
            return true;
    
}

This Method checks whether your device can support Sceneform SDK or not. The SDK requires Android API level 27 or more current and OpenGL ES form 3.0 or fresher. On the off chance that a gadget doesn't bolster these two, the Scene would not be rendered and your application will show a clear screen.

Despite the fact that, you can at present keep on conveying the various highlights of your application which don't require the Sceneform SDK.

Presently with the device similarity check total, we will construct our 3D show and append it to the scene.

You will need to add the 3D models which will be rendered on your screen. Now you can build these models yourself if you are familiar with 3D model creation. Or, you can visit Poly.Where you can get tons of 3d objects to continue with this project.After getting your prefered 3d model Create a new forlder inside the App directory And name it Sampledata Where your 3D objects will be placed.They Can come in various extensions eg

.mtl file
.obj file
.png file

The Next Step We Will be building The Model Through the following line of Code

private static final String TAG = MainActivity.class.getSimpleName();
    private static final double MIN_OPENGL_VERSION = 3.0;

    ArFragment arFragment;
    ModelRenderable lampPostRenderable;
    private Uri selectedObject;

    @Override
    @SuppressWarnings({"AndroidApiChecker", "FutureReturnValueIgnored"})
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ux_fragment);

        initializeGallery();

        
        );

First, we find the arFragment that we included in the layout file. This fragment is responsible for hosting the scene. You can think of it as the container of our scene.

Then we are using the ModelRenderable class to build our model. With the help of setSource method, we load our model from the .sfb file. This file was generated when we imported the assets. thenAccept method receives the model once it is built. We set the loaded model to our lampPostRenderable.

For error handling, we have .exceptionally method. It is called in case an exception is thrown.

Also this happens asynchronously, hence you don’t need to worry about multi-threading or deal with handlers XD.

With the model loaded and stored in the lampPostRenderable variable, we’ll now add it to our scene.

The next step We will be Adding Our Model to the Scene And Here we will need to set the onTap listener to our fragment to register the tap and place an object accordingly. The arFragment hosts our scene and will receive the tap events.Add the following code to onCreate method:

arFragment.setOnTapArPlaneListener(
                (HitResult hitresult, Plane plane, MotionEvent motionevent) -> {
                    if (plane.getType() != Plane.Type.HORIZONTAL_UPWARD_FACING)
                        return;

                    Anchor anchor = hitresult.createAnchor();
                    placeObject(arFragment, anchor, Uri.parse("chair.sfb"));
                }
        );

    }

We Have now set the onTapArPlaneListener to our AR fragment.Then we Continued to create our anchor from the HitResult using hitresult.createAnchor() and store it in an Anchor object.

Next, created a node out of this anchor. It will be called AnchorNode. It will be attached to the scene by calling the setParent method on it and passing the scene from the fragment.

Now we create a TransformableNode which will be our lamppost and set it to the anchor spot or our anchor node. The node still doesn’t have any information about the object it has to render. We’ll pass that object using lamp.setRenderable method which takes in a renderable as it’s parameter. Finally call lamp.select();

The Final MainActivity.Java should be similar to this

package com.danny.arcore;

import android.net.Uri;
import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;
import android.view.MotionEvent;
import android.widget.Toast;

import com.google.ar.core.Anchor;
import com.google.ar.core.HitResult;
import com.google.ar.core.Plane;
import com.google.ar.sceneform.AnchorNode;
import com.google.ar.sceneform.rendering.ModelRenderable;
import com.google.ar.sceneform.rendering.Renderable;
import com.google.ar.sceneform.ux.ArFragment;
import com.google.ar.sceneform.ux.TransformableNode;

public class MainActivity extends AppCompatActivity {
    private static final String TAG = MainActivity.class.getSimpleName();
    private static final double MIN_OPENGL_VERSION = 3.0;

    ArFragment arFragment;
    ModelRenderable lampPostRenderable;
    private Uri selectedObject;

    @Override
    @SuppressWarnings({"AndroidApiChecker", "FutureReturnValueIgnored"})
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ux_fragment);

        // initializeGallery();

        arFragment.setOnTapArPlaneListener(
                (HitResult hitresult, Plane plane, MotionEvent motionevent) -> {
                    if (plane.getType() != Plane.Type.HORIZONTAL_UPWARD_FACING)
                        return;

                    Anchor anchor = hitresult.createAnchor();
                    placeObject(arFragment, anchor, Uri.parse("chair.sfb"));
                }
        );

    }

    private void placeObject(ArFragment arFragment, Anchor anchor, Uri uri) {
        ModelRenderable.builder()
                .setSource(arFragment.getContext(), uri)
                .build()
                .thenAccept(modelRenderable -> addNodeToScene(arFragment, anchor, modelRenderable))
                .exceptionally(throwable -> {
                            Toast.makeText(arFragment.getContext(), "Error:" + throwable.getMessage(), Toast.LENGTH_LONG).show();
                            return null;
                        }

                );

    }

    private void addNodeToScene(ArFragment arFragment, Anchor anchor, Renderable renderable) {
        AnchorNode anchorNode = new AnchorNode(anchor);
        TransformableNode node = new TransformableNode(arFragment.getTransformationSystem());
        node.setRenderable(renderable);
        node.setParent(anchorNode);
        arFragment.getArSceneView().getScene().addChild(anchorNode);
        node.select();
    }

}

The Full Source Code Can be found to This Github link

Happy Learning And more credits to (Ayusch)