Can I render 2D images or GIF images on surface using AR CORE scene-form API?

Question

I am trying to recognise one 2D image and once image tracking/recognition is successfully done,

I have to place one 2D image or GIF image on the surface.

I am using scene-form API.

I can render 3D Image and video but my requirement is to render GIF image or 2D image.

Please suggest!!!

code-

Fragment:

package com.example.setupboxscanner;

import com.google.ar.core.Config;
import com.google.ar.core.Session;
import com.google.ar.sceneform.ux.ArFragment;

public class CustomArFragment extends ArFragment {

    @Override
    protected Config getSessionConfiguration(Session session) {
        Config config = new Config(session);
        config.setUpdateMode(Config.UpdateMode.LATEST_CAMERA_IMAGE);
        config.setFocusMode(Config.FocusMode.AUTO);
        session.configure(config);
        this.getArSceneView().setupSession(session);

        ((MainActivity)getActivity()).setupDatabase(config, session);

        return config;
    }

}

MainActivity:

package com.example.setupboxscanner;

import android.annotation.TargetApi;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.net.Uri;
import android.os.Bundle;

import com.google.android.material.floatingactionbutton.FloatingActionButton;
import com.google.android.material.snackbar.Snackbar;
import com.google.ar.core.Anchor;
import com.google.ar.core.AugmentedImage;
import com.google.ar.core.AugmentedImageDatabase;
import com.google.ar.core.Config;
import com.google.ar.core.Frame;
import com.google.ar.core.Session;
import com.google.ar.core.TrackingState;
import com.google.ar.sceneform.AnchorNode;
import com.google.ar.sceneform.FrameTime;
import com.google.ar.sceneform.Scene;
import com.google.ar.sceneform.rendering.ModelRenderable;

import androidx.appcompat.app.AppCompatActivity;
import androidx.appcompat.widget.Toolbar;

import android.util.Log;
import android.view.View;
import android.view.Menu;
import android.view.MenuItem;

import java.util.Collection;

public class MainActivity extends AppCompatActivity  implements Scene.OnUpdateListener {

    private CustomArFragment arFragment;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        arFragment = (CustomArFragment) getSupportFragmentManager().findFragmentById(R.id.fragment);
        arFragment.getArSceneView().getScene().addOnUpdateListener(this);
    }

    public void setupDatabase(Config config, Session session){
       Bitmap setTopBoxImagesFront  = BitmapFactory.decodeResource(getResources(), R.drawable.front);
      //  Bitmap setTopBoxImagesRear  = BitmapFactory.decodeResource(getResources(), R.drawable.rear);
        AugmentedImageDatabase aid = new AugmentedImageDatabase(session);
        aid.addImage("image1", setTopBoxImagesFront);
       // aid.addImage("image2", setTopBoxImagesRear);

        config.setAugmentedImageDatabase(aid);
    }

    @Override
    public void onUpdate(FrameTime frameTime) {

        Frame frame = arFragment.getArSceneView().getArFrame();
        Collection<AugmentedImage> images = frame.getUpdatedTrackables(AugmentedImage.class);

        for (AugmentedImage image : images) {
            if(image.getTrackingState() == TrackingState.TRACKING) {
                if(image.getName().equals("image1")) {
                    Anchor anchor = image.createAnchor(image.getCenterPose());
                    createModel(anchor);
                }
            }
        }
    }

   // @TargetApi(24)
    private void createModel(Anchor anchor) {

        Log.e("-----><>>",""+Uri.parse("//android_assets/front.jpg"));
        ModelRenderable.builder().setSource(this, Uri.parse("model.sfb")).build()
                .thenAccept(modelRenderable -> placeModel(modelRenderable, anchor));
    }

    private void placeModel(ModelRenderable modelRenderable, Anchor anchor) {
        AnchorNode anchorNode = new AnchorNode(anchor);
        anchorNode.setRenderable(modelRenderable);
        arFragment.getArSceneView().getScene().addChild(anchorNode);
    }

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        // Inflate the menu; this adds items to the action bar if it is present.
        getMenuInflater().inflate(R.menu.menu_main, menu);
        return true;
    }

    @Override
    public boolean onOptionsItemSelected(MenuItem item) {
        // Handle action bar item clicks here. The action bar will
        // automatically handle clicks on the Home/Up button, so long
        // as you specify a parent activity in AndroidManifest.xml.
        int id = item.getItemId();

        //noinspection SimplifiableIfStatement
        if (id == R.id.action_settings) {
            return true;
        }

        return super.onOptionsItemSelected(item);
    }


}

Scene-form assets:

sceneform.asset('sampledata/ArcticFox_Posed.obj',
        'default',
        'sampledata/ArcticFox_Posed.sfa',
        'src/main/assets/ArcticFox_Posed')

ActivityMain.xml

<?xml version="1.0" encoding="utf-8"?>
<androidx.coordinatorlayout.widget.CoordinatorLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">
<fragment
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:id="@+id/fragment"
    android:name="com.example.setupboxscanner.CustomArFragment"/>



</androidx.coordinatorlayout.widget.CoordinatorLayout>

If you can render video, then that should be some type of texture(view). Render your 2D onto that surface. Otherwise show code of what you've tried. — Morrison Chang, Dec 11 '19 at 04:52

score 1 · Accepted Answer · answered Mar 24 '21 at 19:12

thanks for your question. I had a similar issue with trying to get a gif and/or 2D image to render via ARCore. Fortunately, there is a very simple way to do that. You can use this gif image view Adding gif image in an ImageView in android to a new XML layout file. Then use the ViewRenderable to render that layout in an ARcore Session after attaching it to a node (can be an anchor node or just a node relative to the camera [I call this a floating node]). You should be able to view the gif and/or image clearly.

In the new XML layout that will be rendered using ViewRenderable I have this view to load the gif:

 <pl.droidsonroids.gif.GifImageView
        android:layout_width="173dp"
        android:layout_height="183dp"
        android:src="@drawable/loading2"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/imageView" />

Can I render 2D images or GIF images on surface using AR CORE scene-form API?

1 Answers1