结合：[Artoolkit] ARToolKit's SDK Structure on Android
重难点：aRBaseLib/, nftSimple/, libcpufeatures.a
![[Artoolkit] Android Sample of nftSimple](https://image.shishitao.com:8440/aHR0cHM6Ly9iYnNtYXguaWthZmFuLmNvbS9zdGF0aWMvTDNCeWIzaDVMMmgwZEhCekwybHRZV2RsY3pJd01UVXVZMjVpYkc5bmN5NWpiMjB2WW14dlp5OHpNVEF3TVRVdk1qQXhOekF5THpNeE1EQXhOUzB5TURFM01ESXlNakE0TkRjeE5UZzJOaTB4T1RrMU1UazVOekUxTG5CdVp3PT0uanBn.jpg?w=700&webp=1 "[Artoolkit] Android Sample of nftSimple")
- aRBaseLib/
![[Artoolkit] Android Sample of nftSimple](https://image.shishitao.com:8440/aHR0cHM6Ly9iYnNtYXguaWthZmFuLmNvbS9zdGF0aWMvTDNCeWIzaDVMMmgwZEhCekwybHRZV2RsY3pJd01UVXVZMjVpYkc5bmN5NWpiMjB2WW14dlp5OHpNVEF3TVRVdk1qQXhOekF5THpNeE1EQXhOUzB5TURFM01ESXlNakE1TVRjMU5qSTFOeTB4T0RZeU16ZzFORFEzTG5CdVp3PT0uanBn.jpg?w=700&webp=1 "[Artoolkit] Android Sample of nftSimple")
如果摄像头控制只能在android层操作，那么ARBaseLib中的摄像头控制部分就是重点之一。
unsw@unsw-UX303UB$ _cmd-wc-java
./AndroidUtils.java
./NativeInterface.java
./ARToolKit.java
./camera/CaptureCameraPreview.java
./camera/CameraPreferencesActivity.java
./camera/CameraWrapper.java
./camera/CameraEventListener.java
./rendering/gles20/ARRendererGLES20.java
./rendering/gles20/LineGLES20.java
./rendering/gles20/BaseFragmentShader.java
./rendering/gles20/ShaderProgram.java
./rendering/gles20/BaseShaderProgram.java
./rendering/gles20/CubeGLES20.java
./rendering/gles20/OpenGLShader.java
./rendering/gles20/ARDrawableOpenGLES20.java
./rendering/gles20/BaseVertexShader.java
./rendering/Cube.java
./rendering/ARRenderer.java
./rendering/RenderUtils.java
./rendering/Line.java
./assets/AssetFileTransferException.java
./assets/HashComputationException.java
./assets/AssetFileTransfer.java
./assets/Hasher.java
./assets/AssetHelper.java
./FPSCounter.java
./ARActivity.java
total
unsw@unsw-UX303UB$
4000行代码，从摄像头控制开始。(Replaced with opencv4android)
01. rendering.ARActivity是main()，其中：
CameraEventListener接口可替换为opencv4android提供的CameraBridgeViewBase.CvCameraViewListener2接口。
02. 处理每一帧
@Override
public void cameraPreviewFrame(byte[] frame) { if (firstUpdate) {
// ARToolKit has been initialised. The renderer can now add markers, etc...
if (renderer.configureARScene()) {
Log.i(TAG, "Scene configured successfully");
} else {
// Error
Log.e(TAG, "Error configuring scene. Cannot continue.");
finish();
}
firstUpdate = false;
} if (ARToolKit.getInstance().convertAndDetect(frame)) { // Update the renderer as the frame has changed
if (glView != null) glView.requestRender(); onFrameProcessed();
} }
03. ARToolKit处理帧，然后调用native（NDK）
public boolean convertAndDetect(byte[] frame) {
if (!initedNative) return false;
if (frame == null) return false;
if (!NativeInterface.arwAcceptVideoImage(frame, frameWidth, frameHeight, cameraIndex, cameraIsFrontFacing)) return false;
if (!NativeInterface.arwCapture()) return false;
return NativeInterface.arwUpdateAR();
}
libARWrapper.so的native代码在哪里？
nftSimple没有使用Wrapper技术，而是直接调用native: "AR and Rendering Code in Native C/C++ Using Android NDK"
怎么感觉 aRBaseLib 没有大用 ?
@Override
public void onPreviewFrame(byte[] data, Camera cam) { nftSimpleActivity.nativeVideoFrame(data);
cam.addCallbackBuffer(data);
}
04. 一帧的处理
JNIEXPORT void JNICALL JNIFUNCTION_NATIVE(nativeVideoFrame(JNIEnv* env, jobject obj, jbyteArray pinArray))
{
int i, j, k;
jbyte* inArray; if (!videoInited) {
#ifdef DEBUG
LOGD("nativeVideoFrame !VIDEO\n");
#endif
return; // No point in trying to track until video is inited.
}
if (!nftDataLoaded) {
if (!nftDataLoadingThreadHandle || threadGetStatus(nftDataLoadingThreadHandle) < ) {
#ifdef DEBUG
LOGD("nativeVideoFrame !NFTDATA\n");
#endif
return;
} else {
nftDataLoaded = true;
threadWaitQuit(nftDataLoadingThreadHandle);
threadFree(&nftDataLoadingThreadHandle); // Clean up.
}
}
if (!gARViewInited) {
return; // Also, we won't track until the ARView has been inited.
#ifdef DEBUG
LOGD("nativeVideoFrame !ARVIEW\n");
#endif
}
#ifdef DEBUG
LOGD("nativeVideoFrame\n");
#endif // Copy the incoming YUV420 image in pinArray.
env->GetByteArrayRegion(pinArray, , gVideoFrameSize, (jbyte *)gVideoFrame); // As of ARToolKit v5.0, NV21 format video frames are handled natively,
// and no longer require colour conversion to RGBA.
// If you still require RGBA format information from the video,
// here is where you'd do the conversion:
// color_convert_common(gVideoFrame, gVideoFrame + videoWidth*videoHeight, videoWidth, videoHeight, myRGBABuffer); videoFrameNeedsPixelBufferDataUpload = true; // Note that buffer needs uploading. (Upload must be done on OpenGL context's thread.) // Run marker detection on frame
if (trackingThreadHandle) {
// Perform NFT tracking.
float err;
int ret;
int pageNo;
// 又到了熟悉的地方：）
if( detectedPage == - ) {
trackingInitStart( trackingThreadHandle, gVideoFrame );
detectedPage = -;
}
if( detectedPage == - ) {
ret = trackingInitGetResult( trackingThreadHandle, trackingTrans, &pageNo);
if( ret == ) {
if (pageNo >= && pageNo < surfaceSetCount) {
#ifdef DEBUG
LOGE("Detected page %d.\n", pageNo);
#endif
detectedPage = pageNo;
ar2SetInitTrans(surfaceSet[detectedPage], trackingTrans);
} else {
LOGE("Detected bad page %d.\n", pageNo);
detectedPage = -;
}
} else if( ret < ) {
#ifdef DEBUG
LOGE("No page detected.\n");
#endif
detectedPage = -;
}
}
if( detectedPage >= && detectedPage < surfaceSetCount) {
if( ar2Tracking(ar2Handle, surfaceSet[detectedPage], gVideoFrame, trackingTrans, &err) < ) {
#ifdef DEBUG
LOGE("Tracking lost.\n");
#endif
detectedPage = -;
} else {
#ifdef DEBUG
LOGE("Tracked page %d (max %d).\n", detectedPage, surfaceSetCount - );
#endif
}
}
} else {
LOGE("Error: trackingThreadHandle\n");
detectedPage = -;
} // Update markers.
94 for (i = ; i < markersNFTCount; i++) {
markersNFT[i].validPrev = markersNFT[i].valid;
if (markersNFT[i].pageNo >= && markersNFT[i].pageNo == detectedPage) {
markersNFT[i].valid = TRUE;
for (j = ; j < ; j++) for (k = ; k < ; k++) markersNFT[i].trans[j][k] = trackingTrans[j][k];
}
else markersNFT[i].valid = FALSE;
if (markersNFT[i].valid) { // Filter the pose estimate.
if (markersNFT[i].ftmi) {
if (arFilterTransMat(markersNFT[i].ftmi, markersNFT[i].trans, !markersNFT[i].validPrev) < ) {
LOGE("arFilterTransMat error with marker %d.\n", i);
}
} if (!markersNFT[i].validPrev) {
// Marker has become visible, tell any dependent objects.
//ARMarkerAppearedNotification
} // We have a new pose, so set that.
arglCameraViewRHf(markersNFT[i].trans, markersNFT[i].pose.T, 1.0f /*VIEW_SCALEFACTOR*/);
// Tell any dependent objects about the update.
//ARMarkerUpdatedPoseNotification } else { if (markersNFT[i].validPrev) {
// Marker has ceased to be visible, tell any dependent objects.
//ARMarkerDisappearedNotification
}
}
127 }
}
由原来的 mainloop 变为了android preview 一帧来调用，其他native部分都相同。
- libcpufeatures.a
Link: https://developer.android.com/ndk/guides/cpu-features.html
Android 官方提供的读取cpu信息的一个库。我们这里主要对应于线程相关的部分。
Android Studio 2.2.3直接加载Android example出现的问题：
|
@PN1019, this is a tough nut. Let me try summarize and hopefully that'll help you. So there's two ways to get around this. Use the JDK that is embedded with Android Studio 2.2.+. Or, tell Android Studio to use the Oracle Corp JDK installed on your devbox but have it run in such a way to regress in compiling an older version of the Java language creating an older version of runtime Java classes. I recommend the former, please see my entry above now labeled with "Recommended JDK to use:". |
![[Artoolkit] Android Sample of nftSimple](https://image.shishitao.com:8440/aHR0cHM6Ly9iYnNtYXguaWthZmFuLmNvbS9zdGF0aWMvTDNCeWIzaDVMMmgwZEhCekwybHRZV2RsY3pJd01UVXVZMjVpYkc5bmN5NWpiMjB2WW14dlp5OHpNVEF3TVRVdk1qQXhOekF5THpNeE1EQXhOUzB5TURFM01ESXlNekUwTXpjek56UXlPUzB4Tmpnek9UWTVOelF3TG5CdVp3PT0uanBn.jpg?w=700&webp=1 "[Artoolkit] Android Sample of nftSimple")
and Android Studio 2.1.1 is better.
unsw@unsw-UX303UB$ sudo update-alternatives --config javac
There are choices for the alternative javac (providing /usr/bin/javac). Selection Path Priority Status
------------------------------------------------------------
/usr/lib/jvm/java--oracle/bin/javac auto mode
* /usr/lib/jvm/java--oracle/bin/javac manual mode
/usr/lib/jvm/java--openjdk-amd64/bin/javac manual mode
/usr/lib/jvm/java--oracle/bin/javac manual mode Press enter to keep the current choice[*], or type selection number: unsw@unsw-UX303UB$ sudo update-alternatives --config java
There are choices for the alternative java (providing /usr/bin/java). Selection Path Priority Status
------------------------------------------------------------
/usr/lib/jvm/java--oracle/jre/bin/java auto mode
* /usr/lib/jvm/java--openjdk-amd64/jre/bin/java manual mode
/usr/lib/jvm/java--oracle/jre/bin/java manual mode
/usr/lib/jvm/java--openjdk-amd64/jre/bin/java manual mode
/usr/lib/jvm/java--oracle/jre/bin/java manual mode Press enter to keep the current choice[*], or type selection number: unsw@unsw-UX303UB$ javac -version
javac 1.7.0_80 unsw@unsw-UX303UB$ java -version
java version "1.7.0_121"
OpenJDK Runtime Environment (IcedTea 2.6.) (7u121-2.6.-1ubuntu0.14.04.)
OpenJDK -Bit Server VM (build 24.121-b00, mixed mode)
ubuntu14.14, jdk 1.7.x
Next: how to implement multi-nft，start with linux verison.