如何解决 MlKitException: Internal error (Caused by: java.lang.IllegalStateException: Image is already closed)

How to resolve MlKitException: Internal error (Caused by: java.lang.IllegalStateException: Image is already closed)

如何解决 MlKitException: 内部错误?

我正在尝试在 Camerax ImageAnalysis 中使用 TextRecognizer。当我调用 TextAnalyzer.process() 时,它因 ml-kit 内部错误而失败。在打印堆栈框架时,我看到“图像已关闭”异常。我确保在 addOnCompleteListener 中调用 imageProxy.close()。我看过其他 Whosebug 问题。但是运气不好!

更新:

日志

2021-07-14 01:37:13.133 8887-8963/com.example.finder D/StreamStateObserver: Update Preview stream state to STREAMING
2021-07-14 01:37:13.138 8887-8991/com.example.finder D/yoyo: recognizeText: ImmutableImageInfo{tagBundle=androidx.camera.core.impl.TagBundle@fde8840, timestamp=709744567075165, rotationDegrees=90}
2021-07-14 01:37:13.138 8887-8991/com.example.finder D/yoyo: analyze: [Landroidx.camera.core.AndroidImageProxy$PlaneProxy;@c3b379
2021-07-14 01:37:13.151 1694-2205/? I/WindowManager: Window drawn AppWindowToken{6106e1d token=Token{efcb6f4 ActivityRecord{71d0c7 u0 com.example.finder/.MainActivity t135}}}
2021-07-14 01:37:13.155 8887-8992/com.example.finder I/native: I0714 01:37:13.152209    8992 start_ocr_856638336_langid.cc:35] StartOcr with label: latin_script_gmvapi_android
2021-07-14 01:37:13.155 8887-8992/com.example.finder I/native: I0714 01:37:13.155370    8992 jni_helper.cc:122] Loading model config /data/user/0/com.example.finder/cache/vision/ocr/data/models/semanticlift_rpn_lstm_engine_tfmini.bincfg
2021-07-14 01:37:13.156 8887-8992/com.example.finder I/native: I0714 01:37:13.156842    8992 common_util.h:35] Resizing Thread Pool: ocr_det_0 to 4
2021-07-14 01:37:13.157 8887-8992/com.example.finder I/native: I0714 01:37:13.157155    8992 tfmini_model_interface.cc:127] Loaded weights from /data/user/0/com.example.finder/cache/vision/ocr/data/models/tfmini_rpn_detector_space_to_depth_gray_quantized_v2_model.data
2021-07-14 01:37:13.157 8887-8992/com.example.finder I/native: I0714 01:37:13.157252    8992 timer.cc:71] PhotoOcrEngine::Init (detector): 1.06214 ms (elapsed)
2021-07-14 01:37:13.157 8887-8992/com.example.finder I/native: I0714 01:37:13.157447    8992 assist_recognizer.cc:96] Initializing nugget matcher with /data/user/0/com.example.finder/cache/vision/ocr/data/models/nugget_stats_1.0.dat
2021-07-14 01:37:13.157 8887-8992/com.example.finder I/native: I0714 01:37:13.157550    8992 timer.cc:71] PhotoOcrEngine::Init (assist recognizer): 0.174948 ms (elapsed)
2021-07-14 01:37:13.159 8887-8992/com.example.finder I/native: I0714 01:37:13.158957    8992 timer.cc:71] LanguageIdMutator: 1.07484 ms (elapsed)
2021-07-14 01:37:13.159 8887-8992/com.example.finder I/native: I0714 01:37:13.159068    8992 timer.cc:71] BeamSearch::Init (mutator): 1.1963 ms (elapsed)
2021-07-14 01:37:13.160 8887-8992/com.example.finder I/native: I0714 01:37:13.160904    8992 common_util.h:35] Resizing Thread Pool: ocr_segm to 4
2021-07-14 01:37:13.161 8887-8992/com.example.finder I/native: I0714 01:37:13.161382    8992 tfmini_model_interface.cc:105] Loaded fp16 weights from /data/user/0/com.example.finder/cache/vision/ocr/data/models/tfmini_latin_conv_model.data_fp16
2021-07-14 01:37:13.164 8887-8992/com.example.finder I/native: I0714 01:37:13.164457    8992 tfmini_model_interface.cc:105] Loaded fp16 weights from /data/user/0/com.example.finder/cache/vision/ocr/data/models/tfmini_latin_lstm_model.data_fp16
2021-07-14 01:37:13.165 8887-8992/com.example.finder I/native: I0714 01:37:13.165009    8992 timer.cc:71] PhotoOcrEngine::Init (recognizer): 7.40287 ms (elapsed)
2021-07-14 01:37:13.165 8887-8992/com.example.finder I/native: I0714 01:37:13.165483    8992 timer.cc:71] Init: 9.38745 ms (elapsed)
2021-07-14 01:37:13.166 8887-8992/com.example.finder I/native: I0714 01:37:13.165557    8992 start_ocr_856638336_langid.cc:44] StartOcr success
2021-07-14 01:37:13.169 8887-8887/com.example.finder D/yoyo: onFailure: [Landroidx.camera.core.AndroidImageProxy$PlaneProxy;@c3b379
2021-07-14 01:37:13.170 8887-8887/com.example.finder D/yoyo: onFailure: Stack 1
    com.google.mlkit.common.MlKitException: Internal error has occurred when executing ML Kit tasks
        at com.google.mlkit.common.sdkinternal.ModelResource.zza(Unknown Source:64)
        at com.google.mlkit.common.sdkinternal.zzn.run(Unknown Source:10)
        at com.google.mlkit.common.sdkinternal.zzp.run(com.google.mlkit:common@@17.1.1:2)
        at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zze(com.google.mlkit:common@@17.1.1:4)
        at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzc(Unknown Source:8)
        at com.google.mlkit.common.sdkinternal.zzj.run(Unknown Source:2)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzd(Unknown Source:10)
        at com.google.mlkit.common.sdkinternal.zzk.run(Unknown Source:2)
        at java.lang.Thread.run(Thread.java:919)
     Caused by: java.lang.IllegalStateException: Image is already closed
        at android.media.Image.throwISEIfImageIsInvalid(Image.java:72)
        at android.media.ImageReader$SurfaceImage.getPlanes(ImageReader.java:895)
        at kk.b(:com.google.android.gms.optional_vision_ocr@212418065@21.24.18 (100400-0):14)
        at ni.a(:com.google.android.gms.optional_vision_ocr@212418065@21.24.18 (100400-0):4)
        at aq.onTransact(:com.google.android.gms.optional_vision_ocr@212418065@21.24.18 (100400-0):4)
        at android.os.Binder.transact(Binder.java:923)
        at com.google.android.gms.internal.mlkit_vision_text.zza.zzb(com.google.android.gms:play-services-mlkit-text-recognition@@16.3.0:2)
        at com.google.android.gms.internal.mlkit_vision_text.zzlm.zzd(com.google.android.gms:play-services-mlkit-text-recognition@@16.3.0:4)
        at com.google.mlkit.vision.text.internal.zzb.zza(com.google.android.gms:play-services-mlkit-text-recognition@@16.3.0:13)
        at com.google.mlkit.vision.text.internal.zzo.zzc(com.google.android.gms:play-services-mlkit-text-recognition@@16.3.0:2)
        at com.google.mlkit.vision.text.internal.zzo.run(com.google.android.gms:play-services-mlkit-text-recognition@@16.3.0:1)
        at com.google.mlkit.vision.common.internal.MobileVisionBase.zza(com.google.mlkit:vision-common@@16.5.0:2)
        at com.google.mlkit.vision.common.internal.zzd.call(Unknown Source:4)
        at com.google.mlkit.common.sdkinternal.ModelResource.zza(Unknown Source:37)
        at com.google.mlkit.common.sdkinternal.zzn.run(Unknown Source:10) 
        at com.google.mlkit.common.sdkinternal.zzp.run(com.google.mlkit:common@@17.1.1:2) 
        at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zze(com.google.mlkit:common@@17.1.1:4) 
        at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzc(Unknown Source:8) 
        at com.google.mlkit.common.sdkinternal.zzj.run(Unknown Source:2) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 
        at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzd(Unknown Source:10) 
        at com.google.mlkit.common.sdkinternal.zzk.run(Unknown Source:2) 
        at java.lang.Thread.run(Thread.java:919) 
2021-07-14 01:37:13.171 8887-8887/com.example.finder D/yoyo: onComplete: close()1
2021-07-14 01:37:13.183 8887-8955/com.example.finder D/TransportRuntime.SQLiteEventStore: Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
2021-07-14 01:37:13.194 8887-8955/com.example.finder D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
2021-07-14 01:37:13.197 8887-8887/com.example.finder D/yoyo: onFailure: [Landroidx.camera.core.AndroidImageProxy$PlaneProxy;@cc77490
2021-07-14 01:37:13.199 8887-8887/com.example.finder D/yoyo: onFailure: Stack 2
    com.google.mlkit.common.MlKitException: Internal error has occurred when executing ML Kit tasks
        at com.google.mlkit.common.sdkinternal.ModelResource.zza(Unknown Source:64)

CameraPreviewActivity:

public class CameraPreviewActivity extends AppCompatActivity {

    private static final String[] REQUIRED_PERMISSIONS = new String[] {"android.permission.CAMERA", "android.permission.WRITE_EXTERNAL_STORAGE"};
    private static final int REQUEST_CODE_PERMISSIONS = 10;
    private ListenableFuture<ProcessCameraProvider> cameraProviderFuture;
    private final Executor executor = Executors.newSingleThreadExecutor();
    private String message;
    private int count = 0;
    private TextRecognizer recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS);

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        Objects.requireNonNull(getSupportActionBar()).hide();
        setContentView(R.layout.activity_camera_preview);
        getLifecycle().addObserver(recognizer);
        // Get the Intent that started this activity and extract the string
        Intent intent = getIntent();
        message = intent.getStringExtra(MainActivity.EXTRA_MESSAGE);
        Log.d("yoyo", "onCreate: " + message);
        if (allPermissionGranted()) {
            startCamera();
        } else {
            ActivityCompat.requestPermissions(this, REQUIRED_PERMISSIONS, REQUEST_CODE_PERMISSIONS);
        }
    }

    private boolean allPermissionGranted() {
        for(String permission: REQUIRED_PERMISSIONS) {
            if (ActivityCompat.checkSelfPermission(this, permission) != PackageManager.PERMISSION_GRANTED) {
                return false;
            }
        }
        return true;
    }

    @Override
    public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults);
        if (requestCode == REQUEST_CODE_PERMISSIONS) {
            if (allPermissionGranted()) {
                startCamera();
            } else {
                Toast.makeText(this, "Permissions not granted by the user.", Toast.LENGTH_SHORT).show();
                this.finish();
            }
        }
    }

    private void startCamera() {
        cameraProviderFuture = ProcessCameraProvider.getInstance(this);
        cameraProviderFuture.addListener(new Runnable() {
            @Override
            public void run() {
                try {
                    ProcessCameraProvider cameraProvider = cameraProviderFuture.get();
                    bindPreview(cameraProvider);
                } catch (ExecutionException | InterruptedException e) {
                    // No errors need to be handled for this Future.
                    // This should never be reached.
                }
            }
        }, ContextCompat.getMainExecutor(this));
    }

    private void bindPreview(ProcessCameraProvider cameraProvider) {
        Preview preview = new Preview.Builder()
                .build();

        CameraSelector cameraSelector = new CameraSelector.Builder()
                .requireLensFacing(CameraSelector.LENS_FACING_BACK)
                .build();

        PreviewView cameraPreviewView = findViewById(R.id.cameraPreviewView);
        preview.setSurfaceProvider(cameraPreviewView.getSurfaceProvider());
        ImageAnalysis imageAnalysis =
                new ImageAnalysis.Builder()
                        .setTargetResolution(new Size(1280, 720))
                        .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
                        .build();
        imageAnalysis.setAnalyzer(executor, new TextAnalyzer());
        Camera camera = cameraProvider.bindToLifecycle(this, cameraSelector, preview, imageAnalysis);
    }

    public void goBack(View view) {
        this.finish();
    }

    public void showToast(String message) {
        Toast.makeText(this, message, Toast.LENGTH_SHORT).show();
    }

    private class TextAnalyzer implements ImageAnalysis.Analyzer {

        @Override
        public void analyze(@NonNull ImageProxy imageProxy) {
            try (@SuppressLint("UnsafeOptInUsageError") Image mediaImage = imageProxy.getImage()) {
                if (mediaImage != null) {
                    if (count % 5 == 0) {
                        Log.d("yoyo", "recognizeText: " + imageProxy.getImageInfo().toString());
                        Log.d("yoyo", "analyze: "+ imageProxy.getPlanes().toString());
                    }
                    InputImage image =
                            InputImage.fromMediaImage(mediaImage, imageProxy.getImageInfo().getRotationDegrees());

                    Task<Text> result =
                            recognizer.process(image)
                                    .addOnSuccessListener(new OnSuccessListener<Text>() {
                                        @Override
                                        public void onSuccess(Text visionText) {
                                            //processTextBlock(visionText);
                                        }
                                    })
                                    .addOnFailureListener(
                                            new OnFailureListener() {
                                                @Override
                                                public void onFailure(@NonNull Exception e) {
                                                    // Task failed with an exception
                                                    if (count % 5 == 0 ||
                                                        count == 1 || count == 0 || count == 2) {
                                                        Log.d("yoyo", "onFailure: " + imageProxy.getPlanes().toString());
                                                        Log.d("yoyo", "onFailure: Stack " + count, e);
                                                        //Log.d("yoyo", "onFailure: Stack");

                                                    }
                                                }
                                            })
                                    .addOnCompleteListener(new OnCompleteListener<Text>() {
                                        @Override
                                        public void onComplete(@NonNull Task<Text> task) {
                                            imageProxy.close();
                                            //mediaImage.close();
                                            if (count % 5 == 0 ||
                                                    count == 1 || count == 0 || count == 2) {
                                                Log.d("yoyo", "onComplete: close()" + count);
                                            }
                                        }
                                    });
                }
                if (count++ > 1000) {
                    count = 0;
                }
            }
        }

        private void processTextBlock(Text visionText) {
            String resultText = visionText.getText().toLowerCase();
            if (count < 30) {
                Log.d("yoyo", "processTextBlock: " + resultText);
            }
            if (resultText.contains(message.toLowerCase())) {
                try {
                    Uri notification = RingtoneManager.getDefaultUri(RingtoneManager.TYPE_NOTIFICATION);
                    Ringtone r = RingtoneManager.getRingtone(getApplicationContext(), notification);
                    r.play();
                } catch (Exception e) {
                    e.printStackTrace();
                }
            } else {
                showToast(resultText);
            }

            for (Text.TextBlock block : visionText.getTextBlocks()) {
                String blockText = block.getText();
                Point[] blockCornerPoints = block.getCornerPoints();
                Rect blockFrame = block.getBoundingBox();
                for (Text.Line line : block.getLines()) {
                    String lineText = line.getText();
                    Point[] lineCornerPoints = line.getCornerPoints();
                    Rect lineFrame = line.getBoundingBox();
                    for (Text.Element element : line.getElements()) {
                        String elementText = element.getText();
                        Point[] elementCornerPoints = element.getCornerPoints();
                        Rect elementFrame = element.getBoundingBox();
                    }
                }
            }
        }
    }
}

您似乎试图关闭它两次。

imageProxy.close();
mediaImage.close();

删除第二个并重试。它可能会抛出此异常,因为您已经在第一次调用时关闭了它。

问题在于使用临时变量 Image mediaImage = imageProxy.getImage();

我认为在将此变量作为参数传递给 InputImage.fromMediaImage 后,它超出了范围,导致调用 close()。

所以通过 imageProxy.getImage() 直接解决了问题。