1.1.5 bump blocker notes

[charlieforward9]

Building an app is hard. Especially when youre the only one doing it. Especially when half of it is founded in somebody elses' open source code. And especially when you run into problems that are nested deep within a flow of your application, requiring building and live testing the app on your device.

After finishing a huge chunk of work on the CTSIB config, I had a bug that had me inspect our dependencies. Since it had been 8 months since I last touched them, I decided to bump our dependencies to their latest versions and remove anything that was not necessary, in an attempt to resolve the problem and clean the codebase, two birds one stone:

Before: ✓ Built build/ios/iphoneos/Runner.app (141.5MB)
After: ✓ Built build/ios/iphoneos/Runner.app (75.4MB)

However, what I thought was a short bug sweep, ended up being a 3 day bug hunt. Don't update all the dependencies at the same time... These are the notes:

Fixing the version bump

After bumping all our dependencies to their latest versions, one stuck out like a sore thumb... flutter_tflite_helper. This repository was not maintained for the past 3 years. While its parent library flutter_tflite was adopted by Tensorflow, the helper library lagged behind further and further. I did not want to bump because of this, but then - with my now updated version of Flutter/Dart (3.24.3/3.5+), bugs that made it hard to progress. Updating flutter_tflite, forced bumps of some flutter_tflite_helper libraries, and this led to some source code that needed attention.

After all was cleaned up and re-versioned, I launched my code and tried running some image detections, but faced a video stream with no pose overlays. After tedious step debugging, I found that the source of the problem is immediately before sending the camera image to be inferred upon.

TensorImage preProcessData(CameraImage input) {
  final image = ImageUtils.convertCameraImage(input);

  var tensorImage = TensorImage(TfLiteType.float32);
  tensorImage.loadImage(image);
  final imageProcessor = ImageProcessorBuilder()
      .add(ResizeOp(256, 256, ResizeMethod.BILINEAR))
      .add(NormalizeOp(0, 255))
      .build();

  return imageProcessor.process(tensorImage);
}

Something in here changed and broke the pipeline. The rest of this section will cover this debugging process

Related bugs:

• Bad state: failed precondition tensorflow/flutter-tflite#138

Dependency Name	Functional Version	Non Functional Version
tflite_flutter	^0.9.0	^0.11.0
tflite_flutter_helper	url: https://github.com/pnyompen/tflite_flutter_helper.git ref: 43e87d4b9627539266dc20250beb35bf36320dce	url: https://github.com/VisualPT/tflite_flutter_helper.git ref: 949a6f8ca0c8b3b686864727c88cb1c58f80dff0

TLDR; A dependency changed something feeding the flutter_tflite_helper library or I incorrectly updated the library, which now breaks the ImageProcessorBuilder operations...

Step Debug flow

We receive CameraImage in this shape:

Converting to a Flutter-friendly RGBA Image object:

Which is then processed by the tflite_flutter_helper class into a TensorImage that looks like this:

And that shows an overlay on the screen.

I wanted to try to resolve my most recent changes, with the entire config up to the latest stable versions... but velocity is low right now, so I prioritized shipping something that worked over an optimized code base. I went back to the last commit that worked, before I messed with the inference module at all.

From here, I was still getting issues with the UnmodifiableUint8ListView:

• Breaking Change Request: Make typed data classes sealed platform types. dart-lang/sdk#45115
• build issue in dart 3.5.0 - UnmodifiableUint8ListView jonataslaw/get_cli#263.

I needed to downgrade Flutter from 3.24.3 to 3.22.0, to get the right Dart SDK (<3.5?) that hadn't yet deprecated this class.

Since the downgrade worked, I will be shelving further upgrading of the inference module until we NEED to update or another teammate own the module. Use these notes as a starting point for determining how to recover the flutter_tflite_helper class, OR for building the next generation of it.

For a functional application, see 1.1.5!

[charlieforward9]

For now, the unfinished work rests here:

#223