Offer to help

[inakarmacoma]

Apologies if 'New Issue' is not the preferred communication strategy; but wanted to share that I have one of these devices. I came across this repo when trying to figure out how to make sense of the product and its data. In any case, I don't have the technical expertise to crack/decode (but thank you all for trying). If there's anything you think I might be able to contribute, in having another device on hand, please let me know.

[kebasaa]

Perfect, thanks for opening a new issue. At present, I need ideas on how to decode the data (which could even be encrypted). It's 1800 bytes for sample, 1800 for sample dark and 1656 for sample gradient. But we know that the Consumer Physics website returns 331 values. And any known data type is 1, 2, 4 or 8 bytes for each value, so something doesn't add up. Any ideas are welcome at this point

[hbsagen]

I am in the same boat as @inakarmacoma. No knowledge of reverse engineering. But quite curious about the SCIO.

Would taking a batch of samples inside the SCIO housing and comparing the data help? Can we expect the gradient or the dark to be the same? If they are not, and we transform ie the dark with cyberchef to find an operation that generates the same result from different darks with the serial number or timestamp?

[StevenLColeman42]

My suggestion is to do sampling of various pure as possible elements that have a known spectrum and then compare the results of each device/sample. Something commonly available that should have as little variance as possible. A few ideas: Distilled water Brand name products (e.g soap, salt, sugar, etc) ??? We need to control the variance of the samples so that the peak levels can be compared to the known spectra or the spectra of other samples. Sticking to specific brands will hopefully hepl control for this but adding a lot or batch number should be conveyed with the samples if possible. If the peaks all wind up in the right places then we are likely on the right track. The model number in the header should be examined to see if the number of bytes changed at some specific version level. Btw- last I checked my SCiO did not respond to plugging in my USB, so I'm hoping it has not died. (tbd) Steve

…

On Fri, Mar 24, 2023, 2:56 PM hbsagen ***@***.***> wrote: I am in the same boat as @inakarmacoma <https://github.com/inakarmacoma>. No knowledge of reverse engineering. But quite curious about the SCIO. Would taking a batch of samples inside the SCIO housing and comparing the data help? Can we expect the gradient or the dark to be the same? If they are not, and we transform ie the dark with cyberchef to find an operation that generates the same result from different darks with the serial number or timestamp? — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE7QNBOR4UK5VE6F6K5XKALW5XU4PANCNFSM6AAAAAAWG27IZY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[kebasaa]

I would appreciate some help with the data I already provided, as @StevenLColeman42 suggested. We might be able to figure something out like that. Here is the information about it:

1. About connecting the SCiO to a computer: @StevenLColeman42 Does your SCiO charge when connected to USB? It will only be recognised by a computer if it is on, so connect, check if it's charging, then long-press to turn on. Hopefully that will work

2. About measuring known materials: I measured a grey reference plate using both the SCiO and a commercial device (PolyPen, https://handheld.psi.cz/products/polypen/). The Polypen output overlaps the SCiO slightly (by a ca. 50 nm range). The Polypen output is in 01_rawdata/calibration_plate_Polypen.csv. Unfortunately, I seem to have lost the corresponding SCiO output, and I don't have access to the PolyPen any more...

3. I also scanned the same material multiple times, which should result in comparable spectra. If they don't have peaks in the same place, the data is random, and random data might suggest encryption. See 01_rawdata/scan_json/calibrations (the calibration box) and 01_rawdata/scan_json/skin. You can play around with those. I have a feeling that either the device_id (for my scans: 8032AB45611198F1), the serial number (CPPCA0031C6PF0516009W6404386A1DF1816004A), the deviceDspId (e24da26b2304c2c0) or (most likely) the aptinaId (00004c8e3238c8e18032ab45611198f1) could be a decrpytion key. Does one of you have the experience to try and decrypt and play with the data to see if something useful comes out?

4. In the file 01_rawdata/app_researcher_output/SCIO_scans_from_tech_support.csv there is a full output of what customers obtain when using the Consumer Physics online platform. It contains both some calibration (white reference, "wr" and raw data), and might help us figure out what's going on...

[kebasaa]

Actually, regarding point (2) above, I think the grey plate that I scanned with the Polypen has the corresponding SCiO bytes saved in the files starting with file_refl in the folder 01_rawdata/scan_binary_astext/

[kebasaa]

With regards to point (4) above, we should expect 331 values (wavelengths from 740-1070nm, with 12 sensor areas of 27nm range each, see discussion here), with values around 6000-6 in general for the white reference (WR). A simple test using data exported by the Researcher App shows that spectrum = sample / white-reference

[kebasaa]

In the folder 01_rawdata/log_extracted/, you will find the data reported in the SCiO app logs of corresponding data:

1. raw bytes, stored as hex strings, of sample, sample_dark and sample_gradient
2. base64 strings of the same, including the relevant calibration (white reference) base64 strings
3. The resulting reflectance spectrum

This data was extracted from the logs in 01_rawdata/log_files/, using the script 02_extract_log_scan.ipynb. With this data, we may be able to determine how it all gets extracted and converted to reflectance.

[kebasaa]

Based on a discussion in the other issue, the data is most likely encrypted and compressed image data. The question now is how to obtain that. I now provide dark scans, where I blocked the light source and sensor area completely. In theory, this means that if the data is correctly extracted for those, then we should see completely dark images (or at most some sensor noise). The question is how to get there...