Google Research has released findings on PhotoScan that useDeep Learningto assess body composition from smartphone camera images for measuring body fat percentage, but it is still only a prototype research project.
Google Research published the PhotoScan study on 17 August, an experimental Deep Learning approach to evaluate body composition from two-dimensional smartphone photos.
Google stated that experiments showed accuracy close to DXA scanning in predicting insulin resistance and provided more precise body fat values than BIA sensors onsmartwatches.
The PhotoScan system outputs three values: body fat percentage, trunk-to-hip fat ratio, and visceral-to-subcutaneous fat ratio, requiring users to take both front and side paired photos.
Google's research team initially trained a ResNet-50 neural network model with data from 35,323 UK Biobank participants, using 2D projections from 3D MRI images and DXA scans as references, then fine-tuned it with PhotoBIA, a smartphone photo dataset from 677 volunteers, and tested it on an independent group called MetabolicMosaic, consisting of 132 participants from a 30-week longitudinal study in San Francisco.
Testing on the PhotoBIA group showed PhotoScan had an average absolute error for body fat percentage of 2.15, while a BIA sensor model had 2.91. The A/G and V/S ratios were 0.107 and 0.094 respectively. In the MetabolicMosaic group, similar values of 2.13, 0.085, and 0.085 were observed.
For insulin resistance classification, a baseline model using only demographic data such as age, sex, and BMI achieved an AUROC of 0.692. Adding PhotoScan data raised this to 0.760 with an NRI of 0.593. DXA data increased values further to 0.773 and 0.748 respectively. Adding BIA sensor data did not improve results since it only provides total body fat percentage, which is less predictive than A/G and V/S ratios.
This research builds on Google's concept of integrating it into wearable devices and health features, including Insulin Resistance Trends that estimate insulin resistance from heart rate, sleep, and movement data without blood tests. This feature is scheduled for phased release in September 2026 and will support Pixel Watch 3, Pixel Watch 4, Pixel Watch 5, and Fitbit Air.
Competitor Samsung continues to use BIA sensors for body composition on the wrist. Samsung’s Galaxy Watch Ultra2, launched on 22 July, uses the BioActive sensor which includes BIA technology. This method sends a weak electrical current through the body and estimates proportions based on the speed of current passing through fat, muscle, and bone.
Google concluded that while DXA scanning remains the most accurate, it is difficult to scale broadly. Wearable BIA sensors are more convenient but limited to basic body fat percentage. PhotoScan offers a promising middle ground.
The research team emphasized that these results demonstrate technical feasibility only, and the next step is to integrate multiple data types including continuous wearable data, glucose levels, and blood biomarkers.
However, this Google research remains only a prototype study.
Source:TechRadar