Skip to content

Conversation

@kfc35
Copy link

@kfc35 kfc35 commented Dec 17, 2025

Objective

Give the user the ability to select which photo under an observation will be sent to the computer vision model, see https://forum.inaturalist.org/t/use-computer-vision-on-each-photo-in-an-observation/4210 , it had one very recent comment about it. The objective is NOT to allow using multiple photos to get some type of combined score, but just the ability to select ONE photo from the observation.

Solution

This extends the API to accept an additional query parameter, photo_id, that will be used to get the specific observation photo to pass on to the computer vision model. If the photo_id provided is not associated with the observation, the request returns a 422.

If this gets merged, I intend to follow up with a pull-request in inaturalist to take the selected photo’s id (keeping in mind flagged photos I guess!) in obs/show and use it to pass on to TaxonAutocomplete as a part of visionParams prop.

Testing

I extended the existing integration test in the v2 section to test calling the endpoint with the new param (there is already an existing test without the photo_id param). I ran computervision specific tests and they were fine.

Alternatives considered

I did think about just switching the api call in inaturalist to use the score_image endpoint, and adding the additional context manually (like lat, long, observed_on… this seems to be what the observation uploader does [Is this what iNat next is doing too?]), but it looks like the score_image endpoint accepts image bytes, and it does not make sense to me to constantly send the image bytes over the api, especially if people might be switching between the photos to check for different cv suggestions. It makes more sense to me to just reference the photo by id and have the server handle photo fetching / context setting in this case.

If iNat expects to support multiple photos to get a combined score in the near term, I do question if this new query param might make things tougher for that but I suppose that’s up for discussion in this PR. Perhaps that will just be a new endpoint if ever implemented!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant