---
title: Extract data from Image, using Claude 3.5 Sonnet
url: https://calvin.my/posts/extract-data-from-image-using-claude-3-5-sonnet
published: 2024-06-26
updated: 2026-09-17
category: AI
tags:
- Anthropic
- Claude
- Sonnet
- Image
- Ruby
summary: The post tests Claude 3.5 Sonnet’s ability to extract final receipt totals from images with varied layouts, languages, image quality, and capture methods. Using a simple prompt through Anthropic’s Messages API, the model correctly identified totals in three examples, including dense thermal and partially malformed receipts. Cropping whitespace offered little benefit, but resizing images to about 500 pixels reduced input tokens below 1,000 while maintaining success in these tests.
---

# Extract data from Image, using Claude 3.5 Sonnet

## What do I want to do?

In this demo, the objective is to extract the total amount (after discount, and after taxes) from a receipt image.

Some characteristics of these receipts:

- There isn't a fixed format
- It can be in different languages (English, S. Chinese, T. Chinese, Malay)
- It can have more than one language
- It can be an image from a camera snapshot, or an electronic file sent via email

* * *

## How do I plan to do it?

I am using the [Anthropic message API](https://docs.anthropic.com/en/api/messages) and Claude-3-5-sonnet-20240620 model, with a simple system prompt below:

_Extract the total amount from the image. It should be a number, e.g. 100.50, usually next to the word 'total', 'total amount', 'grand total'. It can be in any languages. The currency symbol is RM_

* * *

## Test Results

Test #1: An A4 size receipt, itemized in table format.

Result: ✅ Success

![](https://camy-pub.s3.ap-southeast-1.amazonaws.com/bc3be7f3-c0bf-4c17-8c26-28f143df60a0.png)

Test #2: A 58mm thermal receipt paper, captured by phone camera. The text density is very high, font size is relatively small.

Result: ✅ Success

![](https://camy-pub.s3.ap-southeast-1.amazonaws.com/bb67abb8-68a9-4cd0-aa8a-b36853964545.png)

Test #3 - A landscape A4 paper, captured by a phone camera, some part of the image is malformed.

Result: ✅ Success

![](https://camy-pub.s3.ap-southeast-1.amazonaws.com/d91ca703-2972-4383-b3c5-1ae21aca05ed.png)

It seems the total amount is successfully identified in all 3 tests. Good job Claude 3.5!

* * *

## Cost? And some improvements

The input tokens spent per attempt is about 1500-2000, which is probably equivalent to $0.005. Can we reduce it?

I tried to trim the white space around the image, but they are not useful. And I also resize the image to around 500px before sending it into the API. 

The input token is kept below 1000 and the success rate is still 100%. 

![](https://camy-pub.s3.ap-southeast-1.amazonaws.com/060b09f9-3d5a-420c-a437-f5e2063869db.png)

That's pretty awesome. :)