Automate receipt processing with Encodian Flowr and AI

Power App UI

Gallery View

The home screen of the Power App contains a gallery that displays all the expenses you’ve uploaded. The gallery is filtered using the current user’s Entra ID, ensuring that users can only see their own receipts.

Filter(Reciepts, 'Created By'.'Azure AD Object ID' = User().EntraObjectId)

Upload Interface

The ‘Add Receipt’ button navigates to the Upload Receipt page. Here, users can either upload an image of their receipt or take a photo of it. Each button makes a different container visible with the respective controls inside the container:

• varTakeImage – makes the take image container visible when set to true
• varUploadImage – makes the upload image container visible when set to true

For full details on how the upload image and take image controls work, please look at the instructions on one of our previous blogs below.

Once a receipt has been uploaded/taken, the Save button becomes enabled. When pressed, it creates a new row in the Receipts table and adds the uploaded/taken image to the ‘Receipt Image’ column.

Patch(Reciepts,Defaults(Reciepts),{'Receipt Image': UploadedImage1.Image});
Reset(AddMediaButton1);
Navigate(homeScreen)

If(IsBlank(AddMediaButton1.Media), DisplayMode.Disabled, DisplayMode.Edit)

After saving the receipt, once the data is refreshed it will appear in the gallery. Initially, the gallery will only display the receipt number. This is because the Power Automate flow that is extracting the data from the receipt is still working in the backend. Once the flow has finished, the extracted data will be visible in the app.

Viewing Extracted Data

If we open up the added receipt, we can see a Power Apps form that contains all the extracted data. The form is connected to the Receipts Dataverse table. Because different receipts contain different data, any fields from the table that weren’t populated are hidden from the form to maximise the space. The following formula is being used on each form DataCard visibility property (the DataCardValue must correspond to the DataCard this formula is being added too).

If(IsBlank(DataCardValue7.Value), false, true)

The table control uses the Receipt Items table as its datasource. The data is filtered for the current selected receipt.

Filter('Receipt Items', Receipt.Reciepts = receiptGallery.Selected.Reciepts)

If(varShowReciept, Set(varShowReciept, false), Set(varShowReciept, true))

If(varShowReciept,"Close Receipt", "View Receipt")

Because the image is being saved to Dataverse, we need to access the full size image, otherwise it will display blurry. We can do this by using this formula in the Image control:

$"https://[environmentURL]/Image/download.aspx?Entity=[entityLogicalName]&Attribute=[imageColumnLogicalName]&Id={receiptGallery.Selected.Reciepts}&Full=true"

$"https://org67b56a92.crm11.dynamics.com/Image/download.aspx?Entity=en_reciepts&Attribute=en_receiptimage&Id={receiptGallery.Selected.Reciepts}&Full=true"

Power Automate Flow

1. Trigger – When a row is added to a Dataverse table

The flow is triggered when a new row is added to the Receipts table in Dataverse. Using the Patch function, this happens when the Power App saves the receipt image.

2. Get full image content

To ensure high-quality AI processing, the flow downloads the full-size image of the receipt.

3. Use AI – Process Receipt

Use Flowr’s AI – Process Receipt action, passing the image content as input.

4. Parse JSON Output

The outputs of this action are in a string format. We can use a ‘Parse JSON’ action so that we can easily use the extracted data. The JSON schema being used includes all the possible data fields that can be extracted from the receipt. However, there is only one required field. This ensures the process is dynamic, so if data is missing from one receipt the action won’t fail.

{
    "type": "object",
    "properties": {
        "Items": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "Description": {
                        "type": "string"
                    },
                    "Quantity": {
                        "type": "string"
                    },
                    "TotalPrice": {
                        "type": "string"
                    }
                },
                "required": [
                    "Description"
                ]
            }
        },
        "MerchantName": {
            "type": "string"
        },
        "MerchantAddress": {
            "type": "string"
        },
        "MerchantPhoneNumber": {
            "type": "string"
        },
        "Subtotal": {
            "type": "string"
        },
        "Tip": {
            "type": "string"
        },
        "Total": {
            "type": "string"
        },
        "TotalTax": {
            "type": "string"
        },
        "TransactionDate": {
            "type": "string"
        },
        "TransactionTime": {
            "type": "string"
        }
    }
}

5. Update Receipt Record in Dataverse

Now we can add the extracted data to Dataverse. We will update the Dataverse row in the Receipts table that triggered the flow. To keep the flow dynamic, we can use the coalesce function. This function allows you to substitute a different value if the one provided is null/blank. So, if one of the fields in the table hasn’t been extracted from the receipt, we will be setting the value to null.

coalesce(body('Parse_JSON')?['MerchantAddress'],null)

6. Add Line Items

Loop through the Items array using Apply to Each and add a new row to the Receipt Items table for each detected entry.