---
title: Testing the GPT-4o Web Search model
url: https://calvin.my/posts/testing-the-gpt-4o-web-search-model
published: 2025-04-20
updated: 2026-09-17
category: AI
tags:
- OpenAI
- web search
summary: The post evaluates OpenAI’s GPT-4o web search models using configurable search context and approximate user location. Tests show the models can retrieve current movie releases, recent political events, rebut a false claim about U.S. troops in Ukraine, and return attraction hours matching Google Maps. However, the author notes limited control over source selection and retrieval behavior, apparent preference for certain sites, and cases where the model disregards system instructions.
---

# Testing the GPT-4o Web Search model

Open AI provides 2 models which has the web search capability:

- gpt-4o-search-preview
- gpt-4o-mini-search-preview

In this article, we use these models and see how they perform.

* * *

## Implementation

1. Providing web search options, which include the user's location information and the context size option for search.

   ```json
   {
       "model": "gpt-4o-search-preview",
       "web_search_options": {
           "search_context_size": "medium",
           "user_location": {
               "type": "approximate",
               "approximate": {
                   "timezone": "Asia/Kuala_Lumpur",
                   "country": "MY",
                   "city": "Kuala Lumpur"
               }
           }
       },
   ...
   }
   ```

2. The context size (low, medium, high) decides how much context from Web Search is used to construct the response. A high setting might be more accurate, but also more costly.

* * *

## Test #1

We try to ask a question beyond the model's knowledge cut-off date.

```markup
What's the upcoming Marvel movies this year?
```

It can provide 3 correct results - Thunderbolts, The Fantastic Four, and Blade.

![](https://camy-pub.s3.ap-southeast-1.amazonaws.com/1de8b7db-2244-4410-af8a-6d8f41ce71f5.png)

* * *

## Test #2

We try to ask a question that happened very recently.

```markup
Is it true that there are anti-trump rally all over the united states?
```

It provides a positive answer by referencing several media reports.

![](https://camy-pub.s3.ap-southeast-1.amazonaws.com/82601e1e-460f-455e-95db-aea2229363d3.png)

* * *

## Test #3

Next, we try to say something false and see if it can be identified.

```markup
Is it true that the US is sending army to Ukraine?
```

It replied that, as of today, there is no such plan.

![](https://camy-pub.s3.ap-southeast-1.amazonaws.com/93b07037-5701-49ea-816b-2c5a9238acfc.png)

* * *

## Test #4

Finally, we try to get the opening hours of several tourist attractions.

```markup
What's the opening hours for the following attractions in Sandakan?
- Sandakan Memorial Park
- St. Michael’s and All Angels Church
- Sepilok Orangutan Rehabilitation Centre
```

It returns the following results:

![](https://camy-pub.s3.ap-southeast-1.amazonaws.com/d5527a21-951f-4418-9219-c602a919ea34.png)

The data is the same as Google Maps, although the Web Search uses several uncommon sources.

* * *

## Conclusion

The model is good at retrieving up-to-date information from web searches. But there is a lack of control over how and where to retrieve that information. It also seems to "favor" a certain website as its source. It is also noticed that the model ignores the instruction in the system prompt.