Skip to content
All guides

Asking GIS questions in plain English: why AI spatial queries fail and what fixes them

By Mazdak Ghasemi · Last checked 14 October 2026

Large language models can turn a plain-English question such as "which buildings are within 100 m of the tree canopy?" into a spatial database query. On a simple question they often get it right. On the questions people actually ask of GIS data — several layers, a buffer, a join, sometimes 3D — the models tested in recent research usually failed when working alone, and they failed in predictable ways. Models keep improving, so this guide separates two things: what the published evidence measured with the models of its time, and which of the failures a better model can and cannot fix on its own.

Disclosure: I am one of the authors of the study this guide draws on (published as Mazdak Ghasemi Tootkaboni). The figures below are taken from the published paper, which is open access, so you can check every one of them.

What the evidence shows

The study, published in the Journal of Geovisualization and Spatial Analysis in April 2026, built a system called the Spatial Query and Analysis (SQA) framework and tested it on a large urban university campus. The data, held in PostgreSQL 17 with PostGIS, covered buildings, parcels, roads, vegetation, utilities and more, in both 2D and 3D. Sixty questions were written in three levels of difficulty, twenty each, and a GIS expert checked the correct answer for every one:

  • Level 1: filtering a single table, such as buildings constructed after a given year.
  • Level 2: at least one spatial operation (buffer, intersection or distance), usually joining up to two tables.
  • Level 3: multi-step reasoning across several tables, some involving 3D geometry.

Four set-ups were compared. "Spatially valid" means the logic and geometry were correct; "executed" means the query ran in the database without errors or manual fixes.

Set-upLevel 1 valid / executedLevel 2 valid / executedLevel 3 valid / executed
A single model on its own80% / 15%40% / 0%15% / 0%
Plus a description of the database85% / 80%80% / 65%65% / 60%
Plus an error-recovery loop100% / 100%85% / 85%65% / 60%
The full multi-agent framework100% / 100%100% / 100%100% / 90%

Two things stand out. A model on its own often writes logic that is right in concept, but almost none of it runs, because it does not know what the tables and columns are actually called. And each addition helps, but only the full set-up held up on the hardest questions.

Which models, and what newer models change

The study chose a model for each stage by testing candidates on 15 of its questions. It used GPT-3 to sort spatial from general questions, GPT-4 for general questions, Gemini 2.5 Flash to interpret the question, check the reasoning and generate metadata, GPT-4.1-mini to write the SQL and Qwen 3 32B to validate uploaded data. The single-model baseline in the table was an OpenAI model of the GPT-4o generation.

The paper's own comparison shows that model choice already mattered at the time. Gemini 2.5 Flash was strongest at interpreting what a question meant but weaker at writing executable SQL; GPT-3 handled coordinate systems less well; GPT-4o-mini was stable but did not consistently match the schema.

Models released since will write cleaner SQL, so expect the "on its own" row to improve, especially at Level 1. What a better model cannot do is know what it has not been told. Four of the five failures below come from missing information: what your tables and columns are called, what your codes mean, which coordinate system each layer uses, and the steps a question leaves unsaid. Giving the model that information, and checking its work, stays necessary however capable the model is. That is our reading of the results, not something the study measured, so test it on your own data with the model you use.

The five ways it goes wrong

The paper's error analysis and discussion point to five recurring failures.

  1. Guessing table and column names. Without seeing the schema, the model invents plausible names. Across the sixty questions, a non-existent table was referenced in 4 (6.7%) and a wrong column in 9 (15%), for example room where the column was room_name.
  2. Losing track of which column belongs to which table. Once a question needs a join, a model without the schema misplaces joins and mislabels columns, and the errors compound with each extra table.
  3. Treating values as the wrong type. Even with the schema, long single-pass queries sometimes treated text attributes as numbers or applied operators to mismatched types.
  4. Skipping steps the question did not state. The most damaging errors were steps nobody asked for out loud, such as reprojecting layers to a common coordinate system before measuring. Coordinate system misuse appeared in 3 of the sixty questions (5%), and the first recovery pass did not catch any of them; only a second pass did.
  5. Missing what words mean in the data. If a condition column holds A, B and C, and the documentation says A is best and C is worst, a question about "poor" assets needs to be mapped to C (or B). A single model working alone did not make that leap.

One failure did not appear at all: no spatial predicate errors, such as using ST_Within where ST_Intersects was needed, across the sixty questions.

What fixes them

  • Describe your data, not just its structure. The largest single gain came from giving the model a description of each table and column, with their meaning and up to ten sample values. That is what turned 0% executable Level 2 queries into 65%.
  • Let it see its own errors and try again. A recovery loop that reads the database error, checks the schema and retries fixed every table-name error on the first pass. The study found two passes enough; further passes added little.
  • Separate writing from checking. In the full framework, different agents interpret the question, design the spatial operation, review the design independently, and confirm the query runs. Only the full set-up, with that review step, took the hardest questions from 65% to 100% valid.
  • Clean the data first. The study's data was preprocessed: consistent names, nulls removed, values checked. The system assumes values are meaningful, so an occupancy of "999" would flow straight into the answer.

A checklist before you trust an answer

Whether you build your own tool or use someone else's:

  • Ask to see the query. If the answer cannot show the SQL or the operations it ran, you cannot check it.
  • Write down each layer's coordinate system in the metadata, and expect distances and areas to need a projected one. In Victoria that usually means GDA2020 / MGA zone 54 or 55 (EPSG:7854 or 7855), or the statewide GDA2020 / Vicgrid (EPSG:7899).
  • Put the meaning of codes in the metadata, so "poor", "major road" or "heritage-listed" can be matched to the values that represent them.
  • Test on questions you already know the answer to before relying on new ones, and include multi-table questions, not only simple filters.
  • Look at the result on a map. A buffer drawn in degrees instead of metres, or a join that matched nothing, is usually obvious once you see it.

Limits of the evidence

The study used one campus, sixty questions and clean, preprocessed data, and its authors say plainly that real-world data is noisier and that further testing on degraded or mixed data is needed. The models it tested have already been superseded, so the specific percentages will change with newer models and other data. The pattern — schema knowledge first, recovery second, independent review for hard questions — is what to take from it.

Where Terrascout fits

Terrascout's Copilot applies the same principles to a narrower question: which satellite, aerial and spatial data suits an area. It routes each question to a tool — the Terrascout catalogue, an imagery search, or a read-only database query that is checked before it runs — and the model is instructed to answer only from what that tool returned. When the tool has no answer, it says so instead of filling the gap from memory.

To explore data for a place yourself, draw an area on the Discover map.

Further reading

Related guides

Sources