Context #07 | What should we really fear about artificial intelligence?
The most dangerous error is the one that looks right.

For decades, artificial intelligence held a special place in the popular imagination. The possibility of machines that could think, decide and act autonomously inspired stories about the future of humanity, often shaped by the fear that we might lose control of the technology we had created.
Today, artificial intelligence is part of everyday life. It is built into the tools we use, the systems that organise information and, increasingly, the processes that support professional decisions.
As AI becomes more embedded in our daily lives, new risks are emerging. One is how readily we can come to trust answers even when we struggle to understand or verify what they are based on.
When a convincing answer can be wrong
One of the most striking features of today’s AI systems is their ability to produce structured, coherent and seemingly well-supported answers in seconds. Fluent language, well-organised arguments and clear conclusions can inspire more confidence than the quality of the underlying information warrants.
A system might cite a law that does not exist, misinterpret a planning provision or draw a conclusion from data that cannot support it, all without showing any obvious sign of uncertainty.
This matters in particular when the information starts to inform decisions. An obvious error tends to prompt questions. A plausible conclusion can pass into a decision without anyone properly checking the evidence behind it.
Spatial analysis depends on context
In architecture, urban planning and spatial analysis, decisions rarely depend on a single variable. Assessing a site, for example, means considering spatial planning instruments, regulatory provisions, constraints, physical characteristics, cadastral information and the specific circumstances of the proposed development together.
Even when information is available, it may be incomplete, out of date or linked to the wrong context. A development parameter may apply only to a specific category of land. A planning provision may have been amended, or a constraint may depend on a mapped boundary whose accuracy still needs to be confirmed.
AI can help identify these relationships, organise documents and analyse large volumes of information. Interpreting the results still requires an understanding of the context in which each piece of information applies.
Identifying a rule is one step. Establishing whether it applies can determine whether a development proposal is viable, however sophisticated the system used to assess it.
The risk of no longer asking questions
The growing use of AI is also changing, in less obvious ways, how we research, analyse and verify information.
When a tool provides answers almost instantly, it is tempting to spend less time consulting sources, comparing interpretations and checking results. As we become accustomed to getting satisfactory answers, we may grow more confident in the tool and scrutinise its conclusions less closely.
This deserves particular attention in fields where errors can take time to surface. A misinterpretation may carry through several stages before anyone spots it, leading to poorly judged investments, project redesigns, delays or additional costs.
Using these tools calls for critical judgement. That includes recognising when an answer needs to be checked, even if it sounds entirely convincing.
Responsibility cannot be automated
Technology is changing how we produce and use knowledge. AI applications can connect information, recognise patterns, help identify inconsistencies and carry out tasks that once required considerable manual effort.
These capabilities can improve the efficiency and quality of technical work when appropriate checks and validation methods are in place.
As AI takes on a greater role in analysis, it becomes essential to understand where the information comes from, what criteria underpin the results and what limitations remain. Responsible use depends on being able to trace sources, state assumptions clearly, identify uncertainties and validate results through technical review.
As these systems become more sophisticated, clear accountability remains essential. Automated systems can support a decision, and the reasoning behind it must remain understandable, verifiable and open to scrutiny.
Segmento Urbano’s perspective
At Segmento Urbano, we use AI to extend our analytical capacity while keeping technical interpretation in the hands of professionals.
Its value lies in processing and connecting information more efficiently, giving us more time to focus on questions that require specialist knowledge, an understanding of context and critical thinking.
Bringing these technologies into our work means combining what digital systems can do with professional experience and accountability. It also means recognising the limits of the technology, assessing the quality of the available data and knowing when a conclusion requires further analysis.
As answers become easier to produce, knowing what questions to ask, where uncertainty remains and how to verify conclusions becomes increasingly important.
Perhaps what we should really fear about artificial intelligence is how easily we might stop questioning the answers it gives us.



