AI in Construction: André Borrmann on Knowledge-Based Engineering

Top TUM researcher André Borrmann offers fascinating insights into research on knowledge-based engineering in construction

When it comes to artificial intelligence, the question is no longer if it will change construction, but when and how. What innovations in AI already exist? What are the current trends? Where do the opportunities lie, and how can they be used in engineering offices in the future?

Prof. Dr.-Ing. André Borrmann, Chair of Computational Modeling and Simulation at the Technical University of Munich, is one of the world’s leading researchers in digitalization in construction. During a recent ALLPLAN online event on “AI in Structural Engineering,” he provided exciting insights into the future of knowledge-based engineering with AI.

One of the many AI research projects at TU Munich is the BIM CoPilot. “It’s about using natural human language as input to generate BIM models,” explains Borrmann. The researchers use a series of different large language models (LLMs), with each model acting as an “agent” responsible for a specific task in the modeling process.

Here’s how it works: users begin by describing a building in natural language (for example, an office building with two floors and a particular façade). The first agent converts this description into precise instructions in the form of coordinates, like the position and height of walls. This agent may also consult an architectural AI agent to incorporate comprehensive knowledge about building structures. These detailed instructions are then passed on to a programming agent, which translates them into API commands for a modeling tool.

The resulting model is automatically checked for quality in Solibri. The results of the check are sent back as feedback to the programming agent, creating a correction loop where increasingly better versions of the building model are generated.

Borrmann sees enormous potential in combining generative AI with BIM, especially as the modeling effort remains a major obstacle to BIM adoption during planning. Greater automation through AI could help overcome this in the future.

Another important research focus for Borrmann and his team is the digital reconstruction of existing structures. The basis for creating digital twins of existing buildings is their geometry, which is often captured using laser scanning or photogrammetry. However, the resulting point clouds are only of limited use as a planning basis.

“That’s why we developed a method that can automatically generate high-quality digital twins with consistent geometry and semantics from these point clouds,” says Borrmann.

This also relies on knowledge-based engineering. A neural network is first trained with human knowledge about how – for example – bridges are constructed. Based on this, the point cloud is semantically segmented. “We divide the original raw point cloud into sub-point clouds for individual components like superstructures, abutments, railings, and so on,” Borrmann explains.

Next comes the so-called “model fitting” step, in which preconfigured parametric objects are matched to the sub-point clouds using an optimization algorithm.

“Because not everything is visible in the point cloud, we also use technical drawings to supplement missing elements – again using AI-based methods,” he adds. The drawings are analyzed by an object detection network from image processing, which operates with very high accuracy.

The AI-generated digital twins of bridges are already precise enough for many real-world applications.

These are just two of many possible AI applications in construction that Borrmann and his team are working on. “In our view, AI has enormous potential in the construction sector,” he emphasizes. There are countless possible use cases, particularly in data evaluation and analysis – such as processing point clouds. The potential is also huge for simplifying repetitive tasks using assistant systems.

However, Borrmann also urges caution: since these are statistical methods, the results inherently carry uncertainties and must always be reviewed by qualified engineers. Equally important is the quality of training data. If an AI is tasked with something it was not explicitly trained to do, there’s a high chance of errors. These limitations must always be kept in mind when using AI.