Contact

Digitizing Dutch Heritage:
AI-Powered Semantic Segmentation of 3D Urban Structures

A Brief Glimpse Into Our Client

Our client is a reputed leader in the urban heritage preservation and city planning sector. They are based in the Netherlands. The client has a simple goal - to safeguard traditional Dutch architecture. In order to fulfill this goal, they leverage cutting-edge technologies to digitize heritage buildings and integrate these datasets into modern urban development platforms like CAD and GIS.

The client approached Brainium because they wanted to automate the identification of architectural elements (including roofs, facades, windows, and doors) using 3D point cloud data and AI. Their goal? To enable scalable urban preservation and planning.

Industry

Others

Tech Stack

Machine Learning & AI

Vision Language Models (VLMs)

3D Point Cloud Processing

Python, Open3D, PyTorch

CAD & GIS Integration

Country

Netherlands

The Challenges We Faced Along the Way

Automating traditional Dutch architecture was a unique challenge that our team encountered. This type of architecture is known for its historical depth and visual intricacy. These qualities might enrich the cultural landscape, but complicated the process of conducting an automated digital analysis. The one-of-a-kind gabled roofs, arched windows, and intricate facades were just the beginning of the hurdles that we had to overcome to provide our client with the service they expected. Here’s a brief overview for better understanding.

Irregular Structures and Styles
TTraditional Dutch buildings feature complex elements like the following -

Gabled roofs
Arched windows
Ornate dormers
Non-uniform facades

These features vary across cities, and sometimes even across buildings on the same street. Due to this architectural irregularity, it was difficult to apply generic segmentation rules or use off-the-shelf AI models. Our team had difficulties identifying clean separation boundaries between components. Since this step is critical for semantic labeling, we had to come up with a solution ASAP.

Class Imbalance
The structural elements that we had to focus on were not equally represented in the dataset. Roofs and facades dominated the point cloud data. Meanwhile, windows and doors appeared less frequently because they were smaller and fewer in number. Dealing with this imbalance was challenging, especially since in model training, deep learning models tend to perform poorly on underrepresented classes. We had to take into account possibilities of false negatives for smaller features during inference.

Limited Labeled Data
The scarcity of annotated training data specific to Dutch heritage buildings was the third major hurdle we had to encounter. There were two issues with public datasets -

They lacked architectural specificity.
They did not include detailed component-level annotations.

Supervised learning approaches fell short because of these issues. The team working on the project had to devise a semi-supervised pipeline supported by geometric rule-based labeling techniques to bootstrap a reliable training set.

Our Primary Approach for This Project

While model training and deployment were certainly our priority, our team focused on a structured pre-deployment phase to ensure data quality, architectural relevance, and operational feasibility. This included the following stages -

Collaborative Discovery Sessions
Our team engaged with the client’s domain experts to truly understand the intricacies of Dutch heritage architecture. We drew up a plan to identify visual patterns and constraints crucial to model design.

Data Assessment & Strategy Planning
We analyzed the high-resolution 3D point clouds for density, completeness, and noise levels. With the results, we created a roadmap for segmentation that accounted for domain-specific architectural challenges.

Custom Annotation Protocols
We did not have any labeled data. So, our team developed a semi-supervised annotation strategy. This included geometric rule-based labeling to bootstrap datasets for underrepresented features like windows and doors.

Pilot Testing Environment
We created a scaled-down environment for proof-of-concept validation. This step was taken to ensure that segmentation models generalized well before being scaled to the full dataset.

These foundational steps were crucial for the project. They helped derisk the project and tailor the solution to both technical and heritage-specific needs.

Our Primary Solution for This Project

With the pre-deployment groundwork set, our team moved on to engineering a robust, AI-driven segmentation pipeline customized for the client's urban heritage use case. The solution included the following processes -

Preprocessing & Noise Reduction
We applied voxel downsampling and outlier filtering to optimize point cloud clarity and reduce computation. We also conducted surface normal estimation and clustering to separate major architectural features like roofs and facades.

Geometric Feature Engineering
The team needed to enhance model context awareness. On that note, they extracted structural cues like curvature, point density, surface orientation. In this stage, we developed bounding-box-based detection heuristics aligned with traditional Dutch proportions. The goal was to provide a reliable baseline for smaller components like windows and doors.

AI-Driven Semantic Segmentation
We deployed a Vision Language Model (VLM) architecture adapted for 3D point cloud data. The next step was integrating traditional geometry rules with deep learning inference to improve accuracy. This step was absolutely crucial for hard-to-detect or imbalanced classes.

Modular Integration & Automation
We took on the responsibility of building a modular pipeline that -

Easily plugs into CAD and GIS systems
Generates automated reports for restoration teams
Supports future scalability for other architectural datasets or geographies

This holistic solution addressed the unique challenges posed by historic Dutch structures, achieving both technical precision and domain relevance.

Results

Our tailored solution delivered measurable improvements in accuracy, efficiency, and integration capabilities, empowering the client to take their heritage preservation efforts to the next level.

>95% IoU Accuracy for Roof and Facade Detection

By integrating geometric feature engineering with deep learning, we achieved highly precise segmentation for larger architectural components.

>90% IoU Accuracy for Doors and Windows

The hybrid AI model delivered high segmentation accuracy, which was critical for use cases like restoration planning and historical documentation

60% Reduction in Manual Annotation Efforts

Instead of spending time annotating point clouds, the client could now focus on high-value tasks like analysis, planning, and client communication.

Seamless Integration with CAD/GIS Platforms

We optimized output formats and metadata for direct use in CAD and GIS software, enabling smooth incorporation into the client's existing urban planning workflows.

Fulfilling Client Needs One Step At A Time

This project represents more than a successful implementation. It is a forward-looking blueprint for how AI and machine learning can preserve history while shaping the cities of tomorrow. Brainium’s role went beyond software development. Our team brought together domain-specific consulting, cutting-edge AI research, and a deep understanding of spatial data to co-create a solution that was both technically sound and contextually relevant. As global interest grows in digital preservation and smart urban planning, Brainium remains at the forefront.

Join Hands With Us

Hear What Our Clients Have to Say

We loved working with Brainium on this project. The developers understood the assignment from the get-go and we didn’t have to micromanage them at every turn. We’re looking forward to future collaborations with the team.