Projects

Robots for autonomous data collection on construction sites

Nowadays, a lot of data gets recorded and stored during the construction process by doing different types of inspections on a regular basis. This process is time consuming and not very reliable, since it is dependent on the human factor. This project develops a fully autonomous robotic platform for data collection in construction sites. The proposed platform is equipped with holonomic wheels, allowing the robot to maneuver through cluttered environments in a more precise way. The robot is also equipped with a different array of sensors that allow the platform to perform autonomous navigation, such as a 120 m range LiDAR, an Inertial Measurement Unit (IMU), a depth camera and encoders on each wheel. The platform is carrying a long-range 3D scanner able to collect point clouds with RGB, thermal, and reflectance information. Everything is managed and run by ROS (Robot Operating System) nodes, interfacing with the SDK of all the different sensors. This research project is led by Borja and Samuel.

Autonomous AI-powered inspection of buildings (ABECIS)

This project presents a methodology to use UAVs and deep learning technology to conduct automated inspections for cracks on high-rise buildings to improve process efficiency and safety of workers. The proposed methodology is divided into four components: 1) Developing a UAV system to capture the exterior wall images of the building in an autonomous way, 2) Collecting data, 3) Processing and analyzing the images captured for cracks using deep learning, and 4) Rendering the identified locations of the cracks on a 3D model of the building, constructed using photogrammetry, for clear visualization. Data Collection, Image-Analysis, and Photogrammetry are done using publicly available open-source deep learning and simulation technologies. The generated code for the UAV simulation and the crack detection algorithm with the pre-trained data model is released on GitHub. Prelimiray work can be found in the paper titled "ABECIS: an Automated Building Exterior Crack Inspection System using UAVs, Open-Source Deep Learning and Photogrammetry" presented at ISARC 2021 (10.22260/ISARC2021/0086). This research project is led by Borja and Pi

IFC Extensions for Inspection of Formwork Systems Using Robots

Quality inspection of formwork systems is a tedious and, in some cases, difficult process and is easily affected by the knowledge, skill, and even the worker’s mood during the inspection, making the collection of data subjective and inconsistent. To address these weakness, the application of robots equipped with sensors for the quality inspection of formwork systems is a promising solution. However, the current Industry Foundation Classes (IFC) schema does not include relevant entities and attributes, which limits its digitalization and automation transformation and upgrading. With this background, this paper aims to extend the IFC schema for accommodating the needs of the quality inspection of formwork systems using robots equipped with sensors. A prototype system based on the extended IFC schema is developed to verify the feasibility and effectiveness of the concept. This research project is led by Borja and Keyi.

Two-step projection of multi-label image classification and masking of point clouds

This project uses an extensive image database with more than 11,000 images taken from various construction sites covering different stages of construction processes. Photos are multi-labeled into 13 different coarse categories. The developed model is an integral part of a point cloud segmentation pipeline where 2D imagery is extracted from the 3D point cloud environment using a set of estimated camera poses. The extracted 2D images are segmented and masked using the developed model into the pre-defined coarse categories. The semantically segmented images annotated with different color masking are projected back to the point cloud carrying additional segment attributes. The proposed course labels in this project includes HVAC ducts, pipes, conduits and wiring systems, formwork, drywall components, etc. These specific course segments will be refined to recognize detailed construction elements and detect defects, delays, safety hazards, and construction site working space limitations. This research project is led by Borja and Eyob.

Game engines and reinforcement learning multi-agent-based for bricklaying construction

This research proposes a framework consisting of a communication interface between Unity and ROS for real-world distributed robotic construction. It deploys reinforcement learning in a game engine-based simulation. The Unity scene provides the environment in which agents observe, act, learn and get feedback. We draw from state-of-the-art reinforcement learning techniques for multi-agent-based execution plan generation by establishing connections between the python API and python trainer with the environment. All the learning algorithms have been set up with the TensorFlow platform to communicate with the Unity model. Then Unity passes all the information collected to ROS, namely, the poses of the robot, target object, target location, and the motion plan. In turn, ROS returns a trajectory message to Unity corresponding to the real robot feedback for further simulation of the remaining task. This research project is led by Borja and Xinghui.

Cybersecurity aspects of Operational Technology on construction sites

Digitization in the construction industry is increasing with the integration of information technologies (IT) and operational technologies (OT) into different activities. The levels of OT to control and monitor site activities utilizing (semi)autonomous and remotely controlled machines raise cybersecurity concerns. Safety issues are magnified given the collaborative work of humans and machines/equipment. This study's motivation is to understand the current state of the art and identify gaps to suggest future directions regarding OT in construction from the cybersecurity perspective. To achieve that, a bibliometric analysis is conducted. The analysis utilizes the Scopus database to retrieve related publications and VOSviewer software to visualize bibliometric networks. Main research themes are identified, and each theme is reviewed from a cybersecurity perspective. The gaps in the existing literature and suggestions for further research related to the cybersecurity of OT in construction are provided. This research project is led by Borja and Semih and funded by the Center for Cyber Security at NYUAD (CCS-AD).

Spatio-temporal modeling and 4D simulation

During the construction of high-rises, construction workers and small and medium-sized materials are typically transported using temporary elevators. Temporary elevator planning plays a positive role in the successful completion of a construction project. However, conventional temporary elevator planning has limitations. On the one hand, the optimization of temporary elevator planning is rarely considered from a spatio-temporal perspective. On the other hand, the display of temporary elevator planning mainly adopts abstract (e.g., text), low-dimensional (e.g., 2D drawing), and static (e.g., 3D model) ways, but such practices are not easy to be fully understood by the project team. To address these limitations, this research proposes a framework for the spatio-temporal planning of temporary elevators considering a virtual reality environment, making temporary elevator planning more effective and intuitive. This research project is led by Borja and Keyi.