Visualization of the human trajectory prediction system in a factory setting. The system tracks previous trajectories (green) and predicts future movement (red), enabling proactive workstation preparation and non-invasive mobile robot navigation.
Research Objectives
- Develop algorithms for accurate prediction of human movement in industrial environments
- Enable machines to predict human intentions for enhanced safety and efficiency
- Reduce the need for traditional safety cages through intelligent trajectory prediction
- Increase productivity by proactively activating collaborative workstations before human arrival
- Create solutions that work with limited real-world training data through transfer learning and data augmentation
Research Challenges Addressed
- Limited availability of real-world human trajectory data in industrial settings
- Difficulty in capturing the complexity and variability of human movement patterns
- Need for real-time predictions that are both accurate and computationally efficient
- Balancing privacy concerns with data collection needs
- Creating models that generalize across different industrial environments
Research Approach
Data Collection
Using a state-of-the-art LiDAR system with reflective mirrors that extend the field of view, covering a 26m × 20m industrial floor. Human detection algorithms identify and track operators as they move between workstations.
Data Simulation
Developing synthetic human trajectory data using the Ornstein-Uhlenbeck stochastic process and TimeGAN (Generative Adversarial Networks for time series data) to overcome limited data availability.
Prediction Models
Implementing advanced deep learning architectures including LSTM networks, Transformer models, and other sequence prediction techniques to forecast human movements with high accuracy.
Transfer Learning
Pre-training models on synthetic data before fine-tuning on real-world data to improve performance in data-scarce scenarios.
Heatmap visualization of the human trajectory dataset collected from a real-world industrial environment, showing transition paths between six workstations.
Key Innovations
Transformer-Based Sequence-to-Sequence Model
A novel architecture for human trajectory prediction that outperforms traditional recurrent models by leveraging self-attention mechanisms and positional encoding to capture both short-term and long-term dependencies in movement patterns.
Transfer Learning with Data Augmentation
Combining synthetic trajectory generation with transfer learning to significantly improve prediction accuracy when real-world data is limited. Our approach showed an overall accuracy gain of 4.8 percentage points compared to training on real-world data alone.
Real-Time Prediction Framework
Predicting target workstations with 95.6% accuracy three seconds in advance, enabling systems to proactively prepare for human arrival and optimize collaborative workflows.
Comparative results of trajectory predictions using different algorithms. The transformer-based model (top) shows significantly better performance than traditional approaches.
Research Publications
Industrial Applications
Enhanced Safety
Enabling hazardous machinery to slow down before a human approaches, reducing the risk of accidents without the need for physical safety barriers.
Improved Efficiency
Collaborative workstations can be proactively activated before the human operator arrives, eliminating wait times and increasing productivity.
Space Optimization
Reducing or eliminating the need for traditional safety cages increases usable shop floor space in manufacturing environments.
Mobile Robot Navigation
Providing trajectory predictions for mobile robots to plan efficient paths that avoid disturbing or interfering with human activities.
Possible Future Research Directions
- Building larger and more diverse datasets to improve model generalization across different industrial environments
- Integrating the prediction system with real-time robot control for adaptive collaboration