Latency optimized architectures for a real-time inference pipeline for control tasks Conference Paper