Skip to content

BLCU Troubleshooting

Jump to bottom
Marc Sanchis edited this page Jun 5, 2025 · 1 revision

BLCU Troubleshooting Guide

Boot Loader Control Unit (BLCU) troubleshooting and repair documentation for Hyperloop UPV Control Station.

🎯 Overview

The BLCU system handles firmware upload and download operations to vehicle boards via TFTP protocol. This guide provides comprehensive troubleshooting for BLCU-related issues and the recent system repairs.

BLCU System Components

  • Backend BLCU Handler (pkg/boards/blcu.go)
  • BLCU Topics (pkg/broker/topics/blcu/)
  • TFTP Client for firmware transfer
  • Frontend Interfaces in Control Station and Ethernet View

🚨 Recent Issue Resolution

Issue Summary

The BLCU system was experiencing communication failures due to:

  1. Missing board registration - BLCU board never added to vehicle
  2. Frontend/backend data format mismatch - Different field names and encoding
  3. Incorrect topic handling - Wrong topic names in message routing
  4. Missing response format - No proper success/failure feedback

Fix Implementation (June 2025)

✅ Board Registration: Added BLCU board to vehicle initialization
✅ Data Format Alignment: Fixed frontend base64 encoding with backend byte handling
✅ Topic Routing: Corrected topic names for proper message flow
✅ Response System: Implemented proper JSON response format

🔧 Current BLCU Operation

System Architecture

Frontend (React) → WebSocket → Broker Topics → Vehicle → BLCU Board → TFTP → Target Board
     ↑                                                                           ↓
   Response ← WebSocket ← Broker ← Board Events ← BLCU ← TFTP Response ← Target Board

Message Flow

Upload Process

  1. Frontend: Sends base64-encoded file via blcu/upload
  2. Backend: Decodes to bytes in UploadRequestInternal
  3. Vehicle: Routes to BLCU board with UploadEvent
  4. BLCU: Initiates TFTP upload to target board
  5. Response: Success/failure sent back to frontend

Download Process

  1. Frontend: Sends download request via blcu/download
  2. Backend: Creates DownloadRequest
  3. Vehicle: Routes to BLCU board with DownloadEvent
  4. BLCU: Downloads firmware via TFTP
  5. Response: Firmware data or error sent to frontend

🔍 Diagnostic Procedures

1. System Status Check

# Check if BLCU board is registered
cd backend
grep -r "BlcuId" pkg/vehicle/
grep -r "AddBoard.*blcu" cmd/

# Verify BLCU topics are registered
grep -r "RegisterTopics" pkg/broker/topics/blcu/

2. Log Analysis

# Backend logs
cd backend/cmd
./backend 2>&1 | grep -i blcu

# Look for these messages:
# ✅ "BLCU board registered" 
# ❌ "BLCU board not registered"
# ❌ "unknown topic"
# ❌ "error handling download/upload"

3. Network Connectivity

# Test TFTP connectivity to target board
tftp [TARGET_BOARD_IP]
> connect
> get test_file
> quit

# Check BLCU IP configuration in ADJ
grep -r "BLCU" backend/cmd/adj/

🐛 Common Issues and Solutions

Issue 1: "BLCU board not registered"

Symptoms:

  • Error message in backend logs
  • BLCU operations fail silently
  • No response to upload/download requests

Solution:

// Verify this exists in cmd/main.go:
if blcuIP, exists := adj.Info.Addresses[BLCU]; exists {
    blcuBoard := boards.New(blcuIP)
    vehicle.AddBoard(blcuBoard)
    trace.Info().Str("ip", blcuIP).Msg("BLCU board registered")
} else {
    trace.Warn().Msg("BLCU address not found in ADJ")
}

Issue 2: Frontend/Backend Data Mismatch

Symptoms:

  • Upload requests fail with "unknown field" errors
  • Base64 decode errors in backend
  • Type assertion failures

Solution: Frontend should send:

{
  "board": "VCU",
  "file": "dGVzdCBkYXRh"  // base64 encoded
}

Backend handles:

type UploadRequest struct {
    Board string `json:"board"`
    File  string `json:"file"`  // base64 encoded
}

type UploadRequestInternal struct {
    Board string
    Data  []byte  // decoded bytes
}

Issue 3: Topic Routing Problems

Symptoms:

  • "unknown topic" messages in logs
  • Messages not reaching BLCU board
  • No response from upload/download

Solution: Verify topic consistency:

  • Frontend sends: blcu/download, blcu/upload
  • Backend expects: blcu/downloadRequest, blcu/uploadRequest

Issue 4: TFTP Connection Failures

Symptoms:

  • TFTP timeouts
  • "connection refused" errors
  • File transfer failures

Solutions:

  1. Check Network Connectivity:
ping [TARGET_BOARD_IP]
telnet [TARGET_BOARD_IP] 69  # TFTP port
  1. Verify TFTP Server on Target:
# Most boards run TFTP server on port 69
nmap -p 69 [TARGET_BOARD_IP]
  1. Check Firewall Rules:
# Ensure TFTP traffic is allowed
iptables -L | grep -i tftp

Issue 5: Response Format Problems

Symptoms:

  • Frontend doesn't show progress
  • No success/failure notifications
  • Incomplete upload/download status

Solution: Ensure proper response format:

// Success response
{
  "percentage": 100,
  "failure": false,
  "file": "[download data for downloads]"
}

// Progress response  
{
  "percentage": 45,
  "failure": false
}

// Failure response
{
  "percentage": 0,
  "failure": true
}

🔧 Manual Testing Procedures

1. Backend Unit Tests

cd backend
go test -v ./pkg/boards/... -run TestBLCU
go test -v ./pkg/broker/topics/blcu/...

2. WebSocket Message Testing

Use browser console or WebSocket client:

// Connect to backend WebSocket
const ws = new WebSocket('ws://localhost:8080/ws');

// Send download request
ws.send(JSON.stringify({
  topic: 'blcu/download',
  payload: JSON.stringify({
    board: 'VCU'
  })
}));

// Send upload request  
ws.send(JSON.stringify({
  topic: 'blcu/upload',
  payload: JSON.stringify({
    board: 'VCU',
    file: btoa('test firmware data')  // base64 encode
  })
}));

3. TFTP Manual Testing

# Test TFTP connectivity manually
echo "test data" > test_file.bin

# Upload test
tftp [TARGET_BOARD_IP]
> binary
> put test_file.bin
> quit

# Download test
tftp [TARGET_BOARD_IP]  
> binary
> get firmware.bin
> quit

🏗️ Development and Testing

Test Environment Setup

# Start backend with debug logging
cd backend/cmd
DEVELOPMENT=true ./backend

# Start frontend development server
cd control-station
npm run dev

# Access bootloader interface at:
# http://localhost:5173/bootloader

Mock TFTP Server

For testing without hardware:

#!/usr/bin/env python3
# mock_tftp_server.py

import socket
import threading
import time

def handle_tftp_request(data, client_addr, sock):
    # Simple TFTP ACK response
    response = b'\x00\x04\x00\x00'  # ACK packet
    sock.sendto(response, client_addr)
    print(f"Sent ACK to {client_addr}")

def tftp_server():
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock.bind(('0.0.0.0', 69))
    print("Mock TFTP server listening on port 69")
    
    while True:
        data, addr = sock.recvfrom(1024)
        print(f"Received TFTP request from {addr}: {data[:20]}...")
        threading.Thread(
            target=handle_tftp_request, 
            args=(data, addr, sock)
        ).start()

if __name__ == "__main__":
    tftp_server()

Integration Testing

# Full integration test script
#!/bin/bash

echo "=== BLCU Integration Test ==="

# 1. Start backend
cd backend/cmd
./backend &
BACKEND_PID=$!

# 2. Wait for startup
sleep 3

# 3. Test BLCU board registration
curl -s http://localhost:8080/health | grep -q "blcu" && echo "✅ BLCU registered" || echo "❌ BLCU not found"

# 4. Test WebSocket connection
echo "Testing WebSocket endpoints..."

# 5. Cleanup
kill $BACKEND_PID
echo "Integration test complete"

📊 Monitoring and Maintenance

Performance Metrics

Monitor these BLCU-related metrics:

# TFTP transfer success rate
grep -c "UploadSuccess\|DownloadSuccess" /var/log/backend.log

# Average transfer time  
grep "transfer.*completed" /var/log/backend.log | awk '{print $X}'

# Error frequency
grep -c "BLCU.*error\|TFTP.*failed" /var/log/backend.log

Health Check Script

#!/bin/bash
# blcu_health_check.sh

check_blcu_status() {
    # Check if BLCU board is responding
    curl -s http://localhost:8080/api/boards/blcu/status || return 1
    
    # Check TFTP connectivity
    timeout 5 nc -u -z [TARGET_BOARD_IP] 69 || return 1
    
    return 0
}

if check_blcu_status; then
    echo "$(date): BLCU system healthy"
else
    echo "$(date): BLCU system issues detected"
    # Alert team or restart services
fi

🔗 Related Documentation

📝 Change Log

Version 3.1.0 (June 2025)

  • ✅ Fixed BLCU board registration in vehicle initialization
  • ✅ Aligned frontend/backend data formats for upload/download
  • ✅ Corrected topic routing for proper message flow
  • ✅ Implemented proper response handling with progress feedback
  • ✅ Added comprehensive test suite for BLCU operations
  • ✅ Updated documentation with troubleshooting procedures

Previous Issues (Resolved)

  • ❌ BLCU operations failing silently
  • ❌ Frontend/backend communication errors
  • ❌ Missing progress feedback
  • ❌ Inconsistent error handling

✅ BLCU System Status: Operational

The BLCU system has been fully repaired and tested. For ongoing issues, follow the diagnostic procedures above or contact the development team through GitHub Issues.

Clone this wiki locally