← Back to posts

AI Racing Replay with IBM watsonxai

By Ediz · 14:50, 7 Apr 2026

Required Additional Components:

Table of Contents

  1. Overview
  2. The Data
  3. Setting Up the Unity Project
  4. Parsing the Telemetry CSV
  5. Replaying the Car's Movement
  6. Physics Analysis: Cornering Efficiency
  7. AI Commentary with watsonxai
  8. Text-to-Speech with IBM Watson
  9. Adjusting the Track Model in Blender
  10. Assembly

1. Overview

Let's overview the project before we start writing the code.

Racing cars record data at regular intervals to analyse the happenings on track. This data is called telemetry, and it includes the car's GPS position, speed, heading (direction the nose is pointed), and acceleration. We're going to take that data and do three things with it:

  1. Replay it in 3D.
  2. Analyse the physics to show how close the driver is to losing grip in the corners.
  3. Generate AI commentary that describes what's going on.

Here's the system design diagram, if you are unfamiliar with UMLs, don't worry, you won't need to understand these details to follow along with the guide:

image

The telemetry is in the middle, the AI classes are towards the top, and the physics classes are at the bottom.

2. The Data

A bit on VBOX

VBOX is a device that plugs into a race car and records all the information from the various sensors. The sample data can be found here using the Circuit Tools software, which is free. It dumps everything into a CSV file, which is basically a regular spreadsheet you can open in Excel. But the spreadsheet would be too big to read since telemetry is recorded every 50 milliseconds, you can imagine it gets pretty long during a race session.

This is what a few rows look like for reference:

UTC Time Speed (km/h) Heading (°) Latitude Longitude Height (m) Time (s) Lat.G (g) Lon.G (g)
145858.8 16.19 332.1 52°49.7835 N 1°22.7817 W 89.32 0.0 0.00 0.00
145858.9 16.36 330.8 52°49.7836 N 1°22.7820 W 89.53 0.1 0.11 0.05
145859.0 15.75 328.0 52°49.7837 N 1°22.7822 W 89.63 0.2 0.22 -0.17

The important columns are:

  • Speed: how fast the car is going, in km/h.
  • Heading: the compass direction the car is pointing. 0° is North, 90° is East...
  • Latitude / Longitude: the car's GPS position on Earth.
  • Height: altitude in metres above sea level.
  • Time: just regular old time.
  • Lat.G / Lon.G: the sideways and forwards forces on the car, measured in "g" (where 1g is the force of gravity).

Two Problems

Problem 1: The data only updates 20 times per second. Sounds fast, but a game runs at 60+ frames per second. If we just teleport the car from one recorded point to the next, it'll look jittery. We'll fix this with interpolation, which means to smoothen the motion by estimating the car's trajectory and filling in data for the missing points.

Problem 2: The height data is inaccurate. GPS is decent at telling you where you are on a flat map (accurate to about 1 metre), but it's terrible at altitude. Off by up to 4 metres. A race car is only about 1.2 metres tall. So the GPS might put the car floating above the track or buried under it. We'll fix this with a trick called raycasting later.

3. Setting Up the Unity Project

Open Unity Hub and create a new 3D (Built-in Render Pipeline) project. Call it RacingReplay.

  1. Import the track model. Drag your Donington Park .fbx or .obj file into the Assets folder, then drag it into the scene. Unfortunately due to copyright I can't upload the same track I used. But to get the same model you can download the kn5 files from here and use this converter

  2. Add a MeshCollider to the track. Click the track in the scene, hit "Add Component" in the Inspector, and search for MeshCollider. This is what the raycasting will collide with later.

  3. Import a car model. Grab any free car from the Unity Asset Store, or just use a cube.

  4. Create an empty GameObject for the AI. Right-click on the left window -> Create Empty -> name it AIManager. The commentary scripts will go here.

It should look roughly like this:

image

4. Parsing the Telemetry CSV

First order of business is to read the CSV file and turn each row into something we can work with.

Create a new C# script in Unity called TelemetryReplay and attach it to the car object.

The Data Structure

We'll add a class to store the values from the data.

public class TelemetryFrame
{
    public float time;
    public float speed;       // km/h
    public float heading;     // degrees
    public float latitude;    // decimal degrees
    public float longitude;   // decimal degrees
    public float height;      // metres
    public float latG;        // lateral g-force
    public float lonG;        // longitudinal g-force
    public Vector3 position;  // calculated Unity position
}

Each TelemetryFrame is one snapshot of time in the data. Although position doesn't come from the CSV, that will be calculated using the latitude and longitude.

Reading the CSV

Unity doesn't ship with a CSV parser, but since we know the indexes for the columns already, all we need to do is split the data by the commas.

void LoadTelemetry(string filePath)
{
    string[] lines = File.ReadAllLines(filePath);
    frames = new List<TelemetryFrame>();

    for (int i = headerLines; i < lines.Length; i++)
    {
        string[] cols = lines[i].Split(',');
        TelemetryFrame frame = new TelemetryFrame();
        
        frame.time = float.Parse(cols[7], CultureInfo.InvariantCulture);
        frame.speed = float.Parse(cols[1], CultureInfo.InvariantCulture);
        frame.heading = float.Parse(cols[2], CultureInfo.InvariantCulture);
        frame.latitude = ParseLatLon(cols[3]);
        frame.longitude = ParseLatLon(cols[4]);
        frame.height = float.Parse(cols[5], CultureInfo.InvariantCulture);
        frame.latG = float.Parse(cols[9], CultureInfo.InvariantCulture);
        frame.lonG = float.Parse(cols[10], CultureInfo.InvariantCulture);
        
        frames.Add(frame);
    }
}

As you see the function takes in the filePath of the data and populates the correct values using the hard coded indexes such as 7 for time and 2 for heading.

Couple of things to note. The frame. syntax might seem odd. If you are unfamiliar with OOP, you may imagine the frame. as a box that holds a bunch of other fields and organises our code better.

Secondly, CultureInfo.InvariantCulture is unexplained. Different countries write numbers differently. Some places use commas as decimal points (like 3,14 instead of 3.14) so we need to specify how to read the values.

So the parsing is pretty simple, but you can see we call another function called ParseLatLon for the latitude and longitude.

Parsing the Weird Latitude/Longitude Format

VBOX stores coordinates in a format like 52°49.783494 N. That's degrees and minutes. First we need it as a clean decimal:

float ParseLatLon(string coord)
{
    // Format example: "52^49.783494 N"
    char[] separators = { '^', ' ' };
    string[] parts = coord.Split(separators, StringSplitOptions.RemoveEmptyEntries);

    float degrees = float.Parse(parts[0], CultureInfo.InvariantCulture);
    float minutes = float.Parse(parts[1], CultureInfo.InvariantCulture);
    string direction = parts[2];

    float decimalDegrees = degrees + (minutes / 60.0f);
    if (direction == "S" || direction == "W") decimalDegrees *= -1;

    return decimalDegrees;
}

The conversion is decimal degrees = degrees + (minutes / 60). If the direction is South or West, we flip it negative.

Converting the Decimals to Unity Coordinates

Now we have the latitude and longitude as decimal numbers, but it will need conversion to positions in Unity's 2D world.

Vector3 LatLonToMeters(float lat, float lon, float lat0, float lon0)
{
    float R = 6371000f; // Earth's radius in metres
    float dLat = (lat - lat0) * Mathf.Deg2Rad;
    float dLon = (lon - lon0) * Mathf.Deg2Rad;
    float lat0Rad = lat0 * Mathf.Deg2Rad;

    float x = R * dLon * Mathf.Cos(lat0Rad); // East-West becomes Unity X
    float z = R * dLat;                       // North-South becomes Unity Z

    return new Vector3(x, 0, z);
}

For this we need to project the coordinates from earth to 2D, since the world is elliptoid, so it does not directly translate. This mostly only matters for longitude since latitudes are spaced evenly, but lines of longitude squeeze closer together as you approach the poles. The Cos(latitude) handles a simple geometric projection.

With this setup, we can calculate the position for each frame:

// Use the first frame as our origin
float lat0 = frames[0].latitude;
float lon0 = frames[0].longitude;

for (int i = 0; i < frames.Count; i++)
{
    frames[i].position = LatLonToMeters(frames[i].latitude, frames[i].longitude, lat0, lon0);
}

5. Replaying the Car's Movement

For every frame Unity renders, we'll grab the matching telemetry data and move the car to that spot... for now.

In the Update() method (Unity calls this every single frame):

void Update()
{
    replayTime += Time.deltaTime;  // advance our clock

    TelemetryFrame currentFrame = GetFrameAtTime(replayTime);

    // Move the car
    Vector3 targetPos = currentFrame.position;
    transform.position = targetPos;

    // Rotate the car to face the right direction
    transform.rotation = Quaternion.Euler(0f, currentFrame.heading, 0f);
}

We are using that frame box to render the car's new position each 'frame'.

If you hit Play right now, the car will zip around the track but it'll look bad. The movement is jittery because there is 50ms gaps between data points and the car is hovering.

Fixing the Height Problem with Raycasting

The GPS height is quite inaccurate, so we'll just throw out the GPS height completely and ask Unity where the ground is. I mean the car is probably not flying.

Think of it as pointing a laser down from above the car. Wherever that laser hits the track surface, that's where the car should sit.

// Inside Update(), after calculating targetPos from telemetry:

RaycastHit hit;
Vector3 rayOrigin = new Vector3(targetPos.x, targetPos.y + 100f, targetPos.z);

if (Physics.Raycast(rayOrigin, Vector3.down, out hit, 200f))
{
    targetPos.y = hit.point.y + carHeightOffset;
}

transform.position = targetPos;

Raycasting 101:

  1. We start the ray some metres above where the car should be.
  2. Shoot it straight down (Vector3.down).
  3. If it hits something (the MeshCollider we added to the track earlier), we grab the hit point's Y coordinate as the ground level.
  4. We add carHeightOffset so the car sits on the surface rather than clipping through it.

Is It Necessary?

Here is the car's path when using its own height data.

image

The yellow lines are the GPS paths from multiple laps on the same straight. They are at completely different heights! Raycasting pins the car to the track surface on every frame. Some problems like this require a little backend magic.

Smoothening Motion with Frame Interpolation

At this point our car jumps from point to data point every 50 milliseconds. Unity runs at 60+ frames per second (roughly one frame every 16ms). It looks a little choppy.

We can smoothen it using linear interpolation, "lerp" for short. Between every two data points, we work out where the car would be based on the other two positions we know for certain.

TelemetryFrame GetFrameAtTime(float time)
{
    for (int i = 0; i < frames.Count - 1; i++)
    {
        if (time >= frames[i].time && time <= frames[i + 1].time)
        {
            // How far between the two frames are we? (0.0 to 1.0)
            float t = (time - frames[i].time) / (frames[i + 1].time - frames[i].time);

            TelemetryFrame result = new TelemetryFrame();
            result.position = Vector3.Lerp(frames[i].position, frames[i + 1].position, t);
            result.speed = Mathf.Lerp(frames[i].speed, frames[i + 1].speed, t);
            result.heading = Mathf.LerpAngle(frames[i].heading, frames[i + 1].heading, t);
            result.time = time;
            return result;
        }
    }
    return null; // past the end of the data
}

Why linear interpolation works

A race car can't change speed instantly. It's quite heavy and obeys the laws of physics. In 50 milliseconds, speed and direction change negligibly. So drawing a straight line between two data points is a pretty good guess at what actually happened in between. In cases where rate of change also changes, this approach does not work since it does not behave linearly.

Engine Audio

A replay with no sound is a bit lifeless. You can't feel whether the car is accelerating or braking when the camera is positioned relative to the chassis. There needs to be sound that changes based on what the car is doing.

We'll blend three separate audio clips to achieve the illusion of a real car engine.

Clip When it's loudest
Low engine rumble At low speeds
High acceleration rumble At high speeds while on the throttle
High deceleration rumble At high speeds while braking

Unity Components

  1. Add three AudioSource components to your car object.
  2. Assign each clip to its own AudioSource.
  3. Tick Loop and Play On Awake for all three.

The Blending Code

void UpdateEngineAudio(TelemetryFrame currentFrame)
{
    // Pitch up the audio as speed increases
    float pitch = 1.0f + (currentFrame.speed * pitchMultiplier);

    // How much "high engine" vs "low engine" based on speed
    float highFade = Mathf.InverseLerp(0f, 100f, currentFrame.speed);
    float lowFade = 1f - highFade;

    // How much "accelerating" vs "decelerating" based on acceleration
    float accFade = Mathf.InverseLerp(-1f, 1f, currentFrame.acceleration);
    float decFade = 1f - accFade;

    // Set volumes
    lowSound.volume = lowFade;
    highAccSound.volume = highFade * accFade;
    highDeaccSound.volume = highFade * decFade;

    // Set pitch on all sources
    lowSound.pitch = pitch;
    highAccSound.pitch = pitch;
    highDeaccSound.pitch = pitch;
}

With that the replay logic is done. I know it was a lot to take in, feel free to nab the full code instead of piecing it all together. Go ahead and watch the car zip around for a while.

📋 Full Code: TelemetryReplay.cs

The complete TelemetryReplay class with CSV parsing, coordinate projection, raycasting, interpolation, and audio is below. You can copy this whole file straight into your project.

6. Physics Analysis: Cornering Efficiency

This is where data is transformed into insight (there is your LinkedIn hook). We want to know how close the driver is to the traction limit. Because I mean, it's just cool.

A bit on G-Force

When you take a fast corner, centripetal force pushes you sideways. In this case it is called lateral G-force.

If the lateral G-force exceeds what the tyres can support, the car will slide. We just need to know two things:

  • How much G-force is the car pulling right now? (from the telemetry)
  • How much can the tyres actually handle? (from physics)

Divide the first by the second to get the cornering efficiency. Which will show what percentage of the available limit the driver is using.

Step 1: Calculate the Velocity Vector

The telemetry gives us speed, but we want velocity (speed with a direction) in order to compute the lateral force. Lateral is a direction so we'll need to work with a vector instead of a scalar.

float speedMps = currentFrame.speed / 3.6f;  // km/h to m/s
float headingRad = currentFrame.heading * Mathf.Deg2Rad;

Vector3 velocity = new Vector3(
    Mathf.Sin(headingRad) * speedMps,
    0f,
    Mathf.Cos(headingRad) * speedMps
);

Dividing by 3.6 converts km/h to m/s. Physics standard units are metres and seconds.

Step 2: Calculate Acceleration

Acceleration is change in velocity over time, straight from your middle school physics class:

Vector3 acceleration = (velocity - prevVelocity) / dt;

Then we split it into two parts using some trigonometry:

  • Longitudinal (along the car's nose): speeding up or slowing down
  • Lateral (across the car): the cornering force
Vector3 forward = velocity.normalized;
Vector3 right = new Vector3(forward.z, 0, -forward.x);  // perpendicular to forward

float longitudinalAccel = Vector3.Dot(acceleration, forward);
float lateralAccel = Vector3.Dot(acceleration, right);

Step 3: Calculate the Grip Limit

The grip "budget" depends on three things:

  • The car's weight. Heavier car means more normal force, which is a component of friction.
  • Aerodynamic downforce. At high speed, aero components add to the normal force.
  • Tyre friction. How sticky the tyres are on the track surface affects the grip too.
// Total downward force on the tyres
float normalForce = vehicleMass * GRAVITY + (downforceCoefficient * speedMps * speedMps);

// Maximum sideways force before sliding
float maxLateralForce = tireFrictionCoefficient * normalForce;

// Convert to g-force
float maxLateralG = maxLateralForce / (vehicleMass * GRAVITY);

Customisable fields: vehicleMass, downforceCoefficient, and tireFrictionCoefficient are public fields on the script, so you can tweak them in Unity's Inspector. Different cars have different values. A GT3 race car weighs about 1,300 kg and has a downforce coefficient around 2.5.

Step 4: Cornering Efficiency

Now we can compute the cornering efficiency as a fraction.

float longitudinalGripUsage = Mathf.Abs(currentLongitudinalG) / currentMaxLateralG;
float availableLateralGrip = currentMaxLateralG * 
    Mathf.Sqrt(Mathf.Max(0, 1f - longitudinalGripUsage * longitudinalGripUsage));

float corneringEfficiency = currentLateralG / Mathf.Max(0.01f, availableLateralGrip);
Additional Vector Maths

Braking and turning share the same grip pool. If the driver is braking while also turning (often called trail braking), some of the grip budget goes to braking and there's less left for cornering. The sum of the vectors gives the total amount of force handled by the tyres.

Cornering efficiency near 1.0 (100%) means the driver is close to the edge. Consistently below 0.8 (80%) means there's more potential left on the table. But we need a way to display this metric, since the raw number doesn't mean all that much.

📋 Full Code: TelemetryPhysicsAnalyzer.cs

The complete physics analysis class with velocity calculation, G-force derivation, grip estimation, and cornering efficiency.

Drawing the G-Force Trail

Since the efficiency metric is once per frame, which is 60 times a second, the text would just be flashing incredibly quick, nobody can read that. Instead, we'll adopt a coloured trail behind the car that makes the cornering efficiency easier to grasp.

  • 🟢 Green = loads of grip left, safe
  • 🟡 Yellow = moderate grip usage
  • 🟠 Orange = getting close to the edge
  • 🔴 Red = at grip limit

Setting Up the Trail

Unity's LineRenderer component draws a line through a series of points. We can drop old points and attach new ones, to achieve a snake like trail, just so it doesn't grow infinitely.

void Start()
{
    line = gameObject.AddComponent<LineRenderer>();
    line.startWidth = trailWidth;
    line.endWidth = trailWidth;
    line.material = new Material(Shader.Find("Sprites/Default"));
}

Sampling Points

At regular intervals, take a snapshot of the car's position and its current efficiency:

void Update()
{
    if (Time.time - lastSample >= sampleRate)
    {
        points.Add(new TrailPoint
        {
            position = transform.position,
            efficiency = analyzer.corneringEfficiency,
            time = Time.time
        });
        lastSample = Time.time;
    }

    // Remove anything too old
    points.RemoveAll(p => p.time < Time.time - trailDuration);
}

The Smoothing Problem

First time I built this, the trail looked awful. Jagged and flickery, because the raw data has microstutters between frames. The fix turned out to be a rolling average: instead of using each point's exact position, average it with its neighbours.

Vector3 SmoothPosition(int i)
{
    int half = smoothingWindow / 2;
    int start = Mathf.Max(0, i - half);
    int end = Mathf.Min(points.Count - 1, i + half);

    Vector3 sum = Vector3.zero;
    for (int j = start; j <= end; j++)
        sum += points[j].position;

    return sum / (end - start + 1);
}

This is the same principle behind noise reduction in different areas of computer science. We just average out the bumps.

Mapping Efficiency to Colour

Color EfficiencyToColor(float e)
{
    if (e > 1f) return Color.red;
    if (e < 0.5f) return Color.Lerp(Color.green, Color.yellow, e / 0.5f);
    if (e < 0.75f) return Color.Lerp(Color.yellow, new Color(1f, 0.5f, 0f), (e - 0.5f) / 0.25f);
    return Color.Lerp(new Color(1f, 0.5f, 0f), Color.red, (e - 0.75f) / 0.25f);
}

It's just a simple if condition. But since we are adjusting the intensity of the colour as well, it all blends together.

image

📋 Full Code: GForceTrailRenderer.cs

The complete trail rendering class with point sampling, smoothing, colour mapping, and LineRenderer management.

7. AI Commentary with watsonxai

Now for the final part. We're going to get AI to watch the replay and describe what's happening, like a commentator, albeit an amateur one.

The Problem with Giving AI Raw Numbers

Unfortunately we can't just chuck the telemetry numbers at a language model and say "commentate on this." Language models are trained on words, not numbers. Feeding it "speed is 215.4 km/h, heading changed by 12.3 degrees" produces gibberish because the model has no idea what those numbers actually mean in a racing context.

Instead, we can do the heavy lifting and translate those numbers into text, and then send that to the AI instead.

Step 1: Analyse the Race State

Build a RaceStateAnalyzer that watches the telemetry and works out what's happening in English words.

// Is the car turning?
float headingChange = Mathf.Abs(Mathf.DeltaAngle(prevHeading, frame.heading)) / Time.deltaTime;
bool isCornering = headingChange > corneringThreshold;

// Is it speeding up or slowing down?
float speedChange = frame.speed - prevSpeed;
float acceleration = speedChange / Time.deltaTime;

From these checks we extract text flags like:

  • "BRAKING INTO CORNER" when heading is changing and speed is dropping
  • "ACCELERATING OUT OF CORNER" when heading is straightening and speed is climbing
  • "ON THE STRAIGHT" when nothing's changed direction for a while
  • "HEAVY BRAKING" when there's a big speed drop in a short time

Step 2: Build the Prompt

The CommentaryPromptBuilder takes these flags and constructs the prompt for the language model:

StringBuilder p = new StringBuilder();
p.AppendLine("You are an enthusiastic motorsport commentator.");
p.AppendLine("Give ONE short sentence about the current moment.");
p.AppendLine();
p.AppendLine("SITUATION:");
p.AppendLine($"Event: {eventType}");
p.AppendLine($"Speed: {state.speed:F0} km/h");
p.AppendLine($"Acceleration: {state.acceleration:F1} m/s^2");
p.AppendLine($"Location: {state.section}");

If the event is braking, the prompt gets injected with extra terms like "scrub speed" and "threshold braking" so the AI picks up on racing lingo.

Step 3: Call the IBM watsonx API

Next, send it to the AI. We'll use IBM watsonx.ai for this, it is extremely easy to setup an account for free and get experimenting.

var requestBody = new WatsonxGenerationRequest
{
    model_id = modelId,       // e.g., "ibm/granite-3b-code-instruct"
    project_id = projectId,   // from your IBM Cloud dashboard
    input = prompt,
};

// Convert to JSON and send via HTTP POST
string json = JsonUtility.ToJson(requestBody);
UnityWebRequest www = new UnityWebRequest(apiUrl, "POST");
www.uploadHandler = new UploadHandlerRaw(Encoding.UTF8.GetBytes(json));
www.downloadHandler = new DownloadHandlerBuffer();
www.SetRequestHeader("Content-Type", "application/json");
www.SetRequestHeader("Authorization", "Bearer " + apiKey);

yield return www.SendWebRequest();

// Parse the response
string responseText = www.downloadHandler.text;

Sign up at IBM Cloud, make a watsonx.ai project, and generate an API key in the project settings. The free tier covers quite a lot of queries.

Reason for Small Language Model

The commentary needs to come back within a second, which means we are limited to either SLM or a small LLM. SLMs are also cheaper. Small models like IBM Granite are built for this kind of focused job. To see the language models available, you can right click the script component in Unity, I added a debug function for it in the full code.

📋 Full Code: RaceStateAnalyzer.cs

📋 Full Code: CommentaryPromptBuilder.cs

📋 Full Code: CommentaryManager.cs

📋 Full Code: WatsonxService.cs

8. Text-to-Speech with IBM Watson

The AI is generating text commentary but we don't want to read it ourselves. IBM Watson Text-to-Speech takes the text and turns it into an audio clip.

Making the API Call

string url = $"{serviceUrl}/v1/synthesize?voice={voice}";
string jsonBody = $"{{\"text\":\"{EscapeJson(text)}\"}}";

UnityWebRequest www = new UnityWebRequest(url, "POST");
www.uploadHandler = new UploadHandlerRaw(Encoding.UTF8.GetBytes(jsonBody));
www.downloadHandler = new DownloadHandlerBuffer();
www.SetRequestHeader("Content-Type", "application/json");
www.SetRequestHeader("Accept", "audio/wav");
www.SetRequestHeader("Authorization", "Bearer " + apiKey);

yield return www.SendWebRequest();

This is another hiccup, the API sends back raw WAV bytes but Unity can't play raw bytes. It needs an AudioClip object. So we have to parse it ourselves.

Parsing WAV Audio

A WAV file is laid out like this:

[RIFF header] [fmt chunk with sample rate, channels, etc.] [data chunk with actual audio bytes]

We'll need to read the format and convert it accordingly.

AudioClip ParseWav(byte[] wavData)
{
    // Read format info from the WAV header
    int channels = BitConverter.ToInt16(wavData, 22);
    int sampleRate = BitConverter.ToInt32(wavData, 24);
    int bitsPerSample = BitConverter.ToInt16(wavData, 34);

    // Find where the actual audio data starts
    int dataStart = 44; // standard WAV header size
    int dataSize = wavData.Length - dataStart;
    int totalSamples = dataSize / (bitsPerSample / 8);

    float[] audioData = new float[totalSamples];

    if (bitsPerSample == 16)
    {
        for (int i = 0; i < totalSamples; i++)
        {
            // Each sample is 2 bytes (16-bit), ranging from -32768 to 32767
            short sample = BitConverter.ToInt16(wavData, dataStart + i * 2);
            audioData[i] = sample / 32768f;  // normalise to -1.0 to 1.0
        }
    }

    AudioClip clip = AudioClip.Create("tts", totalSamples, channels, sampleRate, false);
    clip.SetData(audioData, 0);
    return clip;
}

Once we have the AudioClip, playing it is easy:

AudioSource commentarySource = GetComponent<AudioSource>();
commentarySource.clip = clip;
commentarySource.Play();

📋 Full Code: WatsonTTSService.cs

That is all the code. Now let's assemble the parts. Note, this guide is not comprehensive and there are chunks of code left out. If you don't want to make a similar project but instead recreate the outcome, it is easier to go to the GitHub and clone the entirety of the code, from which point you may just attach the scripts onto the objects in Unity as described in the next sections.

GitHub Repo

10. Assembly

Adjusting the Track Model in Blender

We've got a 3D model of Donington Park, and it looks about right, but the exact dimensions are off. Since our car follows real GPS data, any mismatch between the model and reality will be visible. The car might drive off and clip through the grass.

Step 1: Export the Car's Path from Unity

We need to see where the car actually goes so we can compare against the track model. This script exports the telemetry path as an OBJ file which we will be able to import into Blender:

void ExportPathToOBJ(List<TelemetryFrame> frames, string filePath)
{
    StringBuilder obj = new StringBuilder();

    foreach (var frame in frames)
    {
        // Note: OBJ uses different axis conventions than Unity
        obj.AppendLine($"v {frame.position.x:F6} {frame.position.z:F6} {frame.position.y:F6}");
    }

    // Connect the dots with lines
    for (int i = 0; i < frames.Count - 1; i++)
    {
        obj.AppendLine($"l {i + 1} {i + 2}");
    }

    File.WriteAllText(filePath, obj.ToString());
}

Step 2: Overlay in Blender

By opening both the track and the car's path, we can start to modify the corners where there are slight inaccuracies.

image

Step 3: Adjust the Track

Using Blender's vertex editing tools:

  1. Switch to Edit Mode.
  2. Select vertices on the track that need shifting.
  3. Cross-reference against the onboard video footage to check your changes.

image

No need for micro adjustments just make sure the car stays on the track surface.

When it looks right, export the adjusted model as FBX and reimport into Unity to replace the old track.

Putting It All Together

Every piece is built. Now we just wire them up in Unity.

The Car GameObject

Select the car in the scene and attach these scripts:

  1. TelemetryReplay: drag the CSV file into the "Data Source" field in the Inspector.
  2. TelemetryPhysicsAnalyzer: set vehicleMass, tireFrictionCoefficient, and downforceCoefficient for the car class (GT3 car: roughly 1300 kg, 1.2 friction, 2.5 downforce).
  3. GForceTrailRenderer: automatically finds the physics analyser on the same object.

Also add three AudioSource components for the engine sounds and wire them into TelemetryReplay's audio fields.

The Track GameObject

Make sure the track has:

  • A MeshCollider (for raycasting to work).

The AI Manager GameObject

Select the AIManager empty object and attach:

  1. RaceStateAnalyzer: finds the TelemetryReplay automatically at runtime.
  2. CommentaryManager: wires up the state analyser to the API services.
  3. WatsonxService: put in your IBM watsonx API key and project ID.
  4. WatsonTTSService: put in your Watson TTS API key.
  5. An AudioSource for the commentary playback.

The Camera

Make the camera a child of the car object and position it slightly behind and above.

Hit Play

Press Play. You should see:

  • The car following the real telemetry.
  • A coloured trail showing grip usage.
  • Commentary describing the action on track.
  • Engine sounds.

What's Next?

You've got a working replay. Here are some ideas if you want to keep going:

  • Compare two laps. Load two CSV files to see where each driver gains time.
  • Add a minimap. Show a birds eye view of the circuit.
  • Make it an arcade racing game. Add a game mode to race against the real driver.

Everything is structured to be independent. You can swap them around and adjust the classes individually.

Happy racing!