SIM-LDM

Overview

This project is an extension of the autonomous driving simulator "AWSIM".
It enables the extraction of various in-simulator data using scripts in the Assets/AWSIM/Managers directory, and sends the data to an external server via socket communication.

---

External Server Integration Architecture

In this project, data extracted from AWSIM is first received by a Python script (acting as a broker) in the LDM-Server. This script then sends the data to a Kafka topic on the Confluent Platform. The data sent to the Kafka topic is stream-processed using ksqlDB.

Architecture Diagram

[AWSIM: LDM Managers]
    |
    | (socket communication)
    v
[Python Script: Data Broker]
    |
    | (Kafka Topic)
    v
[Confluent Platform(ksqlDB): LDM-Server]
    ^
    | (HTTP/HTTPS)
    v
[External Application: Dashboard / Analytics / Visualization, etc.]

---

Features

• Real-time extraction of various AWSIM simulator data (e.g., vehicle state, sensor information)
• Data transmission to external servers via socket communication (TCP/UDP)
• Extraction and transmission logic implemented in scripts under Assets/AWSIM/Managers
• Automatic generation of Local Dynamic Map (LDM) using simulator data

---

Directory Structure

AWSIM_for_git/
├── Assets/
│   └── AWSIM/
│       └── Managers/   # Scripts for data extraction and transmission
├── docs/               # Documentation
├── ProjectSettings/
├── Packages/
├── ...

---

Setup

1. Setup according to the official AWSIM instructions

   • First, follow the official AWSIM documentation to set up Unity, dependencies, ROS2, etc.

2. Open this project in Unity

   • Open this project (this repository) in Unity Hub (Recommended version: Unity 2021.1.7f1).
   • Check Packages/manifest.json and install required dependencies if needed.
   • The ROS2 distribution is specified in .vscode/settings.json (e.g., humble). Change as needed.

3. Set up Confluent Platform

   • Refer to the section "Confluent Platform Account Creation and On-Premises Installation Steps" below and set up Confluent Platform.

---

Confluent Platform Setup

1. Access the Confluent Platform official site and create an account.

   • You can also sign up with your Google or GitHub account.

2. Install the on-premises (self-managed) version:

   • Download the installer by selecting "Self-Managed" or "On-Premises" from the Confluent Platform download page.
   • Extract the tar.gz or zip file according to your OS.

3. Start the required services (ZooKeeper, Kafka, ksqlDB, etc.):

   • Basically, you can start all major services with the following single command:

confluent local services start

• If you want to start them individually, you can also start each service manually as follows:

# Start ZooKeeper
bin/zookeeper-server-start etc/kafka/zookeeper.properties

# Start Kafka broker (in another terminal)
bin/kafka-server-start etc/kafka/server.properties

# Start ksqlDB server (if needed)
bin/ksql-server-start etc/ksqldb/ksql-server.properties

4. Using Kafka and ksqlDB:

   • You can create Kafka topics, send data, and perform stream processing with ksqlDB.

5. For detailed steps and the latest information, please refer to the official documentation:

   • Confluent Official Guide

Usage

1. Run the Python file (broker script) in LDM-Server
2. Play the scene in the Unity Editor
3. The script in Assets/AWSIM/Managers automatically extracts data and sends it to the Python script in the LDM-Server
4. The Python script forwards the received data to a Kafka topic on the Confluent Platform
5. The data in the Kafka topic is stream-processed by ksqlDB
6. The server receives and utilizes the data

---

License

• Code: Apache License 2.0
• Assets: CC BY-NC

See LICENSE for details.

---

Contribution

Bug reports, feature requests, and pull requests are welcome.

---

Acknowledgement

• This project is an extension based on TIER IV AWSIM.

---