fix: The relation between the source and the listener is unclear

[chrisnorman7]

When moving the listener position, the location of sound handles seems to jump around unpredictably.

import 'dart:math';

import 'package:flutter/material.dart';
import 'package:flutter_soloud/flutter_soloud.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // TRY THIS: Try running your application with "flutter run". You'll see
        // the application has a purple toolbar. Then, without quitting the app,
        // try changing the seedColor in the colorScheme below to Colors.green
        // and then invoke "hot reload" (save your changes or press the "hot
        // reload" button in a Flutter-supported IDE, or press "r" if you used
        // the command line to start the app).
        //
        // Notice that the counter didn't reset back to zero; the application
        // state is not lost during the reload. To reset the state, use hot
        // restart instead.
        //
        // This works for code too, not just values: Most code changes can be
        // tested with just a hot reload.
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  /// Create an instance.
  const MyHomePage({
    super.key,
  });

  /// Create state for this widget.
  @override
  MyHomePageState createState() => MyHomePageState();
}

/// State for [MyHomePage].
class MyHomePageState extends State<MyHomePage> {
  /// The current coordinates.
  late Point<double> coordinates;

  /// The current sound handle.
  SoundHandle? _handle;

  /// Initialise state.
  @override
  void initState() {
    super.initState();
    coordinates = const Point<double>(0.0, 0.0);
  }

  /// Build a widget.
  @override
  Widget build(BuildContext context) {
    final soLoud = SoLoud.instance;
    return Scaffold(
      appBar: AppBar(
        title: Text('Example'),
      ),
      body: Column(
        children: [
          if (_handle == null)
            TextButton(
              onPressed: () async {
                await soLoud.init();
                soLoud.set3dListenerPosition(coordinates.x, coordinates.y, 0);
                soLoud.set3dListenerAt(0, 0, -1);
                soLoud.set3dListenerUp(0, 1, 0);
                final source = await soLoud.loadAsset('sounds/loop.wav');
                final handle = await soLoud.play3d(
                  source,
                  5.0,
                  4.0,
                  3.0,
                  looping: true,
                );
                soLoud.setInaudibleBehavior(handle, true, false);
                if (mounted) {
                  setState(() {
                    _handle = handle;
                  });
                }
              },
              autofocus: true,
              child: Text('Play sound'),
            )
          else
            Focus(
              autofocus: true,
              child: Text(coordinates.toString()),
            ),
          Center(
            child: TextButton(
              onPressed: () {
                coordinates = Point(coordinates.x, coordinates.y + 1);
                SoLoud.instance
                    .set3dListenerAt(coordinates.x, coordinates.y, 0.0);
                setState(() {});
              },
              child: Text('Move north'),
            ),
          ),
          Row(
            children: [
              TextButton(
                onPressed: () {
                  coordinates = Point(coordinates.x - 1, coordinates.y);
                  SoLoud.instance
                      .set3dListenerAt(coordinates.x, coordinates.y, 0.0);
                  setState(() {});
                },
                child: Text('Move west'),
              ),
              TextButton(
                onPressed: () {
                  coordinates = Point(coordinates.x + 1, coordinates.y);
                  SoLoud.instance
                      .set3dListenerAt(coordinates.x, coordinates.y, 0.0);
                  setState(() {});
                },
                child: Text('Move east'),
              )
            ],
          ),
          Center(
            child: TextButton(
              onPressed: () {
                coordinates = Point(coordinates.x, coordinates.y - 1);
                SoLoud.instance
                    .set3dListenerAt(coordinates.x, coordinates.y, 0.0);
                setState(() {});
              },
              child: Text('Move south'),
            ),
          ),
        ],
      ),
    );
  }
}

When I run this code, the sound starts off to the right (and hopefully slightly in front, although that could be confirmation bias), then moves to the centre. When I move east, the sound goes to the right and west to the left which seems backwards.

Now I see from the website that the 3d stuff uses OpenAl algorithms, so it's entirely possible I'm setting this up wrong. If that's the case, can you direct me somewhere a non-maths person can copy-and-paste code to swirl the listener around according to simple things like angles rather than vectors?

Sorry there's no accompanying video. Microsoft GameBar refused to assist.

Cheers,

[alnitak]

Hi @chrisnorman7,

the 3d stuff uses OpenAl algorithms

I think you mean OpenGL coordinate system which is right handed ;)

Here the SoLoud lib documentation for 3D sound for further information, even if I think you read it already.

The set3dListenerAt will let the listener look at the sound, so imagine a fly buzzing around you. If you keep looking at it you won't have the feeling that its noise comes from the right or left, while if you stay still and the fly passes by your right ear you will understand that it is on your right (try to uncomment SoLoud.instance.set3dListenerAt in the code below).

So for this test, I just move the position of the sound and let the listener stay still "looking" always forward (the same could be done by moving the listener).

I would also tend to use Vector3 instead of Point (because for example, the fly can also move vertically).

Unfortunately, I think you should manage both vectors and angles :)

code (add vector_math: ^2.1.4 to pubspec)

import 'package:flutter/material.dart';
import 'package:flutter_soloud/flutter_soloud.dart';
import 'package:vector_math/vector_math_64.dart' as v;
import 'dart:math' show pi;

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  /// Create an instance.
  const MyHomePage({
    super.key,
  });

  /// Create state for this widget.
  @override
  MyHomePageState createState() => MyHomePageState();
}

/// State for [MyHomePage].
class MyHomePageState extends State<MyHomePage> {
  /// The current listener coordinates.
  late v.Vector3 listenerCoord;

  /// The current sound coordinates.
  late v.Vector3 soundCoord;

  /// The current sound handle.
  SoundHandle? _handle;

  final center = v.Vector3.zero();
  final north = v.Vector3(0, 1, 0);
  final west = v.Vector3(-1, 0, 0);
  final east = v.Vector3(1, 0, 0);
  final south = v.Vector3(0, -1, 0);

  /// Initialise state.
  @override
  void initState() {
    super.initState();
    listenerCoord = center;
    soundCoord = center;
  }

  void updateSoundCoordinates() {
    SoLoud.instance.set3dSourcePosition(
      _handle!,
      soundCoord.x,
      soundCoord.y,
      soundCoord.z,
    );

    /// Let the listener "look" at the sound.
    /// This makes the sound to be always in front of the listener and I think
    /// it's not what you want.
    // SoLoud.instance.set3dListenerAt(soundCoord.x, soundCoord.y, soundCoord.z);
  }

  /// Rotate the sound by [angleDegrees] degrees with [pivot] as origin.
  void rotateAroundPoint(v.Vector3 pivot, double angleDegrees) {
  // Convert angle to radians (15 degrees clockwise)
  final angleRadians = v.radians(-15); // Negative for clockwise rotation

  // Translate point to origin
  final translated = soundCoord - pivot;

  // Create rotation matrix
  final rotationMatrix = Matrix4.rotationZ(angleRadians);

  // Rotate point
  final rotated = rotationMatrix.transform3(translated);

  // Translate back to original position
  soundCoord = rotated + pivot;

  updateSoundCoordinates();
}

  /// Build a widget.
  @override
  Widget build(BuildContext context) {
    final soLoud = SoLoud.instance;
    return Scaffold(
      appBar: AppBar(
        title: const Text('Example'),
      ),
      body: Column(
        children: [
          if (_handle == null)
            TextButton(
              onPressed: () async {
                await soLoud.init();
                soLoud
                  ..set3dListenerPosition(
                    listenerCoord.x,
                    listenerCoord.y,
                    listenerCoord.z,
                  )
                  ..set3dListenerAt(
                    soundCoord.x,
                    soundCoord.y,
                    soundCoord.z,
                  )
                  ..set3dListenerUp(0, 1, 0);
                final source =
                    await soLoud.loadAsset('assets/audio/IveSeenThings.mp3');
                final handle = await soLoud.play3d(
                  source,
                  soundCoord.x,
                  soundCoord.y,
                  soundCoord.z,
                  looping: true,
                );
                soLoud.setInaudibleBehavior(handle, true, false);
                if (mounted) {
                  setState(() {
                    _handle = handle;
                  });
                }
              },
              autofocus: true,
              child: const Text('Play sound'),
            )
          else
            Focus(
              autofocus: true,
              child: Text('listener: $listenerCoord\nsound: $soundCoord'),
            ),
          const SizedBox(height: 30),
          Center(
            child: TextButton(
              onPressed: () {
                /// Move the sound to the north.
                soundCoord = north;
                updateSoundCoordinates();

                setState(() {});
              },
              child: const Text('Move north'),
            ),
          ),
          Row(
            mainAxisSize: MainAxisSize.min,
            children: [
              TextButton(
                onPressed: () {
                  /// Move the sound to the west.
                  soundCoord = west;
                  updateSoundCoordinates();
                  setState(() {});
                },
                child: const Text('Move west'),
              ),
              TextButton(
                onPressed: () {
                  /// Move the sound to the east.
                  soundCoord = east;
                  updateSoundCoordinates();
                  setState(() {});
                },
                child: const Text('Move east'),
              ),
            ],
          ),
          Center(
            child: TextButton(
              onPressed: () {
                /// Move the sound to the south.
                soundCoord = south;
                updateSoundCoordinates();
                setState(() {});
              },
              child: const Text('Move south'),
            ),
          ),
          const SizedBox(height: 30),
          Center(
            child: TextButton(
              onPressed: () {
                rotateAroundPoint(center, 15.0 * pi / 180.0); // 15 degrees
                setState(() {});
              },
              child: const Text('Rotate listener'),
            ),
          ),
        ],
      ),
    );
  }
}

[chrisnorman7]

@alnitak
Thank you so much for your response and the code.

This moves the sound how I'd like. How can I rotate the listener instead please? The head will never tilt, so it's just swivelling on the neck.

I'm sorry this has turned more into a maths question than a SoLoud issue. Feel free to tell me to JFGI.

Thanks for your help and patience.

[alnitak]

I think that if you swap listenerCoord and soundCoord in my code, you will get what you want. Doing so, listenerCoord will move and the sound remains still. I think the effect will be the same, but usually, you have to think that it is the sound to move relatively to your head.

Glad to be of any help!

[chrisnorman7]

What about when your player is moving through a space and walking past sound sources? What about when the player is moving past sound sources and there are other sound sources moving in the opposite direction?

[alnitak]

Well, everything is relative (cit. :) ). Anyway, it all depends on what you are trying to do and how you are feeling with 3D space. I would suggest replicating what the environment is and if the listener is moving, you should move it according, and the same for the sound around. I think your example is a good start to experiencing this.