👂🏽 Unreal Engine AI Hearing Perception

In this post, we see some example UE5 AI hearing C++ code. This can be used to have AI non-player characters (NPCs), in your game, react to audio stimuli. I used this code in an Unreal Engine game I created better to understand AI in Unreal. In the game, the player is a special agent on a hostile site, searching for some sensitive documents. The player wants to avoid getting caught by hostile AI NPCs on-site, and uses a drone to check when it is safe to move.

To make the game a little more difficult, I wanted the drone to emit a sound whenever it climbs, which the AI characters would be able to perceive and react to. That is where this code came in.

Learning in Public

I am learning Unreal Engine 5, and have found, generally, there is much more content on how to add features using blueprints, than using C++. Luckily, I found a fantastic and up-to-date UE5 C++ AI tutorial (link below), which covered the basics and sight perception. I used this sight perception code to have the hostile NPCs react to seeing the player, and extended it, adding audio perception, to produce the C++ code example below.

I hope you will find this useful. Also, please let me know if you value this style of learning in public content, especially if there are other topics you would like to see some content on.

🌟 Unreal Engine 5 Learning Resources

I mentioned, Unreal Engine 5 C++ tutorials are harder to come across. There is some UE4 C++ content, though this often needs a lot of work to update it for use in UE5. On AI, I did find some fantastic resources though:

• UE5 C++ AI Series by Mr Cxx: YouTube series of 25 videos, which start from the basics, using blackboards and behaviour trees, to patrol routes and more.
• Learn all About AI in Unreal Engine 5 by Ryan Laley: Ryan focuses on Unreal Engine Editor blueprints, rather than C++, though the quality of the tutorials is right up there. If you prefer to set things up in blueprints, before trying C++, start here!

The official Unreal docs are also helpful for tweaking configurations.

I also found another series of tutorials on Unreal generally. I got inspiration for the drone from this series. Unfortunately, these Reids Channel tutorials are a little older, and were created for Unreal Engine 4. That said, they are still worth a look, because the instruction is of such high quality.

🖥️ UE5 AI Hearing C++ Example Code

My starting point for this code was the MrCxx UE5 C++ tutorial linked above. I added the hearing perception to the NPC_AIController CPP class, first adding a hearing perception field to the class in Source/Game/NPC_AIController.h:

#pragma once

#include "AIController.h"
#include "CoreMinimal.h"

#include "NPC_AIController.generated.h"

UCLASS()
class AIGAME_API ANPC_AIController : public AAIController {
  GENERATED_BODY()

public:
  explicit ANPC_AIController(FObjectInitializer const &ObjectInitializer);

protected:
  void OnPossess(APawn *InPawn) override;

private:
  class UAISenseConfig_Sight *SightConfig;
  class UAISenseConfig_Hearing *HearingConfig;

  void SetupPerceptionSystem();

  UFUNCTION()
  void OnTargetDetected(AActor *Actor, FAIStimulus const Stimulus);
};

Then, I updated the SetupPerceptionSystem method definition in the C++ counterpart, adding hearing perception:

// ...TRUNCATED

void ANPC_AIController::SetupPerceptionSystem() {
  SightConfig =
      CreateDefaultSubobject<UAISenseConfig_Sight>(TEXT("Sight Config"));
  HearingConfig =
      CreateDefaultSubobject<UAISenseConfig_Hearing>(TEXT("Hearing Config"));

  if (SightConfig != nullptr || HearingConfig != nullptr) {
    SetPerceptionComponent(*CreateDefaultSubobject<UAIPerceptionComponent>(
        TEXT("Perception Component")));
  }

  if (SightConfig != nullptr) {
    SightConfig->SightRadius = 500.F;
    SightConfig->LoseSightRadius = SightConfig->SightRadius + 25.F;
    SightConfig->PeripheralVisionAngleDegrees = 90.F;
    SightConfig->SetMaxAge(
        5.F); // seconds - perceived stimulus forgotten after this time
    SightConfig->AutoSuccessRangeFromLastSeenLocation = 520.F;
    SightConfig->DetectionByAffiliation.bDetectEnemies = true;
    SightConfig->DetectionByAffiliation.bDetectFriendlies = true;
    SightConfig->DetectionByAffiliation.bDetectNeutrals = true;

    GetPerceptionComponent()->SetDominantSense(
        *SightConfig->GetSenseImplementation());
    GetPerceptionComponent()->ConfigureSense(*SightConfig);
  }

  if (HearingConfig != nullptr) {
    HearingConfig->HearingRange = 10000.F;
    HearingConfig->SetMaxAge(5.F);
    HearingConfig->DetectionByAffiliation.bDetectEnemies = true;
    HearingConfig->DetectionByAffiliation.bDetectFriendlies = true;
    HearingConfig->DetectionByAffiliation.bDetectNeutrals = true;

    GetPerceptionComponent()->ConfigureSense(*HearingConfig);
  }

  if (SightConfig != nullptr || HearingConfig != nullptr) {
    GetPerceptionComponent()->OnTargetPerceptionUpdated.AddDynamic(
        this, &ANPC_AIController::OnTargetDetected);
  }
}

Finally, in this file, I updated the OnTargetDetected method, to register the sound stimulus as detected:

// ...TRUNCATED

void ANPC_AIController::OnTargetDetected(AActor *Actor,
                                         FAIStimulus const Stimulus) {
  if (auto *const player_character = Cast<AAIGameCharacter>(Actor)) {
    GetBlackboardComponent()->SetValueAsBool("CanSeePlayer",
                                             Stimulus.WasSuccessfullySensed());
  } else
  // check if the actor is the player drone
  if (Actor->GetActorNameOrLabel() == "BP_Drone") {
    {
      GetBlackboardComponent()->SetValueAsBool(
          "CanSeePlayer", Stimulus.WasSuccessfullySensed());
    }
  }
}

The hearing configuration is not too different to sight, just with fewer parameters. You can see the full hearing config in the UAISenseConfig_Hearing docs.

🙉 Updating the AI Character to Receive Audio Stimuli

One final step, similar to sight perception, is to register the character to receive audio stimuli. For this, I updated the SetupStimulusSource method in Source/Game/AIGameCharacter.cpp:

// ...TRUNCATED

void AAIGameCharacter::SetupStimulusSource() {
  StimulusSource = CreateDefaultSubobject<UAIPerceptionStimuliSourceComponent>(
      TEXT("Stimulus"));
  if (StimulusSource) {
    StimulusSource->RegisterForSense(TSubclassOf<UAISense_Sight>());
    StimulusSource->RegisterForSense(TSubclassOf<UAISense_Hearing>());
    StimulusSource->RegisterWithPerceptionSystem();
  }
}

🙌🏽 UE5 AI Hearing C++: Wrapping Up

In this UE5 AI Hearing C++ post, we saw a code example for adding audio perception to your Unreal NPC. More specifically, we saw:

• some links to tutorials for learning Unreal Engine 5 AI using blueprints and C++;
• example C++ code for adding AI hearing perception; and
• C++ code for registering a character to receive audio stimuli.

Do let me know if you found this content useful and would like to see more similar content. Also reach out if there is anything I could improve to provide a better experience for you.

🙏🏽 UE5 AI Hearing C++: Feedback

If you have found this post useful, see links below for further related content on this site. Let me know if there are any ways I can improve on it. I hope you will use the code or starter in your own projects. Be sure to share your work on X, giving me a mention, so I can see what you did. Finally, be sure to let me know ideas for other short videos you would like to see. Read on to find ways to get in touch, further below. If you have found this post useful, even though you can only afford even a tiny contribution, please consider supporting me through Buy me a Coffee.

Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via @askRodney on X (previously Twitter) and also, join the #rodney Element Matrix room. Also, see further ways to get in touch with Rodney Lab. I post regularly on Game Dev as well as Rust and C++ (among other topics). Also, subscribe to the newsletter to keep up-to-date with our latest projects.