Solving “Service Bus account connection string ‘ServiceBus’ does not exist. Make sure that it is a defined App Setting” when building Azure WebJobs

As always, this is a rambling rant about how/why I came across this solution. If you want to skip to the copy/paste, click here.

I used to love webjobs… they were great. I also used to love Azure Service Bus… it offered so many features, and Service Bus Explorer is fantastic for managing the queues (Not so much now that I use a mac 24/7 and there’s no cross platform port other than the basic one in the azure portal though).

The two are a match made in heaven… long running jobs that don’t suffer the issues that Azure Functions do with timeouts, proper compiled .Net code and easy enough to debug on your local machine as they are essentially just console apps.

But man, they seem forgotten about at Microsoft, the two combined are so useful, yet SO hard to configure. On a recent update to .Net 6 I also updated our dependencies with the dotnet outdated tool (You can see details here).

We used to have references to these packages:
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.ServiceBus" Version="3.0.3" /> <PackageReference Include="Microsoft.Azure.WebJobs.ServiceBus" Version="3.0.0-beta8" />

But after the upgrade, we had to come down to just this package to avoid some ambiguous references.
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.ServiceBus" Version="5.1.0" />

This change broke my fairly simple setup in my Program.cs file, where I set the connection string for the ServiceBus. This used to look like this:

builder.ConfigureWebJobs(b => 
{ 
    b.AddAzureStorageCoreServices();
    b.AddTimers();
    b.AddServiceBus(c => { c.ConnectionString =      config["ServiceBusConnectionString"]; }); }).ConfigureServices((context, services) => {
.. etc

Where “ServiceBusConnectionString” was just a normal property in my config file (which makes it really easy to swap out with a value from KeyVault when in the cloud, or from a K8S Secret… or a combination like we use)

Now, however, this looks like this

builder.ConfigureWebJobs(b => 
{ 
    b.AddAzureStorageCoreServices();
    b.AddTimers();
    b.AddServiceBus();
 }).ConfigureServices((context, services) => {
.. etc

With no specific way to define the ConnectionString… which isn’t the end of the world. The documentation tells us we just need to include it as a setting in the right place and we should be golden. So in your localsettings.json (or appsettings.local.json or whatever you load), you just need this setup

{
  "Values": {
    "<connection_name>": "Endpoint=sb://<service_bus_namespace>.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<access key>"
  }
}

Where you can have multiple connections, and specify which <connection_name> you wish to use in your web job definition. For example, this should use the value with the key “ServiceBus”:
public async Task HandleOne([ServiceBusTrigger("legacysendmail", Connection = "ServiceBus")] Message sendmailMessage)

So I tried that… and I get the error:
Service Bus account connection string 'ServiceBus' does not exist. Make sure that it is a defined App Setting.

Huh… weird. So I remove the “Values” as that’s an old fashioned thing, and the docs might not have been updated. Nope.

Well this is a connection string, maybe it needs to be under “ConnectionStrings“. I tried just having the “ServiceBus” key and the connection string as the value… still no dice.

I tried every permutation of this that I could figure out. The error occasionally jumped from looking for “ServiceBus” to “AzureWebJobsServiceBus” (As these seem to be two defaults from different areas of the code)

After a while, I literally tried this setup….

{
    "ServiceBusConnectionString": "Endpoint=sb://myservicebus.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=mykey",
    "ServiceBus": {
        "AzureWebJobsServiceBus": "Endpoint=sb://myservicebus.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=mykey"
    },
    "AzureWebJobsServiceBus": "Endpoint=sb://myservicebus.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=mykey",
    "ConnectionStrings": {
        "ServiceBus": {
            "AzureWebJobsServiceBus": "Endpoint=sb://myservicebus.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=mykey"
        },
        "AzureWebJobsServiceBus": "Endpoint=sb://myservicebus.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=mykey"
    },
    "Logging": { ... etc

There doesn’t seem to be any elaboration on the config file in the documents, but this is an open source project, and you can dig through the source yourself. I was able to find a small clue in their tests where they have pre loaded a dictionary with values and passed it through in some form of closure to some internal test “helper” methods. So I can’t really use those, but I am getting closer.

Using JetBrains Rider, I am able to step into the source for ServiceBusExtensions and see exactly where things are going wrong, it seems to be able to see my config settings, but it just doesn’t ever seem to find the value. The changing errors above (ServiceBus vs AzureWebJobsServiceBus) seem to imply that there’s something hitting, but just not clicking.

I am just wondering at this point, is the HostBuilder not able to see all of my IConfigurationRoot values? The documentation specifies “ApplicationSettings” when running in production, so is there a way to add app settings, to my HostBuilder? (Which presumably happens automatically in production) … it turns out there is.

Before I run the bit of code HostBuilder.ConfigureWebJobs() I can actually force in my own ‘in memory’ Application settings (sort of like their tests did) and this seems to get through to the new packages, and my application connects to service bus. (The bonus being I can roll my appconfig.json back to the way it was, as I am again, picking the variable myself)

So…. long story long, BEFORE you call HostBuilder.ConfigureWebJobs call HostBuilder.ConfigureAppConfiguration and add the value in yourself. The host definitely sees this, and it all seems to work.

//This is just an insane way of configuring the settings
builder.ConfigureAppConfiguration(x =>
    x.Add(new MemoryConfigurationSource()
    {
        InitialData = new Dictionary<string, string>
        {
            {$"AzureWebJobsServiceBus", connectionString }
        }
    }));

config = configBuilder.Build();

builder.ConfigureWebJobs(b =>
{
    b.AddAzureStorageCoreServices();
    b.AddTimers();
    b.AddServiceBus();
    ... etc