Why doesn't AddConfigStoredInConsul() work as normal Consul config?

[ArchitectAllenGitHub]

Expected Behavior / New Feature

The same configuration is used. AddConfigStoredInConsul() request 404, Normal if not in use.
Program.cs It is shown below:

using Ocelot.DependencyInjection;
using Ocelot.Middleware;
using Ocelot.Provider.Consul;

var builder = WebApplication.CreateBuilder(args);
builder.Configuration.AddJsonFile("ocelot.json");
builder.Services
    .AddOcelot()
    .AddConsul()
    .AddConfigStoredInConsul();

var app = builder.Build();

app.UseOcelot();

app.Run();

ocelot.json It is shown below

{
"Routes": [
    {
      "UpstreamPathTemplate": "/Ocelot/{url}",
      "UpstreamHttpMethod": [
        "Get",
        "Post"
      ],
      "DownstreamPathTemplate": "/{url}",
      "DownstreamScheme": "http",
      "ServiceName": "MagicCore.API",
      "LoadBalancerOptions": {
        "Type": "RoundRobin"
      }
    }
  ],
  "GlobalConfiguration": {
    "ServiceDiscoveryProvider": {
      "Host": "localhost",
      "Port": 8500
    }
  }
}

[ggnaegi]

Hello @ArchitectAllenGitHub could you provide more details? Thanks!

[raman-m]

Hello @ArchitectAllenGitHub could you provide more details? Thanks!

@ggnaegi Look at "ServiceDiscoveryProvider" options in global config! They have no a Type property defined!
I would say, this is border conditions case when no Type in the options, but developer has added Consul via AddConsul.
We should substitute default type of discovery provider: Consul or PollConsul.
To be discussed...

[raman-m]

Second, our docs for Load Balancer says nothing about coupling Service Discovery vs Load Balancer. The author was confused. We don't explain correct instructions...

[raman-m]

.AddConfigStoredInConsul();

What's the strange method?
Is it from Ocelot code base?
Or, is it a custom author's extension?

[ggnaegi]

.AddConfigStoredInConsul();

What's the strange method?
Is it from Ocelot code base?
Or, is it a custom author's extension?

@raman-m it's for the consul key value store. You could have dynamic configuration, the ocelot config being stored in consul. It starts an IHostedService instance called FileConfigurationPoller.
https://developer.hashicorp.com/consul/docs/dynamic-app-config/kv
and the funny thing, I'm using it... 😹

public static IOcelotBuilder AddConsul(this IOcelotBuilder builder)
    {
        builder.Services
            .AddSingleton(ConsulProviderFactory.Get)
            .AddSingleton<IConsulClientFactory, ConsulClientFactory>()
            .RemoveAll(typeof(IFileConfigurationPollerOptions))
            .AddSingleton<IFileConfigurationPollerOptions, ConsulFileConfigurationPollerOption>();
        return builder;
    }

    public static IOcelotBuilder AddConfigStoredInConsul(this IOcelotBuilder builder)
    {
        builder.Services
            .AddSingleton(ConsulMiddlewareConfigurationProvider.Get)
            .AddHostedService<FileConfigurationPoller>()
            .AddSingleton<IFileConfigurationRepository, ConsulFileConfigurationRepository>();
        return builder;
    }

[ArchitectAllenGitHub]

@ggnaegi @raman-m
Thank you for your help.Here's my demo
https://github.com/ArchitectAllenGitHub/OcelotDemo

[raman-m]

Uploading dll files and executable ones is very bad!!!
We are not going to run this exe!

You have to upload source code as solid solution!

[ggnaegi]

@ArchitectAllenGitHub could you try a config like this one? like @raman-m mentioned we might have a problem with the configuration since the ServiceDiscoveryProvider type is not defined.

"GlobalConfiguration": {
    "ServiceDiscoveryProvider": {
      "Host": "consul",
      "Port": 8500,
      "Type": "PollConsul",
      "PollingInterval": 3600000
    }
  }

Ok, I'm using a docker environment, so Host would be localhost or 127.0.0.1 for you ;-)

[ArchitectAllenGitHub]

你能试试这样的配置吗？如前所述，由于未定义 ServiceDiscoveryProvider 类型，因此我们的配置可能存在问题。

"GlobalConfiguration": {
    "ServiceDiscoveryProvider": {
      "Host": "consul",
      "Port": 8500,
      "Type": "PollConsul",
      "PollingInterval": 3600000
    }
  }

好的，我正在使用 docker 环境，所以 Host 将是 localhost 或 127.0.0.1 ;-）

Following the configuration above, I added the Type and PollingInterval. All else being equal.
The AddConfigStoredInConsul() method is not used

Use the AddConfigStoredInConsul() method


The specific json in Consul is as follows

{
  "Routes": [
    {
      "DownstreamPathTemplate": "/{url}",
      "UpstreamPathTemplate": "/Ocelot/{url}",
      "UpstreamHttpMethod": [
        "Get",
        "Post"
      ],
      "DownstreamHttpMethod": null,
      "AddHeadersToRequest": {},
      "UpstreamHeaderTransform": {},
      "DownstreamHeaderTransform": {},
      "AddClaimsToRequest": {},
      "RouteClaimsRequirement": {},
      "AddQueriesToRequest": {},
      "ChangeDownstreamPathTemplate": {},
      "RequestIdKey": null,
      "FileCacheOptions": {
        "TtlSeconds": 0,
        "Region": null
      },
      "RouteIsCaseSensitive": false,
      "ServiceName": "MagicCore.API",
      "ServiceNamespace": null,
      "DownstreamScheme": "http",
      "QoSOptions": {
        "ExceptionsAllowedBeforeBreaking": 0,
        "DurationOfBreak": 0,
        "TimeoutValue": 0
      },
      "LoadBalancerOptions": {
        "Type": "RoundRobin",
        "Key": null,
        "Expiry": 0
      },
      "RateLimitOptions": {
        "ClientWhitelist": [],
        "EnableRateLimiting": false,
        "Period": null,
        "PeriodTimespan": 0.0,
        "Limit": 0
      },
      "AuthenticationOptions": {
        "AuthenticationProviderKey": null,
        "AllowedScopes": []
      },
      "HttpHandlerOptions": {
        "AllowAutoRedirect": false,
        "UseCookieContainer": false,
        "UseTracing": false,
        "UseProxy": true,
        "MaxConnectionsPerServer": 2147483647
      },
      "DownstreamHostAndPorts": [],
      "UpstreamHost": null,
      "Key": null,
      "DelegatingHandlers": [],
      "Priority": 1,
      "Timeout": 0,
      "DangerousAcceptAnyServerCertificateValidator": false,
      "SecurityOptions": {
        "IPAllowedList": [],
        "IPBlockedList": [],
        "ExcludeAllowedFromBlocked": false
      },
      "DownstreamHttpVersion": null
    }
  ],
  "DynamicRoutes": [],
  "Aggregates": [],
  "GlobalConfiguration": {
    "RequestIdKey": null,
    "ServiceDiscoveryProvider": {
      "Scheme": null,
      "Host": "localhost",
      "Port": 8500,
      "Type": "PollConsul",
      "Token": null,
      "ConfigurationKey": null,
      "PollingInterval": 3600000,
      "Namespace": null
    },
    "RateLimitOptions": {
      "ClientIdHeader": "ClientId",
      "QuotaExceededMessage": null,
      "RateLimitCounterPrefix": "ocelot",
      "DisableRateLimitHeaders": false,
      "HttpStatusCode": 429
    },
    "QoSOptions": {
      "ExceptionsAllowedBeforeBreaking": 0,
      "DurationOfBreak": 0,
      "TimeoutValue": 0
    },
    "BaseUrl": null,
    "LoadBalancerOptions": {
      "Type": null,
      "Key": null,
      "Expiry": 0
    },
    "DownstreamScheme": null,
    "HttpHandlerOptions": {
      "AllowAutoRedirect": false,
      "UseCookieContainer": false,
      "UseTracing": false,
      "UseProxy": true,
      "MaxConnectionsPerServer": 2147483647
    },
    "DownstreamHttpVersion": null
  }
}

[ggnaegi]

Yeah, but wait a minute, do you have registered your services to consul? Otherwise it's useless... The idea here is the following, you register your services to consul and then you query consul for the service addresses. I have an example here: https://github.com/ggnaegi/SwizlyPeasy.Gateway/blob/master/SwizlyPeasy.Consul/README.md
Then, you can register several service instances to consul and then it will allow the load balancing.

But, @ArchitectAllenGitHub you are right, there is something not consistent here...

[ArchitectAllenGitHub]

Yeah, but wait a minute, do you have registered your services to consul? Otherwise it's useless... The idea here is the following, you register your services to consul and then you query consul for the service addresses. I have an example here: https://github.com/ggnaegi/SwizlyPeasy.Gateway/blob/master/SwizlyPeasy.Consul/README.md Then, you can register several service instances to consul and then it will allow the load balancing.

But, @ArchitectAllenGitHub you are right, there is something not consistent here...

The service is registered, Ocelot can be forwarded normally without adding the AddConfigStoredInConsul() method. Add this method and it's 404, which is very strange

[raman-m]

@ArchitectAllenGitHub
Disable AddConfigStoredInConsul() for now, remove it. And let's check classic setup of Consul config...

Show us what have you registered in Consul Services?

And, make screenshots of response & HTTP status code of direct connection from Ocelot's machine in Postman tool to your downstream service MagicCore.API please.

What is your Ocelot's machine? How did you deploy the machine? How do you host Ocelot app? Is it deployed to Docker image/container or not?

[ggnaegi]

@ArchitectAllenGitHub It's very strange, and even more for me since i'm using this in production... 😆
@ArchitectAllenGitHub And you are starting your ocelot instance on port 8000, right?

[ArchitectAllenGitHub]

yes,Port 8000

[ArchitectAllenGitHub]

@ArchitectAllenGitHub Disable AddConfigStoredInConsul() for now, remove it. And let's check classic setup of Consul config...

Show us what have you registered in Consul Services?

And, make screenshots of response & HTTP status code of direct connection from Ocelot's machine in Postman tool to your downstream service MagicCore.API please.

What is your Ocelot's machine? How did you deploy the machine? How do you host Ocelot app? Is it deployed to Docker image/container or not?

I'm running on Windows.
This is the configuration of the Consul


This is the case with the request through the Ocelot gateway, you can see if it is clear enough

[raman-m]

🆗 Thank you! It seems classic Consul setup is working.
You can re-enable mentioned method call in DI and try to configure...
Let us know a feedback please...

[ArchitectAllenGitHub]

What I'm offering is what you'd like to see? The logs do not have the requested information

[raman-m]

What I'm offering is what you'd like to see? The logs do not have the requested information

Can you debug all extension helpers included to the AddConfigStoredInConsul() ?

We have no other options to help you...
We have to debug the solution...
I have no time for that.
I hope, you and Gui will find the root cause in debugging sessions... Good luck!

[ArchitectAllenGitHub]

Ok, thanks for the help

[raman-m]

We will continue our nice discussion tomorrow...

[ggnaegi]

@ArchitectAllenGitHub I will try to reproduce your config and come back to you. ..

[raman-m]

Let's find a root cause together!....
I want to make sure this service extension works fine ..
In case of found bug, we will open a bug-issue...

[ggnaegi]

@ArchitectAllenGitHub do you have the source code of your APIs (or at least the Program.cs)? I'm really sorry, but I would like to avoid starting the executables you provided.

[raman-m]

@ArchitectAllenGitHub do you have the source code of your APIs (or at least the Program.cs)? I'm really sorry, but I would like to avoid starting the executables you provided.

@ggnaegi
Agreed! Stop helping him until the author will upload all source files as a solid workable solution!

Unfortunately, he has local environment only, we cannot use RDC to connect to remote machine and see the solution.

[ArchitectAllenGitHub]

Absolutely, this project includes the API and the Ocelot gateway
https://github.com/ArchitectAllenGitHub/MagicCore

[ggnaegi]

Ok great @ArchitectAllenGitHub thanks.

[ggnaegi]

@ArchitectAllenGitHub
So, I was a bit lazy, I haven't used your code, but I tried a similar configuration as yours and it's working fine.
Environment: Windows, Ocelot 20.0.0, RoundRobin load balancer, 2 instances of service SuperTestService.
That's an excerpt of the ocelot file stored in consul KV-Store:

{
      "DownstreamPathTemplate": "/test/v{version}/all-labels-and-options",
      "UpstreamPathTemplate": "/test/v{version}/all-labels-and-options",
      "UpstreamHttpMethod": [
        "GET"
      ],
      "DownstreamHttpMethod": null,
      "AddHeadersToRequest": {},
      "UpstreamHeaderTransform": {},
      "DownstreamHeaderTransform": {},
      "AddClaimsToRequest": {},
      "RouteClaimsRequirement": {},
      "AddQueriesToRequest": {},
      "ChangeDownstreamPathTemplate": {},
      "RequestIdKey": null,
      "FileCacheOptions": {
        "TtlSeconds": 0,
        "Region": null
      },
      "RouteIsCaseSensitive": false,
      "ServiceName": "SuperTestService",
      "ServiceNamespace": null,
      "DownstreamScheme": "http",
      "QoSOptions": {
        "ExceptionsAllowedBeforeBreaking": 0,
        "DurationOfBreak": 0,
        "TimeoutValue": 0
      },
      "LoadBalancerOptions": {
        "Type": "RoundRobin",
        "Key": null,
        "Expiry": 0
      },
      "RateLimitOptions": {
        "ClientWhitelist": [],
        "EnableRateLimiting": true,
        "Period": "1s",
        "PeriodTimespan": 1.0,
        "Limit": 5
      },
      "AuthenticationOptions": {
        "AuthenticationProviderKey": null,
        "AllowedScopes": []
      },
      "HttpHandlerOptions": {
        "AllowAutoRedirect": false,
        "UseCookieContainer": false,
        "UseTracing": false,
        "UseProxy": true,
        "MaxConnectionsPerServer": 2147483647
      },
      "DownstreamHostAndPorts": [],
      "UpstreamHost": null,
      "Key": null,
      "DelegatingHandlers": [],
      "Priority": 1,
      "Timeout": 0,
      "DangerousAcceptAnyServerCertificateValidator": false,
      "SecurityOptions": {
        "IPAllowedList": [],
        "IPBlockedList": [],
        "ExcludeAllowedFromBlocked": false
      },
      "DownstreamHttpVersion": null
    },
...
"DynamicRoutes": [],
  "Aggregates": [],
  "GlobalConfiguration": {
    "RequestIdKey": null,
    "ServiceDiscoveryProvider": {
      "Scheme": null,
      "Host": "consul",
      "Port": 8500,
      "Type": "PollConsul",
      "Token": null,
      "ConfigurationKey": null,
      "PollingInterval": 3600000,
      "Namespace": null
    },
    "RateLimitOptions": {
      "ClientIdHeader": "ClientId",
      "QuotaExceededMessage": null,
      "RateLimitCounterPrefix": "ocelot",
      "DisableRateLimitHeaders": false,
      "HttpStatusCode": 429
    },
    "QoSOptions": {
      "ExceptionsAllowedBeforeBreaking": 0,
      "DurationOfBreak": 0,
      "TimeoutValue": 0
    },
    "BaseUrl": null,
    "LoadBalancerOptions": {
      "Type": null,
      "Key": null,
      "Expiry": 0
    },
    "DownstreamScheme": null,
    "HttpHandlerOptions": {
      "AllowAutoRedirect": false,
      "UseCookieContainer": false,
      "UseTracing": false,
      "UseProxy": true,
      "MaxConnectionsPerServer": 2147483647
    },
    "DownstreamHttpVersion": null
  }

[raman-m]

Gui, you need to remove redundant properties in JSON, and the important & required ones should be left.

[ggnaegi]

@raman-m I know, it's the plain value from the kv-store, it's a bit strange to be honest, it's transformed when saved in consul.

[ArchitectAllenGitHub]

Can you give a correct json data?
And I'm running Consul in docker and it's still the same issue

[raman-m]

@ArchitectAllenGitHub
Dear architect from GitHub,
Cannot you compare two JSON blocks as file contents, copy from your block and paste to the second one?

[raman-m]

@ArchitectAllenGitHub
What's up, man?