[BUG] Memory leak issue, when using linq2db efcore library 7.7.0

[aloksharma1]

Hello Guys,

in the version 7.7.0 of linq2db.EntityFrameworkCore, main library version 5.4.0 with ef core 7.0.20. I am seeing a sudden surge and retention of objects in linq2db memory cache for much longer than required and it's not disposing correctly.

if I remove linq2db implementation from that part of code it stops leaking memory. I can confirm this issue was not present in my project with previous versions as its an existing project and we just did a library version update and nothing else.
i did a memory diagnostic too and saw that each of the snapshot linq2db was holding big objects (14-16 mb per request).
its an mvc application, i cant share the code but i think if you can make a complex 3 level table and do merge/complex crud/union on it you will be able to see it.

@MaceWindu @sdanyliv

[MaceWindu]

Could you provide memory profiler screen(s) which show which objects not released and their GC root?

[aloksharma1]

Could you provide memory profiler screen(s) which show which objects not released and their GC root?

ok i will do that tomorrow, right now i have set app to restart in iis if it hits <3gb.

[aloksharma1]

@MaceWindu here is a few screenshots



i also have a full report in .diagsession format (for vs)

with each http request my application memory swells around 40-80 mb until it hits the celling and restarts, this issue was not present in previous version. my app usually stayed at 400-500mb no matter the traffic.

Note: same code is working without issues on another app instance in iis which is using version 7.5.0 of linq2db.EntityFrameworkCore and internally its linq2db version 5.2.1.0 (if it helps) that application hardly hits 650mb in peak traffic.

[MaceWindu]

It looks like some big object catched as a part of parameter of cached query. We made additional fixes in this area in v6 release of linq2db to not cache parameter values, but this release is not ready yet.

There are two options you could try:

1. You should check how your query uses parameters and see if you can change it.

E.g. if you use member of object to get parameter value - whole object could be catched by cache:

// this is bad
where entity.Id = someObject.IdValue

// this will cache only IdValue
// "detach" parameter value from object
var id = someObject.IdValue;
where entity.Id = id

note that this case also applies to closures, which are not so visible as direct object access:

// id - some variable in outer score
// bigObject - another big  variable, not even used by query, but used by lambda
ExecuteAction(() =>
{
    bigObject.CallMethod();
    [this.]ClassMethod();
    db.Table.Where(t => t.Id == id).Single();
});

This code will allocate closure object { id, bigObject, this, db } which will be saved in query cache

2. Release query cache for specific query result type after query executed. Could impact performance of frequent queries.

Query<BlogPostInfo>.ClearCache();

[aloksharma1]

@MaceWindu ok i am going to try your suggestions, However, I do believe that the current approach of caching based on object parameters, as in the case of where entity.Id = someObject.IdValue, poses significant issues for users working with complex data structures, especially when using DTO objects for querying and filling data. DTOs are commonly used in enterprise applications where querying and manipulating complex data structures is routine, and inadvertently caching entire DTOs can result in unnecessary memory retention and long-term performance degradation.

This pattern of caching entire objects—when the query only requires a subset of their properties—has broader implications. It may lead to memory bloat, as large objects (or even object graphs) could end up being cached inadvertently, increasing the risk of memory leaks, particularly in high-traffic applications or those that rely on extensive query generation. For instance, capturing DTOs or other complex objects could lead to heavy memory usage over time, as you've noted, even when only specific values like Id are needed.

So my suggestion is either make an extra method like .NoCache() or .DisableCache() or a parameter in TolinqToDB() when executing query to not do explicit caching or make it part of config. In my case since i am using linq2db.efcore bridge i think global config won't be good, I will need a local level toggle.

[MaceWindu]

There is no need to add extra API. Current situation when parameter value is cached is a bug (probably regression in recent releases) and we expect it fixed in v6

[aloksharma1]

There is no need to add extra API. Current situation when parameter value is cached is a bug (probably regression in recent releases) and we expect it fixed in v6

ok thanks for the prompt reply, but now i have to apply this glue in my mega monolith ;)

[aloksharma1]

one more question, does this bug affect update or merge queries? i need to know which places i need to call clearcache method.

[MaceWindu]

Cannot say for sure. I would recomment to inspect profiler report to see which query keeps reference to unnecessary data

[aloksharma1]

ok with further testing, i can still see linq2db accumulating memory but now with query.clearcaches its not happening immediately but its still holding big objects in long run and overflowing. i think all i can do now is to wait for next version to fix it.