Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory leak for simple callback #174

Open
hakonhagland opened this issue Aug 19, 2019 · 2 comments
Open

Memory leak for simple callback #174

hakonhagland opened this issue Aug 19, 2019 · 2 comments
Labels
🐣Enhancement Things that make it work better

Comments

@hakonhagland
Copy link
Contributor

hakonhagland commented Aug 19, 2019 •
edited
Loading

I have this simple test library in C lib/mylib.c:

#include <stdio.h>
void mylib_func (void (*callback)()) {
    printf( "Inside func..\n");
    (*callback)();
}

I compiled it with

gcc -c -fpic mylib.c
gcc -shared -o libmylib.so mylib.o

Then I created a test Perl script p.pl:

#! /usr/bin/env perl
use feature qw(say);
use strict;
use warnings;
use FFI::Platypus;

my $ffi = FFI::Platypus->new();
$ffi->lib( 'lib/libmylib.so' );
$ffi->type('()->void' => 'callback_t'); 
$ffi->attach( mylib_func => [ 'callback_t' ] => 'void' );
my $callback = $ffi->closure(
    sub { say "Perl callback()" }
);
mylib_func( $callback );

and running the script seems to work fine:

$ p.pl
Inside func..
Perl callback()

Then I wanted to run it through valgrind to check that everything worked.
First I created a debugging version of perl:

$ perlbrew install perl-5.30.0 --as=5.30.0-D3L -DDEBUGGING \
  -Doptimize=-g3 -Accflags="-DDEBUG_LEAKING_SCALARS"
$ perlbrew use 5.30.0-D3L
$ cpanm FFI::Platypus

Then I ran it with valgrind setting PERL_DESTRUCT_LEVEL=2 as recommended in perlhacktips:

$ PERL_DESTRUCT_LEVEL=2 valgrind --leak-check=yes perl p.pl
==18483== Memcheck, a memory error detector
==18483== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==18483== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==18483== Command: perl p.pl
==18483== 
Inside func..
Perl callback()
==18483== 
==18483== HEAP SUMMARY:
==18483==     in use at exit: 12,514 bytes in 65 blocks
==18483==   total heap usage: 174,637 allocs, 174,572 frees, 22,909,636 bytes allocated
==18483== 
==18483== 1 bytes in 1 blocks are possibly lost in loss record 1 of 18
==18483==    at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483==    by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483==    by 0x7511BF6: XS_FFI__Platypus__TypeParser_create_type_closure (TypeParser.xs:216)
==18483==    by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483==    by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483==    by 0x1A17F9: S_run_body (perl.c:2716)
==18483==    by 0x1A0D77: perl_run (perl.c:2639)
==18483==    by 0x15514D: main (perlmain.c:127)
==18483== 
==18483== 8 bytes in 1 blocks are possibly lost in loss record 2 of 18
==18483==    at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483==    by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483==    by 0x751EF14: ffi_pl_type_new (meta.c:269)
==18483==    by 0x750FC74: XS_FFI__Platypus__TypeParser_create_type_basic (TypeParser.xs:37)
==18483==    by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483==    by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483==    by 0x1A2FD9: Perl_call_sv (perl.c:3043)
==18483==    by 0x1ACEE3: Perl_call_list (perl.c:5084)
==18483==    by 0x181233: S_process_special_blocks (op.c:10471)
==18483==    by 0x180989: Perl_newATTRSUB_x (op.c:10397)
==18483==    by 0x16F560: Perl_utilize (op.c:7592)
==18483==    by 0x221027: Perl_yyparse (perly.y:335)
==18483== 
==18483== 56 bytes in 1 blocks are possibly lost in loss record 8 of 18
==18483==    at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483==    by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483==    by 0x751EF14: ffi_pl_type_new (meta.c:269)
==18483==    by 0x7511C15: XS_FFI__Platypus__TypeParser_create_type_closure (TypeParser.xs:217)
==18483==    by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483==    by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483==    by 0x1A17F9: S_run_body (perl.c:2716)
==18483==    by 0x1A0D77: perl_run (perl.c:2639)
==18483==    by 0x15514D: main (perlmain.c:127)
==18483== 
==18483== 72 (64 direct, 8 indirect) bytes in 1 blocks are definitely lost in loss record 10 of 18
==18483==    at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483==    by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483==    by 0x74FA472: XS_FFI__Platypus__Function__Function_new (Function.xs:53)
==18483==    by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483==    by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483==    by 0x1A17F9: S_run_body (perl.c:2716)
==18483==    by 0x1A0D77: perl_run (perl.c:2639)
==18483==    by 0x15514D: main (perlmain.c:127)
==18483== 
==18483== 104 bytes in 13 blocks are definitely lost in loss record 11 of 18
==18483==    at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483==    by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483==    by 0x751EF14: ffi_pl_type_new (meta.c:269)
==18483==    by 0x750FC74: XS_FFI__Platypus__TypeParser_create_type_basic (TypeParser.xs:37)
==18483==    by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483==    by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483==    by 0x1A2FD9: Perl_call_sv (perl.c:3043)
==18483==    by 0x1ACEE3: Perl_call_list (perl.c:5084)
==18483==    by 0x181233: S_process_special_blocks (op.c:10471)
==18483==    by 0x180989: Perl_newATTRSUB_x (op.c:10397)
==18483==    by 0x16F560: Perl_utilize (op.c:7592)
==18483==    by 0x221027: Perl_yyparse (perly.y:335)
==18483== 
==18483== 112 bytes in 14 blocks are definitely lost in loss record 12 of 18
==18483==    at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==18483==    by 0x2C2582: Perl_safesysmalloc (util.c:155)
==18483==    by 0x751EF14: ffi_pl_type_new (meta.c:269)
==18483==    by 0x7510F10: XS_FFI__Platypus__TypeParser_create_type_pointer (TypeParser.xs:127)
==18483==    by 0x32F0A2: Perl_pp_entersub (pp_hot.c:5237)
==18483==    by 0x2C0C50: Perl_runops_debug (dump.c:2537)
==18483==    by 0x1A2FD9: Perl_call_sv (perl.c:3043)
==18483==    by 0x1ACEE3: Perl_call_list (perl.c:5084)
==18483==    by 0x181233: S_process_special_blocks (op.c:10471)
==18483==    by 0x180989: Perl_newATTRSUB_x (op.c:10397)
==18483==    by 0x16F560: Perl_utilize (op.c:7592)
==18483==    by 0x221027: Perl_yyparse (perly.y:335)
==18483== 
==18483== LEAK SUMMARY:
==18483==    definitely lost: 280 bytes in 28 blocks
==18483==    indirectly lost: 8 bytes in 1 blocks
==18483==      possibly lost: 65 bytes in 3 blocks
==18483==    still reachable: 12,161 bytes in 33 blocks
==18483==         suppressed: 0 bytes in 0 blocks
==18483== Reachable blocks (those to which a pointer was found) are not shown.
==18483== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==18483== 
==18483== For counts of detected and suppressed errors, rerun with: -v
==18483== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0)

So it seems there are some memory leaks here? Any idea what the problem is?
@plicease
Copy link
Member

plicease commented Aug 19, 2019 •
edited
Loading

Attaching a function does lock some data structures into memory, which does appear as a memory leak. There is a trade off between memory usage and the increased speed of using an attached XSUB. If I do not attach the same script and use a function object like this:

my $mylib_func = $ffi->function( mylib_func => [ 'callback_t' ] => 'void' );
my $callback = $ffi->closure(
    sub { say "Perl callback()" }
);
$mylib_func->call( $callback );

then I see only two leaks:

root@f004414c4c0d:/work/examples/leak# PERL_DESTRUCT_LEVEL=2 valgrind --leak-check=yes perl p.pl
==8423== Memcheck, a memory error detector
==8423== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==8423== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==8423== Command: perl p.pl
==8423== 
Inside func..
Perl callback()
==8423== 
==8423== HEAP SUMMARY:
==8423==     in use at exit: 6,142 bytes in 46 blocks
==8423==   total heap usage: 43,911 allocs, 43,865 frees, 6,989,634 bytes allocated
==8423== 
==8423== 112 bytes in 14 blocks are definitely lost in loss record 5 of 11
==8423==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==8423==    by 0x2C48BB: Perl_safesysmalloc (util.c:155)
==8423==    by 0x679AD84: ffi_pl_type_new (meta.c:269)
==8423==    by 0x678A740: XS_FFI__Platypus__TypeParser_create_type_basic (TypeParser.xs:37)
==8423==    by 0x331885: Perl_pp_entersub (pp_hot.c:5237)
==8423==    by 0x2C2F8D: Perl_runops_debug (dump.c:2537)
==8423==    by 0x1A2D44: Perl_call_sv (perl.c:3043)
==8423==    by 0x1ACD39: Perl_call_list (perl.c:5084)
==8423==    by 0x180FC7: S_process_special_blocks (op.c:10471)
==8423==    by 0x180720: Perl_newATTRSUB_x (op.c:10396)
==8423==    by 0x16F284: Perl_utilize (op.c:7592)
==8423==    by 0x221678: Perl_yyparse (perly.y:335)
==8423== 
==8423== 112 bytes in 14 blocks are definitely lost in loss record 6 of 11
==8423==    at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==8423==    by 0x2C48BB: Perl_safesysmalloc (util.c:155)
==8423==    by 0x679AD84: ffi_pl_type_new (meta.c:269)
==8423==    by 0x678BE7A: XS_FFI__Platypus__TypeParser_create_type_pointer (TypeParser.xs:147)
==8423==    by 0x331885: Perl_pp_entersub (pp_hot.c:5237)
==8423==    by 0x2C2F8D: Perl_runops_debug (dump.c:2537)
==8423==    by 0x1A2D44: Perl_call_sv (perl.c:3043)
==8423==    by 0x1ACD39: Perl_call_list (perl.c:5084)
==8423==    by 0x180FC7: S_process_special_blocks (op.c:10471)
==8423==    by 0x180720: Perl_newATTRSUB_x (op.c:10396)
==8423==    by 0x16F284: Perl_utilize (op.c:7592)
==8423==    by 0x221678: Perl_yyparse (perly.y:335)
==8423== 
==8423== LEAK SUMMARY:
==8423==    definitely lost: 224 bytes in 28 blocks
==8423==    indirectly lost: 0 bytes in 0 blocks
==8423==      possibly lost: 0 bytes in 0 blocks
==8423==    still reachable: 5,918 bytes in 18 blocks
==8423==         suppressed: 0 bytes in 0 blocks
==8423== Reachable blocks (those to which a pointer was found) are not shown.
==8423== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==8423== 
==8423== For counts of detected and suppressed errors, rerun with: -v
==8423== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

I think this has to do with the way it caches the basic types and global destruction. This could probably be improved, however.

@plicease
Copy link
Member

I think I might be able to improve the situation with attach too.

plicease added a commit that referenced this issue Aug 19, 2019
@plicease
add a failing test for #174
3805c38
plicease added a commit that referenced this issue Aug 19, 2019
@plicease
add a failing test for #174
8c9cdc1
plicease added a commit that referenced this issue Aug 19, 2019
@plicease
add a failing test for #174
6e4d895
plicease added a commit that referenced this issue Sep 18, 2019
@plicease
add a failing test for #174
b6a7d2f
plicease added a commit that referenced this issue Sep 18, 2019
@plicease
add a failing test for #174
f7f18e8
@plicease plicease added the 🐣Enhancement Things that make it work better label Feb 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
🐣Enhancement Things that make it work better
Milestone
No milestone
Development

No branches or pull requests
2 participants
@plicease @hakonhagland