From: Robert Klemme Date: 2003-11-19T00:32:16+09:00 Subject: Re: Long-running daemon acquiring giant memory footprint "Jason DiCioccio" schrieb im Newsbeitrag news:2147483647.1069146662@pcp04900144pcs.anapol01.md.comcast.net... > --On Tuesday, November 18, 2003 6:17 PM +0900 Robert Klemme > wrote: > > > > > "Jason DiCioccio" schrieb im Newsbeitrag > > news:2147483647.1069109940@[10.0.2.2]... > >> Greetings, > >> > >> --On Tuesday, November 18, 2003 2:12 AM +0900 Robert Klemme > >> wrote: > >> > >> > > >> > "Luke A. Kanies" schrieb im Newsbeitrag > >> > news:Pine.GSO.4.51.0311171048090.2656@pixie... > >> >> On Mon, 17 Nov 2003, Jason DiCioccio wrote: > >> >> > >> >> > Luke, > >> >> > After a while of debugging I found a bunch of objects that were > >> > being > >> >> > created that apparently contain one of the primary keys in one of > > my > >> >> > database tables. The thing is is the query should only return one > >> > result. > >> >> > This particular row is hardly ever referenced either. So now I > > have > >> > found > >> >> > a line in my code that is something like you might have been > > referring > >> > to: > >> >> > > >> >> > nsEntryId = nsEntryId[0][0] > >> >> > > >> >> > That is called quite often, would that cause the object to stay > >> > around? > >> > > >> > Not the old nsEntryId unless nsEntryId[0][0] has a reference to it. > >> > >> I've tried removing this line and it does indeed appear to be the > > problem. > >> There was no reference from nsEntryId[0][0] back to nsEntryId. Is this > > a > >> bug? I don't think it had this problem in ruby 1.6.x, so it appears > >> something changed in the handling of arrays between 1.6 and 1.8. > > > > Hm. Since I don't know the data at hand, I'm not in a position to comment > > more specific. I do wonder however, how this assignment should be > > responsible for growth of mem consumption if nsEntryId[0][0] does not > > backreference nsEntryId. > > > > I'd do a to_yaml of nsEntryId before and after the assignment to be able > > to analyse the object graph. Maybe there's a backreference you > > overlooked. > > > > The other option seems to be a bug in the garbage collector but I'm not > > very inclined in believing this. > > > > If there is an extension involved that could hold on to the memory in ways > > not easy to determine. That's another option that comes to my mind. > > > > Kind regards > > > > robert > > > > > > Robert, > The object in question is just data from ruby dbi object. You could be > right about the back reference, however, the back reference was definitely > not something that I am aware is there. It could be a pointer that ruby > DBI puts into all of its objects? > > Here's a code snippet to illustrate what it contains: > > nsEntryId, = Thread.current[:sql].query( "SELECT NSEntryId FROM NSEntry > WHERE LOWER(Host) = LOWER(?) AND DomainId=?::bigint AND OwnerId=?::bigint > AND UserId=?::bigint #{type} #{ target if target }", domain[0], > Domain::id(domain[1]), Domain::ownerId(domain[1]), @aUser.id ) > > The Sql#query method is below as well: > > def query( sqlRequest, *vars ) > @conn.prepare( sqlRequest ) do |sqlVar| > begin > sqlVar.execute( *vars ) > rescue => detail > if detail.message =~ /server closed the connection unexpectedly/ > log("Connection to DB unexpectedly closed") > $sqlPool.reopen!( @conn ) > end > > Thread.current.raise NoncritError, "An error occurred: #{detail}, > \"#{sqlRequest}\"" > end > > case sqlRequest.word( 0 ) > when /^INSERT/ then return > when /^UPDATE/ then return > when /^DELETE/ then return > end > > result = [] > rcnt = 0 > while (row = sqlVar.fetch) != nil > result.push row.dup > rcnt = rcnt.next > end > > return [nil, rcnt] if result.length == 0 > return [result, rcnt] > end > end > > Any ideas where this reference might be coming from if it exists? Only the row.dup. What type does it return? Is it an Array or some DBI internal class? If it's DBI internal, I'd try row.to_a instead to make sure, that only values are retained and not the row collection. Cheers robert